Publications

2024

1. HDPlanner: Advancing Autonomous Deployments in Unknown Environments through Hierarchical Decision Networks

   Jingsong Liang, Yuhong Cao, Yixiao Ma, Hanqi Zhao, and Guillaume Adrien Sartoretti

   In submission to IEEE Robotic and Automation Letters (RA-L), 2024

   Abs Bib PDF Code

   In this paper, we introduce HDPlanner, a deep reinforcement learning (DRL) based framework designed to tackle two core and challenging tasks for mobile robots: autonomous exploration and navigation, where the robot must optimize its trajectory adaptively to achieve the task objective through continuous interactions in unknown environments. Specifically, HDPlanner relies on novel hierarchical attention networks to empower the robot to reason about its belief across multiple spatial scales and sequence collaborative decisions, where our networks decompose long-term objectives into short-term informative task assignments and informative path plannings. We further propose a contrastive learning-based joint optimization to enhance the robustness of HDPlanner. We empirically demonstrate that HDPlanner significantly outperforms state-of-the-art conventional and learning-based baselines on an extensive set of simulations, including hundreds of test maps and large-scale, complex Gazebo environments. Notably, HDPlanner achieves real-time planning with travel distances reduced by up to 35.7% compared to exploration benchmarks and by up to 16.5% than navigation benchmarks. Furthermore, we validate our approach on hardware, where it generates high-quality, adaptive trajectories in both indoor and outdoor environments, highlighting its real-world applicability without additional training.

   @inproceedings{liang2024hdplanner,
     title = {HDPlanner: Advancing Autonomous Deployments in Unknown Environments through Hierarchical Decision Networks},
     author = {Liang, Jingsong and Cao, Yuhong and Ma, Yixiao and Zhao, Hanqi and Sartoretti, Guillaume Adrien},
     booktitle = {submission to IEEE Robotic and Automation Letters (RA-L)},
     year = {2024},
     organization = {IEEE},
   }

2. 

   Privileged Reinforcement and Communication Learning for Distributed, Bandwidth-limited Multi-Robot Exploration

   Yixiao Ma, Jingsong Liang, Yuhong Cao, Derek Ming Siang Tan, and Guillaume Adrien Sartoretti

   In International Symposium on Distributed Autonomous Robotic Systems (DARS), 2024

   Abs Bib PDF Code

   Communication bandwidth is an important consideration in multi-robot exploration, where information exchange among robots is critical. While existing methods typically aim to reduce communication throughput, they either require significant computation or significantly compromise exploration efficiency. In this work, we propose a deep reinforcement learning framework based on communication and privileged reinforcement learning to achieve a significant reduction in bandwidth consumption, while keeping exploration efficiency very high. Specifically, our approach allows robots to learn to embed the most salient information from their individual belief (partial map) over the environment into fixed-sized messages. Robots then reason about their own belief as well as received messages to distributedly explore the environment while avoiding redundant work. In doing so, we employ privileged learning and learned attention mechanisms to endow the critic (i.e., teacher) network with ground truth map knowledge to effectively guide the policy (i.e., student) network during training. Compared to state-of-the-art baselines, our model allows the team to reduce communication by up to two orders of magnitude, while only sacrificing a marginal \textbf2.4% in total travel distance, paving the way for efficient, distributed multi-robot exploration in bandwidth-limited scenarios.

   @inproceedings{ma2024context,
     title = {Privileged Reinforcement and Communication Learning for Distributed, Bandwidth-limited Multi-Robot Exploration},
     author = {Ma, Yixiao and Liang, Jingsong and Cao, Yuhong and Tan, Derek Ming Siang and Sartoretti, Guillaume Adrien},
     booktitle = {International Symposium on Distributed Autonomous Robotic Systems (DARS)},
     year = {2024},
     organization = {Springer},
   }

3. IR2: Implicit Rendezvous for Robotic Exploration Teams under Sparse Intermittent Connectivity

   Derek Ming Siang Tan, Yixiao Ma, Jingsong Liang, Yi Cheng Chng, Yuhong Cao, and Guillaume Adrien Sartoretti

   In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2024

   Abs Bib PDF Code

   @inproceedings{derek2024context,
     title = {IR2: Implicit Rendezvous for Robotic Exploration Teams under Sparse Intermittent Connectivity},
     author = {Tan, Derek Ming Siang and Ma, Yixiao and Liang, Jingsong and Chng, Yi Cheng and Cao, Yuhong and Sartoretti, Guillaume Adrien},
     booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
     year = {2024},
     organization = {IEEE},
   }

2023

1. 

   Context-Aware Deep Reinforcement Learning for Autonomous Robotic Navigation in Unknown Area

   Jingsong Liang, Zhichen Wang, Yuhong Cao, Jimmy Chiun, Mengqi Zhang, and Guillaume Adrien Sartoretti

   In Conference on Robot Learning (CoRL), 2023

   Abs Bib PDF Code

   Mapless navigation refers to a challenging task where a mobile robot must rapidly navigate to a predefined destination using its partial knowledge of the environment, which is updated online along the way, instead of a prior map of the environment. Inspired by the recent developments in deep reinforcement learning (DRL), we propose a learning-based framework for mapless navigation, which employs a context-aware policy network to achieve efficient decision-making (i.e., maximize the likelihood of finding the shortest route towards the target destination), especially in complex and large-scale environments. Specifically, our robot learns to form a context of its belief over the entire known area, which it uses to reason about long-term efficiency and sequence show-term movements. Additionally, we propose a graph rarefaction algorithm to enable more efficient decision-making in large-scale applications. We empirically demonstrate that our approach reduces average travel time by up to 61.4% and average planning time by up to 88.2% compared to benchmark planners (D*lite and BIT) on hundreds of test scenarios. We also validate our approach both in high-fidelity Gazebo simulations as well as on hardware, highlighting its promising applicability in the real world without further training/tuning.

   @inproceedings{liang2023context,
     title = {Context-Aware Deep Reinforcement Learning for Autonomous Robotic Navigation in Unknown Area},
     author = {Liang, Jingsong and Wang, Zhichen and Cao, Yuhong and Chiun, Jimmy and Zhang, Mengqi and Sartoretti, Guillaume Adrien},
     booktitle = {Conference on Robot Learning (CoRL)},
     pages = {1425--1436},
     year = {2023},
     organization = {PMLR},
   }