Deep Reinforcement Learning for Rough Terrain Traversal

Autonomous driving over rough terrain is challenging. The robot experiences complex dynamics that are difficult to model accurately. Data-driven approaches for controls can be useful as they don't require dynamics to be hand modeled. However, reinforcement learning algorithms often require the robot to collect large amounts of training data through ε-greedy exploration. This process is time consuming and causes extensive damage to the robot. We aim to improve the sample efficiency of reinforcement learning through use of sim2real domain transfer, intelligent exploration, and other methods.

The video above shows rough terrain navigation using model-based reinforcement learning. The robot first trains a dynamics model using collected motion data. It then uses the dynamics model to optimize a trajectory to the goal. Once a trajectory to the goal is found, the robot tracks it using LQR.

Project Collaborators

Aaron Johnson

Soil Sampling Robot

Characterizing the extent of contamination is important for environmental remediation. The current method of contamination characterization involves workers hiking through large environments to collect soil samples for lab analysis. This method is time consuming and results in poor delineation. We aim to automate this process using soil sampling robots to collect more soil samples efficiently.

The video above shows a robot we are working on named "Patrick." Patrick is a tracked robot with soil collection mechanisms. We are currently developing new sensing methods to measure soil contamination in-situ.

Project Collaborators

Nicholas Jones, Valeria Nava, Joe Norby, Catherine Pavlov, Greg Lowry, Aaron Johnson

Planning for Environmental Sampling

Robotic soil sampling can help us understand the distribution of contaminants in an environment. However, the process can be time consuming and costly for larger environments. The robot must travel longer distances and collect more samples to gain a good understanding of the contaminant distribution. We aim to reduce the cost of information by having the robot intelligently select sampling locations that maximize information gain while minimizing operation costs.

The agent in the video above is using a greedy sampling algorithm. The robot estimates the distribution of contaminants in the environment by fitting a gaussian process to its previously collected measurements. It then selects its next sampling location by finding the location at which estimate entropy (information gain) is the highest. The figure on the left shows the true distribution, and the figure on the right shows information gain (estimate entropy) of different locations as the robot continues to sample. The figure at the end of the video shows the estimated contaminant distribution after the robot is done sampling.

We are currently working on non-greedy methods that optimize multiple future sampling locations instead of just the immediate next sampling location.

Project Collaborators

Hannah He, Aaron Johnson

Proprioceptive Contact Localization

Read "Contact Localization using Velocity Constraints" in publications section.

Project Collaborators

Ankit Bhatia, Matt Mason, Aaron Johnson

Isla

Legged robots have the ability to step over obstacles, making them well suited to rough terrain environments. However, they often have limited payload and travel range since their locomotion is less efficient than wheeled locomotion. Isla aims to capture the advantages of both legged and wheeled locomotion. It is able to and step over small obstacles. It can also roll on its body for efficient locomotion.

Project Collaborators

Michelle Coyle, Craig Stephen, Shankar Srinivasan, Bryan Zhao

Cloud-supported coverage control for multi-agent surveillence missions

Read "Cloud-supported coverage control for persistent surveillance missions" and "Coverage control with anytime updates for persistent surveillance missions" in publications section.

Project Collaborators

Jeff Peters, Amit Surana, Francesco Bullo

Advance Imaging Drone

More Information

Project Collaborators

Alan Cao, Jake Carrade, Landon Peik, Viswahindu Rao