Reinforcement learning for the next generation of autonomous aircraft

Credit: travelarium.ph/shutterstock.com

The U.S. Navy is pursuing a solution capable of adapting to these difficult conditions and is turning to Texas A&M researchers to develop the next generation of fully autonomous vertical takeoff and landing (VTOL) aircraft. By combining an optimal aircraft design with a robust machine learning algorithm, the researchers are proposing a new approach to automated aircraft ship landing at rough seas.

“When a helicopter pilot tries to land on a ship deck, they don’t actually look at the moving deck,” said Dr. Moble Benedict, associate professor in the Department of Aerospace Engineering at Texas A&M and the project’s principal investigator (PI). “If they look at the moving deck, it will disorient the pilot, so they are trained to look at a specialized equipment on the ship called the horizon bar, which is a green, lighted, gyro-stabilized strip that provides the pilot an artificial horizon.”

Recent studies focused on tracking the ship’s deck rather than the horizon bar by using cameras, GPS and lidar to track the moving ship and adjust the aircraft to match its motion. Instead, Benedict and Dr. Dileep Kalathil, assistant professor in the Department of Electrical and Computer Engineering and co-PI on the project, are automating the landing process by mimicking a pilot’s behavior while tracking the horizon bar.

“Reinforcement learning is a class of machine learning for developing the control algorithm for autonomous systems,” said Kalathil. “We are developing a reinforcement learning control algorithm so precise that even if a vehicle is changing course or is in the presence of heavy winds, it can still track the horizon bar.”

Benedict and Kalathil have proven success in using reinforcement learning to track and safely land an unmanned aerial system (UAS) in various conditions, including moderate horizontal winds, foggy visibility and changes in course and speed. Now, they’re merging their respective disciplines of aerospace engineering and electrical and computer engineering to build on these advancements.

“My focus will be on designing a new generation of UASs for robust ship-based operation and high efficiency while understanding their flight dynamics,” said Benedict. “And Dr. Kalathil’s focus is on using reinforcement learning to make this autonomous process more robust for highly uncertain environments.”

Ultimately, the Navy is interested in three elements: an aircraft that is runway independent, meaning it can take off and land vertically; cruise efficiency so the aircraft can fly for long durations at a time; and lastly, the ability to land on moving ship decks safely and successfully.

Benedict is applying his expertise in rotorcraft to designing VTOL aircraft concepts that are gust-tolerant and efficient, which may include foldable wings when transitioning from vertical flight to fixed-wing cruise. Using simulations, wind tunnel testing and flight tests, he’ll analyze the performance and dynamics of these concepts to build a subscale model that will complement the control systems developed by Kalathil.