Electrical vertical take-off and landing (eVTOL) vehicles are becoming promising for ondemand air transportation in Urban Air Mobility (UAM). However, successfully bringing such vehicles and airspace operations to fruition will require introducing orders-of-magnitude more aircraft to a given airspace volume. Although there are existing solutions for communication technology, onboard computing capability, and sensor technology, the computation guidance algorithm to enable safe, efficient, and scalable flight operations for dense self-organizing air traffic still remains an open question. In this paper, a message-based decentralized computational guidance algorithm is proposed and analyzed for multiple cooperative aircraft by formulating this problem using Multiagent Markov Decision Process and solving it by Monte Carlo Tree Search algorithm. A novel coordination strategy is introduced by using the logit level-k model in behavioral game theory. To achieve higher scalability, we introduce the airspace sector concept into the UAM environment by dividing the airspace into sectors, so that each aircraft only needs to coordinate with aircraft in the same sector. At each decision step, all of the aircraft will run the proposed computational guidance algorithm onboard, which can guide all the aircraft to their respective destinations while avoiding potential conflicts among them. For validation and demonstration, a free-flight airspace simulator that incorporates environment uncertainty is built in an OpenAI Gym environment. Numerical experiment results over several case studies including the roundabout test problem show that the proposed computational guidance algorithm has promising performance even with the high-density air traffic case.
PDF Video Code

Experience replay is widely used in deep reinforcement learning algorithms and allows agents to remember and learn from experiences from the past. In an effort to learn more efficiently, researchers proposed prioritized experience replay (PER) which samples important transitions more frequently. In this paper, we propose Prioritized Sequence Experience Replay (PSER) a framework for prioritizing sequences of experience in an attempt to both learn more efficiently and to obtain better performance. We compare performance of uniform, PER and PSER sampling techniques in DQN on the Atari 2600 benchmark and show DQN with PSER substantially outperforms PER and uniform sampling.
PDF Code

In urban air mobility (UAM), flying with electrical vertical takeoff and landing (eVTOL) aircraft will bring fundamental changes to city infrastructures and daily commutes. In order to enable safe and efficient autonomous on-demand free flight operations for the eVTOL aircraft in UAM, a centralized computational guidance algorithm is proposed and analyzed for multi cooperative aircraft. The approach proposed in this paper is to formulate this problem as a Markov Decision Process (MDP) and solve it using an online algorithm Monte Carlo Tree Search (MCTS). A coordination mechanism is designed to manage multiple cooperative aircraft. By generating real-time actions for all the cooperative aircraft to follow, the algorithm can guide all the aircraft to their respective destinations while avoiding potential conflicts between them. For the sake of illustration, a free flight airspace simulator is created to test the performance of this algorithm. Results show that this algorithm can help all the aircraft reach their trip destinations while only having 0.2% conflicts during the flights.
PDF Video Code

Vertical takeoff and landing (VTOL) aircraft for personal air transportation or on-demand air taxi will bring fundamental changes to city infrastructures and daily commutes. NASA, Uber, and Airbus have been exploring the exciting concept of Urban Air Mobility (UAM). In order to enable safe and efficient autonomous on-demand free flight operations in this UAM concept, a computational guidance algorithm was designed and analyzed with collision avoidance capability. The approach is to formulate this problem as a Markov Decision Process and solve it using an online algorithm called Monte Carlo Tree Search. For the sake of illustration, a simplified numerical experiment was created to test the performance of this algorithm. Results show that this algorithm can help aircraft quickly reach the trip destination and avoid conflicts with other aircraft.
PDF Video Code