Deep Reinforcement Learning for Spatial Motion Planning in 3D Urban Environments

Abstract—In this paper, we present spatial motion planner in
3D environments based on Deep Reinforcement Learning
(DRL) algorithms. We tackle 3D motion planning problem by
using Deep Reinforcement Learning (DRL) approach, which
learns agent’s and environment constraints. Spatial analysis
focuses on visibility analysis in 3D setting an optimal motion
primitive considering agent’s dynamic model based on fast and
exact visibility analysis for each motion primitives. Based on
optimized reward function, which consist of generated 3D
visibility analysis and obstacle avoidance trajectories, we
introduce DRL formulation, which learns the value function of
the planner and generates an optimal spatial visibility
trajectory. We demonstrate our planner in simulations for
Unmanned Aerial Vehicles (UAV) in 3D urban environments.
Our spatial analysis is based on a fast and exact spatial
visibility analysis of the 3D visibility problem from a viewpoint
in 3D urban environments. We present DRL architecture
generating the most visible trajectory in a known 3D urban
environment model, as time-optimal one with obstacle
avoidance capability.