# Joint Vision-Based Navigation, Control and Obstacle Avoidance for UAVs in Dynamic Environments

## TL;DR

This paper presents a real-time, vision-based navigation and obstacle avoidance system for UAVs that integrates optimal control with ellipsoidal obstacle modeling, validated through extensive simulations and open-source code.

## Contribution

It introduces a novel optimal control framework combining vision-based navigation with ellipsoidal obstacle avoidance for UAVs, optimized for real-time performance.

## Key findings

- Effective obstacle avoidance in dynamic environments
- Robust navigation with safety margins
- Real-time trajectory optimization in simulations

## Abstract

This work addresses the problem of coupling vision-based navigation systems for Unmanned Aerial Vehicles (UAVs) with robust obstacle avoidance capabilities. The former problem is solved by maximizing the visibility of the points of interest, while the latter is modeled by means of ellipsoidal repulsive areas. The whole problem is transcribed into an Optimal Control Problem (OCP), and solved in a few milliseconds by leveraging state-of-the-art numerical optimization. The resulting trajectories are well suited for reaching the specified goal location while avoiding obstacles with a safety margin and minimizing the probability of losing the route with the target of interest. Combining this technique with a proper ellipsoid shaping (i.e., by augmenting the shape proportionally with the obstacle velocity or with the obstacle detection uncertainties) results in a robust obstacle avoidance behavior. We validate our approach within extensive simulated experiments that show effective capabilities to satisfy all the constraints even in challenging conditions. We release with this paper the open source implementation.