Newton-PnP: Real-time Visual Navigation for Autonomous Toy-Drones

Ibrahim Jubran; Fares Fares; Yuval Alfassi; Firas Ayoub; Dan Feldman

doi:10.1109/IROS47612.2022.9981922

Newton-PnP: Real-time Visual Navigation for Autonomous Toy-Drones

Ibrahim Jubran, Fares Fares, Yuval Alfassi, Firas Ayoub, Dan Feldman

Department of Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

The Perspective-n-Point problem aims to estimate the relative pose between a calibrated monocular camera and a known 3D model, by aligning pairs of 2D captured image points to their corresponding 3D points in the model. We suggest an algorithm that runs on weak IoT devices in real-time but still provides provable theoretical guarantees for both running time and correctness. Existing solvers provide only one of these requirements. Our main motivation was to turn the popular DJI's Tello Drone (<90gr, <100) into an autonomous drone that navigates in an indoor environment with no external human/laptop/sensor, by simply attaching a Raspberry PI Zero (<9gr, <25) to it. This tiny micro-processor takes as input a real-time video from a tiny RGB camera, and runs our PnP solver on-board. Extensive experimental results, open source code, and a demonstration video are included.

Original language	English
Title of host publication	IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2022
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	13363-13370
Number of pages	8
ISBN (Electronic)	9781665479271
DOIs	https://doi.org/10.1109/IROS47612.2022.9981922
State	Published - 2022
Event	2022 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2022 - Kyoto, Japan Duration: 23 Oct 2022 → 27 Oct 2022

Publication series

Name	IEEE International Conference on Intelligent Robots and Systems
Volume	2022-October
ISSN (Print)	2153-0858
ISSN (Electronic)	2153-0866

Conference

Conference	2022 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2022
Country/Territory	Japan
City	Kyoto
Period	23/10/22 → 27/10/22

Bibliographical note

Publisher Copyright:
© 2022 IEEE.

ASJC Scopus subject areas

Control and Systems Engineering
Software
Computer Vision and Pattern Recognition
Computer Science Applications

Access to Document

10.1109/IROS47612.2022.9981922

Cite this

Jubran, I., Fares, F., Alfassi, Y., Ayoub, F., & Feldman, D. (2022). Newton-PnP: Real-time Visual Navigation for Autonomous Toy-Drones. In IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2022 (pp. 13363-13370). (IEEE International Conference on Intelligent Robots and Systems; Vol. 2022-October). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/IROS47612.2022.9981922

@inproceedings{fd93b93b63b24b61a5506c717db76930,

title = "Newton-PnP: Real-time Visual Navigation for Autonomous Toy-Drones",

abstract = "The Perspective-n-Point problem aims to estimate the relative pose between a calibrated monocular camera and a known 3D model, by aligning pairs of 2D captured image points to their corresponding 3D points in the model. We suggest an algorithm that runs on weak IoT devices in real-time but still provides provable theoretical guarantees for both running time and correctness. Existing solvers provide only one of these requirements. Our main motivation was to turn the popular DJI's Tello Drone (<90gr, <100) into an autonomous drone that navigates in an indoor environment with no external human/laptop/sensor, by simply attaching a Raspberry PI Zero (<9gr, <25) to it. This tiny micro-processor takes as input a real-time video from a tiny RGB camera, and runs our PnP solver on-board. Extensive experimental results, open source code, and a demonstration video are included.",

author = "Ibrahim Jubran and Fares Fares and Yuval Alfassi and Firas Ayoub and Dan Feldman",

note = "Publisher Copyright: {\textcopyright} 2022 IEEE.; 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2022 ; Conference date: 23-10-2022 Through 27-10-2022",

year = "2022",

doi = "10.1109/IROS47612.2022.9981922",

language = "English",

series = "IEEE International Conference on Intelligent Robots and Systems",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "13363--13370",

booktitle = "IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2022",

address = "United States",

}

Jubran, I, Fares, F, Alfassi, Y, Ayoub, F & Feldman, D 2022, Newton-PnP: Real-time Visual Navigation for Autonomous Toy-Drones. in IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2022. IEEE International Conference on Intelligent Robots and Systems, vol. 2022-October, Institute of Electrical and Electronics Engineers Inc., pp. 13363-13370, 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2022, Kyoto, Japan, 23/10/22. https://doi.org/10.1109/IROS47612.2022.9981922

Newton-PnP: Real-time Visual Navigation for Autonomous Toy-Drones. / Jubran, Ibrahim; Fares, Fares; Alfassi, Yuval et al.
IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2022. Institute of Electrical and Electronics Engineers Inc., 2022. p. 13363-13370 (IEEE International Conference on Intelligent Robots and Systems; Vol. 2022-October).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Newton-PnP

T2 - 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2022

AU - Jubran, Ibrahim

AU - Fares, Fares

AU - Alfassi, Yuval

AU - Ayoub, Firas

AU - Feldman, Dan

PY - 2022

Y1 - 2022

N2 - The Perspective-n-Point problem aims to estimate the relative pose between a calibrated monocular camera and a known 3D model, by aligning pairs of 2D captured image points to their corresponding 3D points in the model. We suggest an algorithm that runs on weak IoT devices in real-time but still provides provable theoretical guarantees for both running time and correctness. Existing solvers provide only one of these requirements. Our main motivation was to turn the popular DJI's Tello Drone (<90gr, <100) into an autonomous drone that navigates in an indoor environment with no external human/laptop/sensor, by simply attaching a Raspberry PI Zero (<9gr, <25) to it. This tiny micro-processor takes as input a real-time video from a tiny RGB camera, and runs our PnP solver on-board. Extensive experimental results, open source code, and a demonstration video are included.

AB - The Perspective-n-Point problem aims to estimate the relative pose between a calibrated monocular camera and a known 3D model, by aligning pairs of 2D captured image points to their corresponding 3D points in the model. We suggest an algorithm that runs on weak IoT devices in real-time but still provides provable theoretical guarantees for both running time and correctness. Existing solvers provide only one of these requirements. Our main motivation was to turn the popular DJI's Tello Drone (<90gr, <100) into an autonomous drone that navigates in an indoor environment with no external human/laptop/sensor, by simply attaching a Raspberry PI Zero (<9gr, <25) to it. This tiny micro-processor takes as input a real-time video from a tiny RGB camera, and runs our PnP solver on-board. Extensive experimental results, open source code, and a demonstration video are included.

UR - http://www.scopus.com/inward/record.url?scp=85146307319&partnerID=8YFLogxK

U2 - 10.1109/IROS47612.2022.9981922

DO - 10.1109/IROS47612.2022.9981922

M3 - Conference contribution

AN - SCOPUS:85146307319

T3 - IEEE International Conference on Intelligent Robots and Systems

SP - 13363

EP - 13370

BT - IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2022

PB - Institute of Electrical and Electronics Engineers Inc.

Y2 - 23 October 2022 through 27 October 2022

ER -

Jubran I, Fares F, Alfassi Y, Ayoub F, Feldman D. Newton-PnP: Real-time Visual Navigation for Autonomous Toy-Drones. In IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2022. Institute of Electrical and Electronics Engineers Inc. 2022. p. 13363-13370. (IEEE International Conference on Intelligent Robots and Systems). doi: 10.1109/IROS47612.2022.9981922

Newton-PnP: Real-time Visual Navigation for Autonomous Toy-Drones

Abstract

Publication series

Conference

Bibliographical note

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this