VIO-DualProNet: Visual-inertial odometry with learning based process noise covariance

Dan Solodar, Itzik Klein

Research output: Contribution to journalArticlepeer-review


Visual-inertial odometry (VIO) is a vital technique used in robotics, augmented reality, and autonomous vehicles. It combines visual and inertial measurements to accurately estimate position and orientation. Existing VIO methods assume a fixed noise covariance for the inertial uncertainty. However, accurately determining in real-time the noise variance of the inertial sensors presents a significant challenge as the uncertainty changes throughout the operation leading to suboptimal performance and reduced accuracy. To circumvent this, we propose VIO-DualProNet, a novel approach that utilizes deep learning methods to dynamically estimate the inertial measurement unit (IMU) noise uncertainty in real-time. By designing and training a deep neural network to predict inertial noise uncertainty using only inertial sensor measurements, and integrating it into the VINS-Mono algorithm, we demonstrate a substantial improvement in accuracy and robustness. Our system outperformed constant covariance methods in 9 out of 11 test sequences, with an average improvement of 25% compared to the baseline and a 12.5% improvement over the best constant covariance combination.

Original languageEnglish
Article number108466
JournalEngineering Applications of Artificial Intelligence
StatePublished - Jul 2024

Bibliographical note

Publisher Copyright:
© 2024 Elsevier Ltd


  • Autonomous navigation
  • Deep learning
  • Inertial measurement unit
  • Sensor fusion
  • Simultaneous localization and mapping
  • Visual-inertial odometry

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Artificial Intelligence
  • Electrical and Electronic Engineering


Dive into the research topics of 'VIO-DualProNet: Visual-inertial odometry with learning based process noise covariance'. Together they form a unique fingerprint.

Cite this