A Benchmark Comparison of Monocular Visual-Inertial Odometry Algorithms for Flying Robots

Flying robots require a combination of accuracy and low latency in their state estimation in order to achieve stable and robust flight. However, due to the power and payload constraints of aerial platforms, state estimation algorithms must provide these qualities under the computational constraints of embedded hardware. Cameras and inertial measurement units (IMUs) satisfy these power and payload constraints, so visual-inertial odometry (VIO) algorithms are popular choices for state estimation in these scenarios, in addition to their ability to operate without external localization from motion capture or global positioning systems. It is not clear from existing results in the literature, however, which VIO algorithms perform well under the accuracy, latency, and computational constraints of a flying robot with onboard state estimation. This paper evaluates an array of publicly-available VIO pipelines (MSCKF, OKVIS, ROVIO, VINS-Mono, SVO+MSF, and SVO+GTSAM) on different hardware configurations, including several single-board computer systems that are typically found on flying robots. The evaluation considers the pose estimation accuracy, per-frame processing time, and CPU and memory load while processing the EuRoC datasets, which contain six degree of freedom (6DoF) trajectories typical of flying robots. We present our complete results as a benchmark for the research community.
Reference:
J. Delmerico, D. Scaramuzza
A Benchmark Comparison of Monocular Visual-Inertial Odometry Algorithms for Flying Robots
IEEE International Conference on Robotics and Automation (ICRA), 2018
PDF: rpg.ifi.uzh.ch/docs/ICRA18_Del...
Our research page on visual-inertial odometry:
rpg.ifi.uzh.ch/research_vo.html
Affiliation:
Robotics and Perception Group,
Dep. of Neuroinformatics, ETH Zurich & University of Zurich ,
Dep. of Informatics, University of Zurich,
rpg.ifi.uzh.ch/

[@ThelleKristensen]

Nice overview - thanks for sharing!

[@zakariachekakta3305]

great, it's very helpful thank you

[@tomaskrejci5319]

Impressive work! When will the MSCKF_mono be public?

[@zhoueric7914]

thx for the nice sharing ! vins is the best algor!

[@cloudmodyt4502]

hmm. How do you think OpenVINS, a fork of VINS-Mono, would compare to these other VIO systems?

[@sk000rp]

What about ORB SLAM or DSO?

[@MrSersmax]

Could be that it was to new for the Paper to be included? And i am not sure if DSO deserves the definition "Visual-Inertial" as i don't know what defines a VO as a VIO

[@faridalijani1578]

Odometry is using a (any) sensor to determine how much distance has been traversed, so visual odometry (VO) is just clarification in which particular sensor for odometry is vision (a camera, typically).

[@francescovultaggio781]

citing the complete paper "We do not consider non-inertial visual simultaneous localization and mapping (SLAM) systems, for example ORBSLAM and LSD-SLAM. While these methods
could potentially also be used for flying robot state estimation, we focus this benchmark on visual-inertial methods."

[@ruanjiayang]

Howto get Ground truth of trajectory?

[@phase5216]

It depends on each dataset. But as far as I know, ground truths for this kind of problem are collected by high-end IMUs (NovAtel, OXTS, ...). The ready-to-use results of these sensors are put out after some algorithm to reduce noise and be considered with The Earth's magnetic field, ... However, I don't know, metrically, how those results are proved to be correct and how reliable they are. Anyone knows? 😅

[@ruanjiayang]

Even the highest-end IMUs will drift and accumulate. In a small in-door case, one of the method I can conceive is to add a QR code on the robot, and use a fixed camera to capture its position. In outdoor case, perhaps use a high-precision GPS-RTK. The hardest case is large-scale in-door case.@216