Real-time Visual-Inertial Odometry for Event Cameras using Keyframe-based Nonlinear Optimization

Event cameras are bio-inspired vision sensors that output pixel-level brightness changes instead of standard intensity frames. They offer significant advantages over standard cameras, namely a very high dynamic range, no motion blur, and a latency in the order of microseconds. We propose a novel, accurate tightly-coupled visual-inertial odometry pipeline for such cameras that leverages their outstanding properties to estimate the camera ego-motion in challenging conditions, such as high-speed motion or high dynamic range scenes. The method tracks a set of features (extracted on the image plane) through time. To achieve that, we consider events in overlapping spatio-temporal windows and align them using the current camera motion and scene structure, yielding motion-compensated event frames. We then combine these feature tracks in a keyframe-based, visual-inertial odometry algorithm based on nonlinear optimization to estimate the camera’s 6-DOF pose, velocity, and IMU biases. The proposed method is evaluated quantitatively on the public Event Camera Dataset and significantly outperforms the state-of-the-art, while being computationally much more efficient: our pipeline can run much faster than real-time on a laptop and even on a smartphone processor. Furthermore, we demonstrate qualitatively the accuracy and robustness of our pipeline on a large-scale dataset, and an extremely high-speed dataset recorded by spinning an event camera on a leash at 850 deg/s.
Reference:
Rebecq, Horstschaefer, Scaramuzza
Real-time Visual-Inertial Odometry for Event Cameras using Keyframe-based Nonlinear Optimization
PDF: rpg.ifi.uzh.ch/docs/BMVC17_Reb...
Our research page on event based vision:
rpg.ifi.uzh.ch/research_dvs.html
For event-camera datasets and event camera simulator, see here: rpg.ifi.uzh.ch/davis_data.html
Other resources on event cameras (publications, software, drivers, where to buy, etc.):
github.com/uzh-rpg/event-base...
Robotics and Perception Group, University of Zurich, 2017
rpg.ifi.uzh.ch/

[@AlexisPaques]

This is the future of VO for robotics!

[@hayyandaoud]

Very impressive work! Good job. I am really interested in working on event-based cameras. I also recommended buying it in two research labs here in Malaysia. We were hoping to have you in Kuala Lumpur after ICRA in Singapore.

[@prandtlmayer]

Ok, just wow

[@user-wf8pp3ee9q]

WOW! I want to know where can I purchase the DVS.

[@user-ll8mt2me8i]

Can you integrated this in ROS?