Deep Visual SLAM Frontends: SuperPoint, SuperGlue, and SuperMaps (

Abstract: Mixed Reality and Robotics require robust Simultaneous Localization and Mapping (SLAM) capabilities, and many researchers believe that deep learning is the solution. This talk will discuss the frontend/backend distinction in Visual SLAM systems as well as discuss my team’s work on deep learning-based frontends for visual SLAM. The talk will focus on the applications of Convolutional and Graph Neural Networks for visual localization and SLAM, as well as novel self-supervised ways of training such networks. This talk will cover our work on SuperPoint and SuperGlue, and conclude with a discussion of future directions for building robust SuperMaps using deep learning concepts.
Presentation Slides: tom.ai/present...
Presenter: Tomasz Malisiewicz, Ph.D.
Bio: Tomasz Malisiewicz is a Principal Engineer at Magic Leap, Inc. His research lies at the intersection of Deep Learning and SLAM. Previously he was a co-founder of VISION.AI, LLC, an object detection startup and a Postdoctoral Fellow at MIT CSAIL, working with Antonio Torralba. He received his Ph.D. in Robotics from Carnegie Mellon University in 2011, supervised by Alyosha Efros. During his Ph.D., he was a recipient of the NSF Graduate Research Fellowship, spent two summers as an intern in Google Research, as well as one semester as a visiting student at École Normale Supérieure in Paris.
Researcher homepages:
Tomasz Malisiewicz tom.ai
Daniel DeTone danieldetone.com
Paul-Edouard Sarlin psarlin.com
#computervision #deeplearning #slam #localization

[@TomaszMalisiewicz]

If you want to watch the 5-minute video about our "SuperGlue: Learning Feature Matching with Graph Neural Networks" CVPR2020 paper, go here: th-cam.com/video/BNaIGI4VncM/w-d-xo.html

[@davidmorillacabello1106]

Thank you for the talk! It was really clear, easy to understand and inspiring. It's amazing to see how self-supervised training leverages deep learning over human intuition.

[@finkelmann]

Great presentation - super interesting, and very eloquent and clear.

[@martinschulze5399]

Im rather new to the field of neural SLAM (more experience with spiking neural networks etc.) and have given out a master thesis on visual SLAM on graph NNs to get me on track ASAP and your work is one of the top ones on my interest list. Good job and thanks ;)

[@TomaszMalisiewicz]

Hi Martin, Thanks for the kind words! I'm glad that by making my team's papers and this talk publicly available, others can learn about all of this exciting work.

[@darrenruben2981]

Keep making pmhs proud

[@amortalbeing]

Loved it. Thanks a lot. you are really very good at presenting this.

[@James_Albert]

Hi @Tom
Thank you for the fantastic presntation!
I want to play with your code, but access was denied

[@jandresjn]

Hi, I'm now a fan of your research. I plan to carry out my master's research project on how to optimize these "feature matching" processes or stages where I can apply deep-learning to improve the Slam technique. I would like to know what are the main problems you have had.

[@TomaszMalisiewicz]

It's been pretty difficult to get quantities like camera pose (Rotation and translation) to be the output of a deep neural network. It's pretty easy to turn keypoint identification and matching into a deep network. There are lots of interesting problems at the intersection of Deep Learning and SLAM. Good luck with your research!

[@jandresjn]

@@TomaszMalisiewicz Thank you for the answer. Actually, i am going to focus on the pose problem with a deep network, I hope to be able to contribute something in the future, since I am at the beginning of the master, but I already had experience with SFM in the undergraduate. Regards!

[@rashikshrestha1847]

Thank you for the talk Tomasz Malisiewicz.
Can we train SuperGlue to match ORB features ? How much effective might be the result ?

[@TomaszMalisiewicz]

Yes, you can train a SuperGlue variant to work with whatever features you want. We evaluated two variants in our SuperGlue paper -- one network was trained to work with SuperPoint features and the other network was trained with SIFT features. You can make SuperGlue work with just about any descriptor you want, but keep in mind that ORB was designed to be lean and fast while deep networks are much slower and bigger.

[@antoinenomad1566]

Thank you very much

[@xingxingzuo7028]

Really fancy!

[@VeerDaVlog]

Super Good

[@roma-pk4vf]

很好