Pascal Mettes - Hyperbolic and Hyperspherical Visual Understanding

June 30th, 2022. Columbia University
Abstract:
Deep learning for visual recognition thrives on examples but commonly ignores broader available knowledge about hierarchical relations between classes. Me and my team focus on the question of how to integrate hierarchical and inductive knowledge about categorization into deep networks. In this talk, I will dive into three of our recent works that integrate such knowledge through hyperbolic and hyperspherical geometry. As a starting point, I will shortly outline what hyperbolic geometry entails, as well as its potential for representation learning. The first paper introduces a hyperbolic prototype network that is able to embed semantic action hierarchies for video search and recognition [CVPR’20]. The second paper introduces Hyperbolic Image Segmentation, where we provide a tractable formulation for hierarchical pixel-level optimization in hyperbolic space, opening new doors in segmentation [CVPR’22]. For the third paper, we go back to a classical inductive bias, namely maximum separation between classes, and show that contrarily to recent literature, this inductive bias is not an optimization problem but has a closed-form hyperspherical solution. The solution takes the form of one fixed matrix and only requires a single line of code to add to your network, yet directly boosts categorization, long-tailed recognition, and open-set recognition [Preprint’22]. The last part of the talk discusses open research questions and future potential for hyperbolic and hyperspherical learning in computer vision.
Bio:
Pascal Mettes is an assistant professor in computer vision at the University of Amsterdam. He previously received his PhD (2017) under prof. Cees Snoek and was a post doc (2018-2019) at the University of Amsterdam. He was furthermore a visiting researcher at Columbia University (2016) under prof. Shih-Fu Chang and University of Tübingen (2021) under prof. Zeynep Akata. His research focuses on discovering and embedding prior knowledge in deep networks for visual understanding.