Young Min Kim, associate professor at Seoul National University and leader of the 3D Vision Lab SNU, gave a presentation titled "Modular Superpixel Tokenization in Vision Transformers" on December 5th 2024 as part of the Visual Intelligence Online Seminar series.
Abstract:
Foundational models contain impressive emergent knowledge demonstrated by seminal success in various multimodal generative tasks. Nonetheless, these models are known to fail in basic spatial perception tasks despite requiring a significant amount of energy and training data. In this talk, I would like to share various 3D representations that encapsulate necessary 3D knowledge. Deep learning architecture on 3D data often utilizes regular structure but may take advantage of the sparsity of surface geometry. However, the information can still be extensive, and we propose using line structure to convey compact structural information. Additionally, functional representations, combined with latent representations, may provide compressed embedding that might be useful for various downstream applications. Finally, I will conclude with future directions incorporating natural dynamics to blend physical interpretation into the representations.