ฝัง
- เผยแพร่เมื่อ 12 พ.ย. 2024
- BeyondScene: Higher-Resolution Human-Scene Generation With Pretrained Diffusion
Speaker Bio: Gwanghyun (Bradley) Kim is a Ph.D. candidate in Electrical and Computer Engineering (ECE) at Seoul National University (SNU), under the supervision of Prof. Se Young Chun. He completed his M.S. degree at KAIST, where he was advised by Prof. Jong Chul Ye.This year, he is interning at NVIDIA Research, working with Umar Iqbal, Xueting Li, and Ye Yuan. Last year, he interned at Google Research with Alonso Martinez, Krishna Somandepalli, and Yu-Chuan Su.His research focuses on artificial intelligence (AI), particularly in computer vision (CV) and its intersection with Generative AI. His work emphasizes multimodal, high-dimensional, and human-centric generative AI. Gwanghyun has received the Qualcomm Innovation Fellowship and the Yulchon AI Star Scholarship.
Title: BeyondScene: Higher-Resolution Human-Scene Generation With Pretrained Diffusion
Abstract: We propose BeyondScene, a novel framework that overcomes prior limitations, generating exquisite higherresolution (over 8K) human-centric scenes with exceptional text-image correspondence and naturalness using existing pretrained diffusion models. BeyondScene employs a staged and hierarchical approach to initially generate a detailed base image focusing on crucial elements in instance creation for multiple humans and detailed descriptions beyond token limit of diffusion model, and then to seamlessly convert the base image to a higher-resolution output, exceeding training image size and incorporating details aware of text and instances via our novel instance-aware hierarchical enlargement process that consists of our proposed high-frequency injected forward diffusion and adaptive joint diffusion. BeyondScene surpasses existing methods in terms of correspondence with detailed text descriptions and naturalness, paving the way for advanced applications in higher-resolution human-centric scene creation beyond the capacity of pretrained diffusion models without costly retraining. Project page:janeyeon.githu...
