Can you imagine what a Google Street View would be like with this technology? Being able to navigate the streets as if you were walking there, rather than just viewing a static photo as it is today.
Thanks! Yeah bummer on the no code but I’m glad the examples are public. I suspect people will repro it just like they did for DreamFusion. And consumers will prolly see it in Google Maps
What i'm waiting for is 4DGS (4D Gaussian Splatting for Real-Time Dynamic Scene Rendering) I bet that kind of emerging tech could be applied to NERF/SMERF too
the main goal would be having a realtime nerf/smerf/3DGS on a mobile phone to be able to have a realtime photoscamner to get feedback to capture more images on areas that need it that would looks blurry or distorted in the realtime view on a mobile phone.
I think nerfs and splats are already better than matterport. But the 360 pano + simplified mesh model paradigm has a lot of staying power in the industry.
These are some pretty amazing developments, but where the rubber meets the road is in the tools that are going to be available to make use of all of this. I would love to see videos and links of actual tools that a creator can use.
i tried one of the demo scenes on a google pixel 6 Pro in highend pc quality and got around 3-20 fps in the gardenvase demo and it looked way way better than the smartphone demo quality
There still hasnt been a very accessible solution for artists who want to make high quality nerfs. Like poly cam is pretty terrible compared to what you see here. I think once there are more easy to use programs for zipnerf and smerf we will see some really amazing stuff
Not yet. But they dropped code for Zip-NeRF (the teacher model) recently, so perhaps SMERF will follow? More here if u wanna dig in: twitter.com/bilawalsidhu/status/1749133782672670740
woe i thought i was watching a channel and video with maybe 5 million views but only 7.4k wow. Also I wish i could bring this into the game i play transport fever 2 😭😭
i like the part where SMERF is not used anywere and it only exists on github
Can you imagine what a Google Street View would be like with this technology? Being able to navigate the streets as if you were walking there, rather than just viewing a static photo as it is today.
SMURF training time is still magnitudes longer than GS.
great breakdown of this area!! also sick editing 👀
🎉 thanks for this. I love how you present this amazing tech
No code! Damn that's a shame. I heard google is known for not sharing these with us common folk
Thanks! Yeah bummer on the no code but I’m glad the examples are public. I suspect people will repro it just like they did for DreamFusion. And consumers will prolly see it in Google Maps
Definitely getting a like, comment, subscribe and rewatch for this one. Great video! I hadn't even heard of this yet!
Brilliant stuff ! Also great vulgarisation !
Thanks!
What i'm waiting for is 4DGS (4D Gaussian Splatting for Real-Time Dynamic Scene Rendering)
I bet that kind of emerging tech could be applied to NERF/SMERF too
These videos are so high quality, keep it up dude.
🫡
the main goal would be having a realtime nerf/smerf/3DGS on a mobile phone to be able to have a realtime photoscamner to get feedback to capture more images on areas that need it that would looks blurry or distorted in the realtime view on a mobile phone.
Nice video with really cool editing. What programs do you use for editing?
PLEASE MAKE MORE VIDS LIKE THIS!! SUPER FAN OF YOU!
🫡 more on the way - trying to get a regular cadence going
Will it replace Matterport
I think nerfs and splats are already better than matterport. But the 360 pano + simplified mesh model paradigm has a lot of staying power in the industry.
These are some pretty amazing developments, but where the rubber meets the road is in the tools that are going to be available to make use of all of this. I would love to see videos and links of actual tools that a creator can use.
Would the SMERF technique now be possible with the new Nvidia H100 GPU?
i tried one of the demo scenes on a google pixel 6 Pro in highend pc quality and got around 3-20 fps in the gardenvase demo and it looked way way better than the smartphone demo quality
Papa SMERF is throwing down the gauntlet. Great explainer and overview
Thanks Joey! Loved your recent coffeezilla video btw
@@spatialintel thanks!
There still hasnt been a very accessible solution for artists who want to make high quality nerfs. Like poly cam is pretty terrible compared to what you see here. I think once there are more easy to use programs for zipnerf and smerf we will see some really amazing stuff
Is the code out for SMERF?
Not yet. But they dropped code for Zip-NeRF (the teacher model) recently, so perhaps SMERF will follow? More here if u wanna dig in: twitter.com/bilawalsidhu/status/1749133782672670740
My head is still spinning from Gaussian Splats it can't keep spinning like this!
I'm subscribing alright, no worries 👌🏻
Great video! Is your background virtual? It's sick!
Thank you! Real background, though I want to play with chroma key and unreal at some point
Sweet dude! Looked chill with the fire, thought it was a virtual enviro, Ha! Good stuff. @@spatialintel
Nice video
just subscribed! great video. Are they or do they have plans to release code to play with?
Thanks! Re- code release id guess in a year? They just release zip-nerf a while after it was published
This looks very promising but how we can implement it in 3d softwares.
This continues to be a challenge for neural approaches to radiance fields versus Gaussian splatting
Come on luma .. do an update and add this method
I’m sure they’re cooking some stuff, though they seem to be doing a lot more text to 3D stuff lately
Very cool tech indeed but plz don't keep calling it "Reality Capture" (photogrammetry software)
Ha! I’m sorry I get that argument all the time but the phrase reality capture is way bigger than one photogrammetry tool
@@spatialintel Just use something else like Spatial Capture then
When will Google street view use this to become 3d? I want to play GTA on Google maps!
Dudeee, This is soo fuckinnn good 😮
Seriously amazing results. I spent like an hour just flying through the sample captures looking at the view dependent effects 😂
woe i thought i was watching a channel and video with maybe 5 million views but only 7.4k wow. Also I wish i could bring this into the game i play transport fever 2 😭😭
Just keep them away from Gargamel
😂 😂
No released models or implementations?
I think people expect SMeRF will be out soon-ish
Yeah the teacher model for SMERF - zip nerf just got open sourced. Hopefully SMERF follows soon