NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections [updated]

Project Page: nerf-w.github.io/
Ricardo Martin-Brualla*, Noha Radwan*, Mehdi S. M. Sajjadi*, Jonathan T. Barron, Alexey Dosovitskiy, and Daniel Duckworth
(* Denotes equal contribution)
We present NeRF-W, a system for 3D reconstruction of landmarks from unconstrained, "in-the-wild" photo collections. Given a set of posed photos, NeRF-W is able to disentangle the shared, underlying 3D geometry from transient objects and photometric variations, producing a consistent, photorealistic scene representation that can be rendered from novel viewpoints.

[@mark_zuck]

I really hope that one day this is used to make street view on google completely 3D, you could literally walk around cities in virtual reality. Imagine when your older getting to see child hood locations in 3D VR

[@mixery212]

Next you would be able to do this with video. Imagine a rendered concert out of short video snippets from the crowd. You can also apply this to events. 9/11? Kilimanjaro explosion? Wow. Just wow.

[@jimj2683]

In the future you could just feed the AI with all the photos and videos ever taken and out comes a dynamic 3d world that can be used in a realistic game. GTA Earth...

[@DannoHung]

Very impressive results!

[@justanotherhotguy]

I wonder what this could mean to the movie industry; you could have a place captured with several images and have the scene be purely CGI. That looks promising!

[@ONDANOTA]

Hello! Nice content. How long does it take to render? Any info about hardware usage, memory, time etc, everything you know would be much appreciated.
Also, can radiance field be converted into an obj or any other 3d format?

[@nicolasportu]

Outstanding!

[@Tmmy22]

Awesome - In terms of measuring in the model, what's the accuracy like?

[@explainmemllike5]

This is nice!

[@wolfgangsam4652]

Wonderful! I came here from nvidias nerf, so you developed this beauty 2 years ago ?

[@loofers]

insanely amazing work. next you just need to let the ai detect and semantically tag separate objects in the scene, tag humans, vehicles, etc. and then you could reimport it and change elements in existing scenes like with deepfakes, but in 3d. has insane implications for video encoding, ai vision and ai object comprehension, etc, semantic ai understanding of whole scenes and relevance. you could upsample lower quality movies into high resolution 3d scenes. when this is applied to video, we would potentially be able to import whole scenes from a movie into blender or similar software. ai vision will be one of those items mentioned in the top 50 inventions since the wheel in a century's time. you guys are literally knee deep in creating history.
Arthur C. Clarke was prophetic; "Any sufficiently advanced technology is indistinguishable from magic."

[@risingtide_official]

how big are the files?

[@dinokaiser]

Is that awful looking blur a result of nerf or done in post?

[@Saxonite_]

Any publicly available programs or links that allows anyone to do something close to this at home?
It's mighty impressive and a step beyond regular photogrammetry.

[@TheMcSebi]

it's possible now, the youtube channel bycloudai as uploaded a video explaining how to easily setup instant-ngp (from nvidia) on windows.
It looks good (and is blazing fast). only caveat is, the meshes it outputs are unusable.

[@odi_de_podi]

For anyone looking, it's this video: th-cam.com/video/fvXOjV7EHbk/w-d-xo.html

[@andre0000000007]

please re-do this with gaussian splatter