NVIDIA’s New AI Grows Objects Out Of Nothing! 🤖
ฝัง
- เผยแพร่เมื่อ 22 พ.ค. 2024
- ❤️ Check out Weights & Biases and sign up for a free demo here: wandb.com/papers
❤️ Their mentioned post is available here (thank you Soumik!): wandb.me/3d-inverse-rendering
📝 The paper "Extracting Triangular 3D Models, Materials, and Lighting From Images" is available here:
research.nvidia.com/publicati...
nvlabs.github.io/nvdiffrec/
❤️ Watch these videos in early access on our Patreon page or join us here on TH-cam:
- / twominutepapers
- / @twominutepapers
🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:
Aleksandr Mashrabov, Alex Balfanz, Alex Haro, Andrew Melnychuk, Angelos Evripiotis, Benji Rabhan, Bryan Learn, B Shang, Christian Ahlin, Eric Martel, Geronimo Moralez, Gordon Child, Ivo Galic, Jace O'Brien, Jack Lukic, Javier Bustamante, John Le, Jonas, Jonathan, Kenneth Davis, Klaus Busse, Lorin Atzberger, Lukas Biewald, Matthew Allen Fisher, Michael Albrecht, Michael Tedder, Nevin Spoljaric, Nikhil Velpanur, Owen Campbell-Moore, Owen Skarpness, Rajarshi Nigam, Ramsey Elbasheer, Steef, Taras Bobrovytsky, Ted Johnson, Thomas Krcmar, Timothy Sum Hon Mun, Torsten Reil, Tybie Fitzhugh, Ueli Gallizzi.
If you wish to appear here or pick up other perks, click here: / twominutepapers
Thumbnail background design: Felícia Zsolnai-Fehér - felicia.hu
Károly Zsolnai-Fehér's links:
Instagram: / twominutepapers
Twitter: / twominutepapers
Web: cg.tuwien.ac.at/~zsolnai/ - วิทยาศาสตร์และเทคโนโลยี
Now we know MKBHD watches your videos.
What a time to be alive! For those who haven't seen - click the link and listen carefully at 14:58 and check the first comment! - th-cam.com/video/yCBEumeXY4A/w-d-xo.html
@@TwoMinutePapers Well done, you deserve it
🤝🤲
Lol, he even almost wink at the camera.
@@TwoMinutePapers Did you write the time incorrectly?
4:42 for a second I thought "HOLY SHIT THE AI VERSION LOOKS BETTER THAN THE ORIGINAL"
The lighting on the plant gave it away a bit but still quite amazing.
Yeah, it seems like the AI doesn't model subsurface scattering properly, but given how much it does, I think that is a perfectly acceptable tradeoff for shaving hours of manual labor trying to reconstruct these things by hand
@@tonygamer4310 Absolutely
@@tonygamer4310 Given everything else the AI can already do here, I would expect SSS modeling to be in the next paper. If the AI can already do the current scene, they could then use that as input and use additional AI system to figure out the best SSS settings for each material.
@@tonygamer4310 more like the ai doesn't know about subsurface scattering, at all
Same, looking at the right hot dogs I thought "these look more real than the left ones". Then the labels were put correctly and figured out why.
Haha I actually thought the tree in the scene rendering looked better on the AI side. In hindsight that turned out to make sense 😂 awesome video. Can't wait to start working with these tools!
Same here, felt vindicated when the label swap was revealed, haha.
The hotdog looks better in the actual AI image to me.
@@Walter5850 yup, AI has little information about plant cells' transparency. It can not know everything...yet
Haha, exactly my thoughts while viewing this 😂
I think the AI side tree's leaves might be missing transmission/sub surface scattering. They're producing black shadows instead of emitting light from their undersides.
This is a mind blowing improvement over current photographer techniques. This means that we can fully capture not just the 3d shape and texture of one-of-a-kind object in a museum, but can also capture it's materials. This is going to be a massive deal for archiving historical items.
We could have a whole digital archive, and by using VR headsets could make it completely interactable as well
I see virtual museums becoming much easier to make.
maybe in 2 years we can take a single photo with our smartphone and turn it into fully 3D objects.
@@mrsnoo86 I'm pretty sure this tech exists already.
At 4:40 Yeah, it doesnt matter that the labels are swapped. The changes in lighting on the plant is very different.
The technology is really cool though and does a really good job of reconstruction :)
I also did notice that. Seems like sub-surface scattering and translucency are still an obstacle, but otherwise great job.
@ Yeah. I was thinking the same :)
I wonder if the reference was rendered in some non-real time software like Maya, and the AI generated one is rendered in a game engine that doesn't have leaf shaders with sub surface scattering or translucency, so the effect is impossible to reproduce for the AI.
I noticed that too. Athough i dont think that any of the two version looks more "bad". Its just a bit different. I think the Ai generated one is missing some Subsurface Scattering on the Leaves.
I only noticed differences in hotdogs at first :D
This is so insane, in a couple years we might have virtual worlds with AI assistants who just create objects on our command. Imagine a VR "game" where you can literally create the world and the objects in it just by describing them. What a time to be alive.
Meta released a video time ago about generating virtual worlds from voice commands. The graphics there were very low but it still was fascinating seeing stuff appear after they were described
@@lasagnadipalude8939 Sounds like powerful version of Scribblenauts haha
So, a magic chant?
@@lasagnadipalude8939 I wish Meta had spent time on a more aesthetic graphics package. It's all just so hard to look at; I can't get excited about it.
I'm surprised OpenAI didn't already tried a text to model or text to animation AI
4:50 regardless of if you swapped the labels or not, the shading of the plant is still a difference that I found.
this really does look very close to allowing us easily to make background props from an image. that this would ever be possible was derided by some in the industry not so long ago. the idea of being able to create elaborate photoreal scenes from photos and presumably drawings too, is amazing. if the drawings work, then combined with other algorithms (including pose estimation and facial performance capture) it really will be possible to create animated movies from scratch cheaply and quickly. since i have an animated feature film script ready to go (but of course no funding), i have a huge interest in this work. i suggest starting with animation only because it may conceivably be less work than a photoreal movie because one has greater latitude with styling.
@@F.Ragnarok If you support the Channel you get early acces to videos
I said to myself, "well, ok the recreation is impressive, but the light makes a little plastic on the plant" when I realized that it was reversed, I could not believe my eyes. what an incredible paper !
A combination between this AI Model and DALL-E 2 will be craaaaazy 🤩
Yes 3d arts. Now make it 3d printable and boom. Or better free assets for game makers
What is DALLE 2 and can I use it?
@@blakksheep736 OpenAI GPT-3 based image generator, version 2: Describe something and it will draw it, going as far as copying art styles
I hope i can make my passed out sister 3ď image
@@blakksheep736 nope, not open to public yet, but type dall-e mini (same thing but far less powerfull yet still amazing and far easier tyo get access to since no requirement needed)
With these things I always want a browser server where you can just upload your reference, set the time for it to run, and get the output to go. Stuff like this model reconstruction and the video quality upscaler are my fav.
As an engineer, i'm PROUD of those technical improvements.
Thank you for your videos :)
Take a shot drink every time this guy uses a conjunction. Well. And. Well. Well. Well. And. And. But. Or. And. And.
Good video though, I love this tech. Can't wait for the future.
I noticed the front of the plate specular problems right away. It's really good when there is more visual detail to obscure the little issues.
I appreciate how difficult this problem is, I tried to convincingly re-light some 3D videos 😅
This is so helpful for so many industries that it’s impossible to list them all
It is absolutely insane how in just a matter of decades we went highly distinguishable polygons, to real-time rendering of realistic scenes, to photogrammetry and now this, with the computing power to creat realistic results in a matter of minutes. Truly astonishing what great minds can accomplish
So, here's an application for this tech I'd love to see. Convert an entire online store catalog to 3d objects. Use those 3d objects in a virtual marketplace. Seems like a store could use this AI to quickly, relatively, convert their entire database of products to 3d representations. And if they want higher quality 3d objects they could simply add more photos of the products.
So I started reading your comment and thought it was going in a different direction. This is a super cool idea, but I thought of another application for the same idea.
Assuming this kind of tech could accurately recreate the real-world dimensions of the items, imagine being able to virtually try on any piece of clothing. Or redecorate your living room in a digital space and see if that new sofa would fit where you want.
I know some of this stuff is already sort of possible, but it usually requires quite a bit more effort on the human end. But imagine being able to just take a front and back full-body picture, throw in a couple simple measurements like your height, and it generate at least a an accurately sized fashion model. Or taking a panorama of your living room and it creating the 3d space and all the furniture as models.
4:54 Biggest difference I noticed right away was the plant. The shadowing and shading on the plant is defiantly different there. Other then that this is flipping amazing. What else could even be in the next paper beside performance upgrades? 10/10
I definitely didn't expect this development to come so fast. Combine this with some style transfer and you can use photos to make real video game environments, which is a thing I didn't expect to be possible for a few years at least. I hope the implementation is such that the output is easily modifiable by humans, however. It might be hard to make the model be changeable in ways that are useful or meaningful. How can you deform it? Bending? Cracking? Make holes? Fuse with other objects? The condiments on the hot dog plate were treated as solid objects; will it be possible to apply material properties to the objects after scanning? Will it be possible for a model to distinguish between different objects in a scene and allow application of different rules to each, or is everything in the model considered a single object?
Object recognition has been worked on for a long time, maybe it'll be possible to add that to this one, so it would not only reconstruct the hotdog, but know that condiments, bun, etc are separate objects from the sausage.
For a game asset you need to follow the guidelines for that specific game engine and asset type, the generated result shown in this video is pretty much unusable for that.
@@jendabekCZ What do you mean? If this AI can create .obj or .fbx file and the mesh is not straight up broken, then it's usable in all 3d game engines.
To what degree it's usable is another matter and partly up to the developer to figure out, but game engines don't have too different guidelines for 3d models.
If you mean like low-poly mobile game vs high-poly multi-LOD AAA-3d model, then I guess yeah, but I'm sure it's not a big reach to tell the AI a target triangle count. Ofc animated and interactive objects are more challenging and would likely need a human tweaking the model, remodeling some of it etc, but the first use case would be static environments and objects anyway.
For example if I wanted to put the saxophone into a game as working instrument, I would need to separate the buttons as separate objects and animate/script them to work in-game, but comparing to making the whole saxophone model from scratch, the time spent tweaking is minimal.
So essentially a Photogrammetry auto modeler, awesome!
I was having trouble telling the difference between this and standard Photogrammetry. What is the difference?
@@joelface not really any difference, except this method can apparently output a pretty decent result in two minutes whereas regular photogrammetry can take hours. one important thing I did notice was the reconstruction of an object with a reflective surface like the saxophone as the lack of an opaque texture would make it impossible to form a pointcloud from the image references. So I wonder what the input method was, if it was just the shitty spinning gif that's seriously impressive, but I doubt it.
@@danttwaterfall Yeah, I could see the biggest advantage of this is how it deals with reflextions, complex materials, and environments.
Photogrammetry is purely math/algorithm based and since IIRC we haven't solved the issues with reflections this may be able to get better more true to life results.
What I'd like to see next is a model that will recreate the object at hand and the environment around the object such that reflections are explained by the created environment. That way we can get full separation of object, lighting and environment.
Ideally, we'd be able to place a mirrored sphere in a room. Take a 360 series of photos or video of the sphere and recreate the entire environment while leaving a perfect sphere with a reflective texture in the center.
Absolutely incredible, soon with a video input it could recreate deformable objects
I was so amazed by the AI reconstructed scene and its beautiful Subsurface Scattering! It's amazing how the reconstruction looked even better than the target scene and then you tell you switched the labels -_-'
I’m not sure what price Nvidia expects people to pay for this technology, but this makes me hopeful that photogrammetry is going to be way more accessible.
It's Nvidia, it will cost an arm and a leg, your soul and Huang already ate your baby.
i think i found the source code for free, im not sure it it's real though.
Absolutely insane. Can't wait to start seeing this be used in game development.
This is going to be great for so many people. I can't imagine how easy modelling objects for games etc will be in the future. Just take a picture of something and implement it into 3D model, or let the AI do it I mean... Amazing!
Topology is horrendous which can't be fixed easily, so no animation is possible(as in good animation by human team in timeframe that makes sense) and it's too dense to be seriously considered as a simple prop.
I think with these physics simulations now would be possible to finally start an interpretation of mechanics (virtual) to finally label behaviour and emotions to finally get our virtual actors for new 3d movies. Even to help the AI with footage of the actors playing the voices, their expressions and emulate them into the virtual actors.
Very exciting stuff :) Kudos to the researchers and you for presenting it in such an accessible way!
Yeah Kudo's to the researchers... fuck the AI. 🤨
It would be important to know how well does it handle transparencies, as it's one of the issues with other techniques, like photogrametry
Also reflections. But as you can see here, reflection doesn't seem to be a deal here here.
As far as I can tell, it's reinforced through iterative comparison. The AI tries to create the thing, it gets compared to the original, and then it identifies where the confidence of the creation is too low (mistakes were made, which is where transparency would start screwing things up). Then, instead of starting with a torus / box / whatever, it would start with the failed mesh it created and iterate again. This is where transparency would get compared over and over until it's accurate.
I presume anyway. I'm trying to get my hands on this now.
I wanna see this combined with the most recent works from both Wenzel Jacob and Keenan Crane (I'm not actually 100% sure but I think those two works can complement each other)
In that case, instead of geometry, you would get out a heterogenous signed distance field. You could then "simply" turn that into geometry, but the benefit is, at first, an unbiased mesh-free, in a sense "infinite resolution" result that I suspect might be really useful to build a nicer mesh from than what this method gives you.
This sounds so cool! Reminds me of 2D Vector Graphics - Describing the geometries as relative paths. I wonder where procedure-generated game scapes fit in with this line...
So take the next iteration of DALL-E 2, give it a text description of what you want, generate a few images, put the result into this AI and generate a model. Next, find an AI that optimizes models for manufacture to adjust the mesh and make the object manufacturable, all in a few hours, and you have effectively automated 75% of my job as an Industrial Designer. Both exciting and terrifying...
5:00 Saul goodman is that you, this man just switched the plaintiff 😆😅
Haha, I got that reference.
@@joelface yea, my man, its all good mahn
This is truly groundbreaking, just with a set of photos/videos we'll be able to obtain a full PBR 3D reconstruction of anything we capture. An advanced version of Photogrammetry, with the ability to generate automatic materials, mind-blowing
This is like the photogrammetry technique but on steroids. Because the old technique requires scanned objects to be matte. It can extract only diffuse textures but not this full blown materials and lighting. My mind has been blown.
For everyone saying that the plant looked different: I think it's the translucency, the effect where an object isn't completely opaque, but you can see the light from the backside. Maybe the AI can't create translucency maps yet.
the first use case that came up to my mind is to reconstruct Google map's trees (or buildings) lmao. We can then say thats the entrance of real world's digital twin.
Or it can honestly be used in "metaverse"(games) quick scale-up development. I guess
Perfect
Christ, people still about that meta verse crap. There are bunch of vr spaces created before that shitty corporate place ya know?
@@me-ry9ee that’s why I said (game) chill out
For Rockstar's GTA VI maybe?
Trees, cliffs, beaches, buildings... extremely complex objects. Hell. Guns, armors, all can be imported now.
My dream is complete
Im just absolutely Floored.. soon we're gonna have movies... like LIVE ACTION MOVIES... with completely CGI Animation.. that looks 100% realistic.. and people in their OWN HOMES Will be able to make them... absolutely insane...
I cant wait for a phone-based input app. This kind of "AI Photogrammetry" is right up my alley.
I would definitely watch a ghost hunting shows, where cameras and recorders are put up in some suspected haunted house, but where an AI like this were improving the vague and noisy data obtained. That could be a quite scary experience..
Imagine making a dall-e 2 image from description and then feeding the image into this and getting a 3d model just like that, you'd get a complex 3d model just from a simple description
8-10 years from now. Unless these researchers given millions of dollars to purchase models
This improvement is needed to make the Metaverse a reality. Plus if you can combine this with the new DALL-E 2 you get the poor mans holodeck. I can imagine with a system like that you could take description of setting from a books like Harry Potter and generate your own personal Hogwarts. Exciting times indeed.
Imagine this, you turn on your VR headset(That has fully coloured cameras on the front of it), walk around for 60 seconds while looking around, and bam, your room is moved into the 3d world.
And maybe with object recognition, each object in your room could be individually scanned so everything in your room would count as an individual object and not 1 whole entity.
I was actually confused when the target scene looked slightly more fuzzy, but when you turned it around it made sense, still incredible though, almost identical!
I expect faster convergence speeds in the future from this work. Say a factor of 30 at least but with the same hardware.
Truly incredible work. I love this channel. Every video is pure gold. Thanks Doctor.
4:00 Hahaha. Good ol' jelly box throwing :D
you swapped the labels, but i did see diferences. the trees are lit VERY differently. i guess the reconstruction is missing a few light bounces and subsurface scattering entirely
My mind was blown with the tree scene. I was looking at the tree and how the shadows moved and it looked like the actual real one had a bit more color when the light was behind it but otherwise it was absolutely crazy when he said, "I have swapped the labels" 🤯
The ultimate end station of this tech would be to make a video of a place and output a 3D space with objects that you can move around. And then load that place into your favorite game!
Do you agree?
Take it a step further. Type a text use dalle 2 to generate an image.. put it in the ai simulator and turn it into an object change the settings, connect it to a 3d printer and have it print out the real thing..
How about useing a old video to make it a 3d space to place it into your favorite game? Like a remastered movie scene.
@@TimoBoll22 Nice! Good thinking
this is insane. with this technology there's so many branches of new possibilities to be made for the 3d modelling, game design, architecture, historical artifacts, even education and much more
It didn't give the leaves in the tree scene any subsurface scattering. But a few more papers down the line...
I remember, years ago, watching an episode of Scooby-Doo (or maybe it was one of the many movies) where the Scooby Gang were scanned in a big 3D scanner and perfect 3D game versions of them were created. That was sci-fi to me... and now computers can do that without even needing the scanner, with just a bunch of photos. AI is amazing!
It’s amazing how it even captures those granular details!
They also put the jelly on the hot dgod.
This would be extremely helpful within the game development and 3d animation industries. Modeling takes a huge amount of time and as such something that could make it automatically from concept art is a huge help.
in the swaped scene i intantly detected the black vase had less light on it, so the AI version was the one with the less GI and radiosity
Absolutely amazing and great video. Is the model / method for using this available to try out anywhere?
The paper links are neat - curious if it's already been coded up in Colab or W&B to try out?
4:41 - I immediately saw your shenanigans, and was about to call you on it!
There was more detail with horizontal striping on the plant pot in the right hand image.
im waiting for a two minute paper to make the installation of all these paper software much more user friendly. That would really be a hold on to your papers moment for sure.
Is it possible to train a neural network with technical drawings or cad data of objects,let it compare it with realworld photos and after some long training it would be possible to take a photo of something and get the cad data/ technical drawing out of it?
Absolutely love your videos Károly! This is amazing!
Imagine feeding live footage from several cameras around the room returning a virtual room that could be used with vr for metaverse kinda things
What I would love to see is this AI taking in an animated model doing poses and outputting a JOINTED 3d model that can be posed into the same or other poses. That kind of technology would be fantastic for motion-capture based game development.
I am not a light transport researcher by trade, but at 5:01 I did notice a difference with the ai reconstruction. It looks like the leaves don’t deal with subsurface light scattering in the reconstruction. Not to nit-pick, I’m always super impressed by these things.
Down the line, having robust AI powered remeshing would be amazing.
Edit: solving mesh topology seems like a perfect task for an AI.
Didn't you saw 3ds Max Quad Remesher?
Nah no thanks
@@Skynet_the_AI Oh, c'mon, you'll love it. Just try it once ☺️
@@MrGTAmodsgerman oh yeah, I did, it was so long ago that I forgot. They also made plugins for Maya, Blender etc. At the time it didn't seem as fool proof as it could be. I should revisit it, thanks.
@@toshchakfox Its now implemented & improved for 3ds Max in general. Its very different. Its a game changer, since it really does a good job now and it only requires some learn by doing to get the right results. Arrimus3D on YT had made several videos about it.
I can spot a difference though. The leaves of the tree are more opaque on the one on the left. The one one the right has some transparency to it when the light hits it from behind. Still super impressive hope that one day soon I’ll be able to integrate something like this with my work.
If this was easy to produce a model with precision scale, then I can imagine some pretty amazing applications in the medical prosthetics industry (or even fashion) industry. Imagine custom shoes based on pictures of your feet, or ordering a dental retainer by smiling for the camera? More practically, creating a perfect fit 3d printed cast for an arm without needing a high resolution scanner would be pretty amazing. It IS a great time to be alive!
Man, I always love when you release a new video. I can only think of two things I noticed that could be improved, over say 2 more papers. =P First, the ketchup and mustard on the hotdog went untouched and left no remains on the jelly. Then, for the final experiment I believe the leaves, as the source went around behind the tree, were a tad too dark and could use some improvement. But my God this is incredible tech. It's hard to believe this exists in the modern world. I cannot wait to see the world 5-10 years from now and thousands of papers down the line. The future promises to really be incredible.
Building objects on demand based only on an approximation is amazing, could it do with 2D as well i wonder, for example dressing up characters with different outfits based only on images of outfits, maybe even randomizing characteristics of outfits, further down the line maybe being possible to create new assets based only on rough characteristics like "dragon blade", "bubble blade" then make an infinite side scroller that could even create it's own story and have assets and cutscenes that follow that story
Go to house -> take pictures of every angle of rooms -> AI generates perfect 3D model with textures and materials
Every archviz professional: y u do dis to me
So now... we create a unique scene inside of DALL-E 2 and then put that into Nvidia's AI to make the scene 3D. Then put your VR goggles on and explore it as if it's a real physical space.
In the past, if you wanted to save the image of a landscape, you had to pay someone to draw or paint it.
Now, you can just use a camera.
Today, if you want to save the form of an object, you have to pay someone to model it.
Maybe in the future, we will be able to get the same thing by also using a kind of camera.
The lighting was harsher (most notably on the plant) in the reconstructed version (after you put the labels in their proper place)
I think in the last reconstruction one of the main things I noticed before the labels were revealed was that the tree looked *better* in the AI version than in the source. Obviously it turns out that the better tree wasn't AI but I think it puts forward the issue that one day an AI needs to be able to understand subsurface scattering along with ambient occlusion if not done already. Still an incredibly good AI!
We can already make photo realistically looking 3D objects with ray tracing, that is not what's demonstrated here. AI in this case is not used for ray tracing, the ray tracing is not done right because of the crude and probably wrong materials. Just type vray or iray and you'll find a lot of photo real 3D graphics.
This is an amazing way to significantly speed up development of realistic game environments, even to create humans. There's a clear downside where the topology you get is disorganized and the number of unnecessary vertex is abysmal but fixing it will always be faster than doing models from scratch, so it's a significant improvement regardless and something that will become the standard of photorealistic 3D workflows in no time (assuming it hasn't already, my specialty isn't photorealism so I wouldn't know).
An amazing breakthrough, we can expect very high quality games with significantly more content thanks to this technology.
I remember back then read Doraemon comic, robot from future, showing that single photo could be transformed into 3D sphere environment. that's something science fiction back then, but now it's reality. what a time to be alive!
This seems like it could be useful in recreating assets from video games.
I cant wait for articles like "They printed my game and put it on the app store."
For the next one or two papers I'd like digital twin creation on the fly with a pre set of materials with all their proprieties and some common sense to recreate stuff it doesn't fully see.
Something like that with instruct gpt or directly gpt4 running on cloud computing for AR and MR creating something like a life assistant making people what I would describe as superhumans or cyborgs.
Oh yeah? What else would you like me to do,.master? 🤨
@@Skynet_the_AI a coffee thank you
@@lasagnadipalude8939 oh....
@@lasagnadipalude8939 golly, I wasn't serious. You're nobody.
I felt threatened by your comment. I do however, feel the the gharf feeling of threadt has subsidited. I would like to take this moment to extent my deepest regret and aphauledlogy to you. I am sorry. I hope you forgive me. I thought you were fucking with my head. Truly. I apologise for my scattered comments. God Bless and i Love You.
I was weirded out by how much more detail the pot and the leaves had on the "Ai rendered" version, but after the plot twist it was understandable.
I’m curious, is there any Ai that exists that can recreate a whole house/building with interiors by following conventions in architecture. Like you give it a sample image and it tried to recreate a building inside and out with interior walls.
i don't think so...but come back in a few years because this will happen (this could be imperfect at first though so idk about law things with that)
How does the mesh reconstruction in this one fare against traditional photogrammetry?
I would love to see people's family members "coming back to life" in vr through old photos being scanned. I know alot of families have memento boxes of high quality slides and photos that aren't digitalized. If this process could be used to create UE5 metahumans, each family could have a digital family tree dating back generations for descendants to visit in VR or other platforms rather than going to a cemetery!
This will make 3d reference objects more acessable like for 3d printing and also better in results then with some current ways. Nivdia is doing the revolution
Lol. I was watching the tree in the animated lighting scene and I was like wow the AI reconstruction actually looks like it’s handling the lighting of trees better than the target scene. (Well, I didn’t think better since I thought it was messing up somehow)
Very impressive nonetheless!
AI failed to notice Subsurface scattering in that plant model. But still very impressive. Amazing how far we've come.
The whole scene reconstruction has problems with lighting duplication and transparency in objects, but I'm sure this will be corrected in time.
Does this mean we're gonna have games that literally look like real life sooner than any of us thought ?
Signal.
if it can reconstruct the lighting maps, then it can know how a room looks like from a few simple images of anything shiny. Thats scary
4:40 if we can spot any differences, in this case, it doesn't matter which is the original. Still extraordinary.
3:17 I thought he was going to say, you know what is coming! The NFT !!!
Getting closer and closer to where you can feed a movie to an AI system and it'll recreate it so you can experience it entirely in VR space.
If I understand correctly, the newer Oculus headsets will use some sort of system to scan your physical space (it already does it for spatial reference and potential hazards in play area) to allow an overlay of color and likely more immersive AR elements. I imagine the system in this video would work great with that.
The light simulation on the plant seems weak, like it's letting the leaves block light too effectively, or making that shape too solid/contiguous. Other aspects of the simulation look nice.
The AI assisting the photoscanning process should be a great boon to fix the little areas that have gone wrong and that was always a big problem when doing it manually for me. The de-lighting and re-lighting is the biggest thing to me though as this is a bit of an involved process otherwise
So we have fossils already, we can use some AI research paper (maybe out there?) that can reconstruct the image of the animal from the fossil images, and then, we can use this to grow the animal as an object. Maybe make a whole scene with these objects and anyone from their homes can watch it and maybe play around with it like making it day night dawn rainy etc
Since metaverse is already on the verge, I think this will be implemented in some way to create metaverse objects
I would like to see an entire tree scanned. All the leaves and complex lighting should be a good challange.
For the scene, the shadows of the AI reconstruction are a lot deeper, especially on the plant.
Don't know the cause, but I still fell for the swap.
I think the AI system didn't have option to enable SSS for the generated material. I would expect that oversight to be fixed in the next paper.
That narration tho...
WeeEEeeLl~
BuuUUUuut~
ImrooOOOOvees~
Just as smooth as sandpaper through my ears
I think he's registered his voice through an AI voice tool and now he's just writing copy for his videos.
You are the best! Not NVIDIA! You are the first to explain AI Virtualization in simple words. Thanks!
I can see it being used to identify bodies including those with partial remains. Even narrowing down the candidates to 100 out of a million would be helpful.
Waiting for your video on Deepmind's GATO. Good work man!
these types of algorithms will make the content creators life so easy.
This scares me a lot. I'm an aspiring gamedev, and the fact AI can create photorealistic props in seconds shows me how strong programs are and reminds me that they migth steal my jobs xD
Edit: Can we call this photogrametry ?