NeRF in the Dark: High Dynamic Range View Synthesis from Noisy Raw Images
ฝัง
- เผยแพร่เมื่อ 20 พ.ค. 2022
- We present RawNeRF, a method for optimizing neural radiance fields directly on linear raw image data. More details at bmild.github.io/rawnerf
CVPR 2022
![[Full Episode] MasterChef Junior Thailand มาสเตอร์เชฟ จูเนียร์ ประเทศไทย Season 3 Episode 3](http://i.ytimg.com/vi/A1XXYbV048k/mqdefault.jpg)
It never ceases to amaze me every time i watch this, great work!
This is some next level stuff ..I hope it will make it to pixel devices very soon.
This is absolutely incredible.
When the picture started to move HOLY MOLLY this is insane!
Absolutely stellar work that's being done. Super impressive
This is fabulous, congratulations! As a professional portait photographer, I can't wait to have such abilities at the photo-taking and post-processing stages
You can lose your job with this software
@@8511a This is not a robot taking photos by itself.
@@Aztonio doesn't matter. you won't need a professional photographer to take pictures if Mr Nobody can do the same with no experience
@@YuriG03042 Not the same, there will always be more tricky parts of the job.
I'm so shocked and impressed of this technology. This is revolutionary in every aspect
Well, this is... without hyperbole, possibly the most incredible video I've ever seen in my life.
Absolutely stunning, awesome work.
Incredible results and its really impressive
This is incredible work. Gives a glimpse into what we can expect from the future of cameras. Awesome presentation as well! 🙌
My mind is blown! Amazing work 🤯🙌👏
This is insane level working, unbelievable 🤯
The focus changing is super cool
Wow! Excited to try this for myself one day.
Very impressive, well done.
Fantastic work. Good going!
This is so crazy. Wow, great job. Welcome to the future ;-)
Very impressive work.
This is awesome!!
This is incredible
Incredible. Great Job.
Just bought a Light L16 and this seems like it'll be perfect for it
Woow! I like the possibility to change exposure !
This is awesome!
Jaw-dropping results
love it!
wow, this is amazing!
so impressive!
That is amazing.
Amazing!
05:17 This looks just like a model to me, not a real exterior, crazy stuff!
This whole thing is pure madness, I think these possibilities will soon be implemented in most devices, and I am most looking forward to the day when I will have a smartphone with this application or Camera capabilities.
Well done, I hope the author makes a good profit from this, he deserves it! 👍
Looking forward to seeing what actually happened during The Battle with the Night King in GoT lol😄😆😀
gorgeous
Mind blowing
If I hadn’t seen the other magic “ENHANCE!” technologies in 2020 and 2021, I wouldn’t believe this. The 2020s will be quite something.
this is insaneee
Amazing.
Amazing
Blinkin incredible.
4:03 is really impressive
No limits ! 👏👏👏👏
Nie powiem, robi wrażenie.
Almost unbelievable. I'm trying to grasp the consequences for digital imaging, but technologies like this (and those that are to come in the next months) will have far too much impact in too many areas to make any serious predictions. The article that led me here was titled "(...) noise reduction out of this world". Sounds like clickbait, but it's totally true.
That's wild
This is insane
I can't wait for this to come to a pixel and I'd love it's implementation to astrophotography and night images. So how long until this comes to a pixel device?
This is insane.
Wow!
Is it possible to use only rawnerf as a denoiser? Im currently using image stacking to reduce noise on pictures by taking 2+ images matching them up and then combining pixels to reduce noise. Would this be able to replace that?
Try DXO PR, I'm able to get clean images at ISO 51200 taken with my Nikon Z6.
@@USGrant21st I find Topaz Labs DeNoise AI pretty good, too.
While these are both great options, they are not able to be used in my usecase. Neither of those options are free and topaz does not offer a cli interface and is resource intensive, not sure about dxo since ive never used it. My usecase is a image processing pipeline that automatically runs whenever a set of photos are taken on a linux phone. In my case thats a pinephone using the megapixels camera app. That will take a burst of photos depending on the iso used and will then call my custom bash script to do all its processing. Im having it call a docker image where i have all the tools and machine learning packages setup and ready to go. It also needs to be something fast as the pinephone is pretty anemic in terms of processing power so something heavy like topaz would take to long even if i could use it.
@@luigi3111311 Time to invest in better hardware?
@@MrLeiduowen not sure what you mean by better hardware? All pictures from a smartphone are mediocre when raw until lots of postprocessing is done. Im doing some techniques similar to what google is currently doing but i of course dont have the smarts like them which is why im trying to see if i can piggyback off some of the opensource work they are doing in machine learning so i can add that to my open source image pipeline for all linux phones as of right now there is 0 post processing happening other than converting from raw images to jpgs. Even if upgrading to better hardware was the solution theres not really a better hardware for me to upgrade too since the pinephone is one of the only mainline linux smartphones besides the pinephone pro which camera development is still happening on and the other freedom phone thats super expensive and takes forever to ship out.
It would be wholly appropriate to classify this under Dark Arts.
Are their apps like this for video?!
Wow! It blows any professional camera out of water!
I was ready to get a virus on my pc lol, but it actually worked!
Is there any downloads downloadable code that will render 3D data? It looks like that this would be useful for 3D reconstruction?
How many input pictures did the scenes need?
If I'm understanding what's happening here, then I happen to have a very large library of 9-shot HDR photos (taken over the course of many years) that will be very fun to run through this process!
I think they need to have slightly varying angles.
@@Pixelsplasher That's what I was afraid of.
You need only one img. The perspective / deep-information stuff is made by another ai ;) th-cam.com/video/MrIbQ0pIFOg/w-d-xo.html
wow
Does it require multiple pictures from different angles? I wonder how it knows the information behind the objects
yes i believe they say in the beginning with the flower it had 20 some input photos
Could be apply NERF on photos of telescope James Webb to we see more detail. Its Work?
do you need a cam with Lidar?
WOW
This technology will add to video on pixels??? Or any for photos?
It's not clear, did you have multiple images for all these examples?
That's the sense that I'm getting. They feed in a dozen or so images and it uses those to de-noise and stitch together the animation.
Would love to have seen more raw images if not all raw images which are used for the examples
flip 3d stereo images, search and you will find. Photos for left eye and right eye alternating.
@@mrkitty777 what does this have todo with stero images? This doesn't use stereo images as far as I know
Could this be used to clean up videos, taking advantage of similarities between frames? Eg to resolve license plates from car dashcam footage that would otherwise be unreadable? And also to construct 3D models from videos? I seem to remember hearing that US police have some tech to clean up video, read license plates, and get clearer images of people.
It sure can, would love to see how the now popular ghost videos can easily be debunked with this tool.
So when does this trickle down to the Pixel 6? I've got plenty of disappointing low light phots where the subject was too far away.
Where can I get the app?
I'm confused about one thing: in each case (each physical scene) it seemed like you took many images. Are all of these images required for this to work?
In other words, given just 1 noisy image of just 1 scene, does this still work?
No. The point is that by taking multiple noisy images of a scene you can use the non noisy parts to infer information about the noisy parts in the other photos. Keep doing this recursively and you end up with a 3D "space" with no noise. Then take your new photo from this 3D space you have constructed and it will be clear and noise free.
@@wiredvibe1678 In essence, then, this is like learning that a 'stacking' type noise averaging can work even in 3D; and as a bonus you get scene reconstruction and alteration capabilities too.
Is there an order of magnitude estimate for how many images one needs to overcome how much noise?
@@d3xdrive I think the bonus is the noise reduction because this technique was originally developed to create 3d scenes out of a series of images
These guys watched CSI and NCIS too many times and made "Let's enhance" real.
Where to get this NeRF in the Dark?
So, please tell me we are going to be able to use our Pixel phone cameras to create real world scenes for VR. It's amazing. So much more then old photogrammetry technology. Is there a software available to play with this capability? Matterport will be so much better once this becomes the standard.
Not today or tomorrow but probably eventually
@@wiredvibe1678 I'm thinking Meta Project Cambria will take over Materports camera and provide next gen live NeRF SLAM mapping. I hope some one is working on that. Also a button in VRChat to create the IRL room you're standing in.
@@RayRift again, you are probably on the wrong path, just underestimating how long it will take to get there
@@wiredvibe1678 again? Matterport looks like a pos toy compared to what Cambria can do if you even understand what I'm talking about.
@@RayRift idk maybe your on the Cambia development team and you know more than I do
otherworldly stuff, imagine using it with cameras, which already offers great dynamic range and sensitivity performance...
😱😱😱 whaou 😲
Jawdropping.
Can you imagine putting this tech in a mirrorless camera?
No pre-trained model available?!
no 4k?
"Enhance!"
OK, but how do I use it?
Will the dataset be published?
storage.googleapis.com/gresearch/refraw360/raw.zip
@@benmildenhall4102 Does this only work on raw sensor data from the cameras? If not it would be an amazing tool to reconstruct detail from noisy CCTV images recorded in multiple crimes
Do you see this being used to restore poor film quality for the purpose of film preservation?
Okay, how can I use this?
So now when the say "Zoom in and Enhance" on NCIS they can actually do it.
😮
Waiting for a tutorial... ⭐
when is this coming to google photos
🪄 magic
how to install this?
Many methods use multiple images to remove noise. Where is the comparison to relevant methods?
Apple buy this now!
Es lo más parecido a la vista humana
You need to make this as a plugin for Davinci resolve
Are you one of the folks that came over from Lytro?
In other news, the US government just admitted some UFO sightings are indeed Alien. Crazy times we live in.
So it seems CSI might "endless zoom" might not be that far from truth :)
Luckily my skull contained the remnants of my blown mind.
Вот и процессор tensor от Google пойдет в дело. Кстати умно придумали технологию удаленной обработки информации с носителя. Так и смартфона меньше работать надо и качество выше!
Ye got some nerf showing stuff like this, i tell you that much
It would be impressive if this was done from ONE grainy noisy image.
But since it requires multiple images in various angles and viewpoints,.. it basically just fills in the missing details from other pixels and normalizes the result.
Still nice,... but not "Impressive", at least not to me.