Nvidia Instant-ngp: Create your own NeRF scene from a video or images (Hands-on Tutorial)
ฝัง
- เผยแพร่เมื่อ 22 พ.ค. 2024
- This video is all you need to convert a video or image to NeRF scenes on Ubuntu 22.04 machine with Cuda-nn, Python 3.9, colmap, ffmpeg and other libraries..
The new NVIDIA NGP Instant NeRF is a great introduction to getting started with neural radiance fields. With your own video and/or image dataset, you can compile the codebase, prepare your images, and train your first NeRF. Unlike other NeRF implementations, Instant NeRF only takes a few minutes to train a great-looking visual.
GitHub Resources:
github.com/prodramp/DeepWorks...
Research Paper and Code:
github.com/NVlabs/instant-ngp
▬▬▬▬▬▬ ⏰ TUTORIAL TIME STAMPS ⏰ ▬▬▬▬▬▬
- (00:00) Quick Demo
- (00:45) NeRF intro
- (01:45) Content details
- (03:35) Setting Ubuntu 22.04 Environment
- (08:30) Building instant-ngp library
- (10:00) Convert video to NeRF Data
- (18:05) NeRF Scene Demo starts
- (21:18) NeRF Rendering Options
- (24:28) Debug Visualization
- (26:35) Saving NeRF Snapshot
- (27:26) NeRF Histogram
- (28:20) NeRF Volume
- (29:32) NeRF Gigapixel Images
- (31:52) Conclusion
Connect
------------------
- Prodramp LLC (@prodramp)
- Website - prodramp.com
- LinkedIn - / prodramp
- GitHub- github.com/prodramp/
- AngelList - angel.co/company/prodramp
- Facebook - / prodramp
Content Creator: Avkash Chauhan (@avkashchauhan)
- / avkashchauhan
- / avkashchauhan
Tags:
#nvidia #ai #nerf #cnn #ml #lime #aicloud #h2oai #driverlessai #machinelearning #cloud #mlops #model #collaboration #deeplearning #modelserving #modeldeployment #pytorch #datarobot #datahub #streamlit #modeltesting #codeartifact #dataartifact #modelartifact #onnx #aws #kaggle #mapbox #lightgbm #xgboost #classification #dataengineering #pandas #keras #tensorflow #tensorboard #cnn #prodramp #avkashchauhan #LIME #mli #xai #instant-ngp #cuda #cuda-nn - วิทยาศาสตร์และเทคโนโลยี
![MOBYE - มีใจทำไมไม่จีบ (ASAP) [Official MV]](http://i.ytimg.com/vi/dCOdz92FU18/mqdefault.jpg)
Seeing this stuff from start to finish caters to my learning style. Thanks for the tutorial! Have you seen Nvidia's 'nvdiffrec' yet? Apparently it's like photogrammetry, but it spits out a model AND a complete PBR material set!
Thanks for your positive feedback, glad to be helpful. Yes I have started digging into nvdiffrec and a video is on its way too.
For those who wants to save the out to MP4 please use the run.py (scripts/run.py) with parameter --video_output to save your NeRF output as mp4. Please open the run.py and look for various options supported with the run.py.
Thank you 🙏🏻
I hope the technology improves in the future. Because even from two photos you can get a stereo image with a more accurate and detailed 3D model. I remember seeing experiments over a year ago.
Thanks for your comment, appreciate it sincerely. OpenAI Point-E is very encouraging as your suggested.
Can you give some references? I agree that two photos can give you a great 3D model with more traditional techniques, but rendering other views seems to be a harder problem.
aaaand crash at 28:12 lol. Thanks for the video.:)
You can keep tracking the memory consumption to avoid crash..
@@650AILab Indeed you can. I was referencing the fact your program crashed at that moment, but you cut it out of the video lol.
I've been having a lot of fun playing with instant-ngp the last couple of days and I'm very excited to see how far NVIDIA can take it before release. :)
when i create custom dataset my images are always outside the visual unit box in instant ngp
Do you give procces servics ?
Hi, it's a great tutorial, thanks! BTW I've got a question. I saw in the video when you convert video to NeRF Data, it costs less than 1 minute to finish colmap2nerf with 200 pictures. However, when I was running the same command with 3080ti, it will cost me about 20minutes or more to get a transform.json for only 100 pictures. Do you possibly know how I can improve the speed of converting data? Thanks a lot!
Thanks for the comment and I am glad you liked it. It depends the GPU configuration based on exact GPU class I'd when you install the libraries. Please make sure your GPU is configured correctly as I did see longer time to get NeRF content out from the pictures.
@@650AILab Thanks for sharing. What do you mean by GPU configuration? Does different GPU need different libraries installed? Or what can I do to check if it's the right configuration.
Very nice video helped a lot, how can i take a screenshot / render of the scene?
Please use script/run.py to save video as MP4 and then you can export images from MP4.
Hello!
Maybe you can help me with training
Im running without gui (docker container), and have no idea how to save result into msgpack
&
What full pipeline of train?
I should run
scripts/colmap2nerf.py
then
./build/testbed --scene /app/transforms.json --no-gui
and then pass msgpack into run
scripts/run.py, is it correct ?
Thanks for your comment and question.
I just study the run.py code and where you can use the following option to save the snapshot:
scripts/run.py --save_snapshot [name.msgpack]
Sir, could i use NeRF to 3D scan an object with instant NGP then import obj into mesh lab and export it as STL for 3D printing?
I know that you sure can export it to the mesh lab and after that I am not sure how to covert the meshlab object to STL as I have never done it.
I am a 360VR videographer, can I use LatLong videos as my source?
You need various photos from all around the object to generate NeRF content. The focus in outside to inside, In 360 videos the camera is in the center so it will not work.
@@650AILab san Thank you. I thought shooting with 360 video would be easy.
Sir *which Graphic card of Nvidia is* *best for this kind of Instant NGP* *(AI)* .
The latest is better as you will get a higher refresh frequency however it is very costly to get it so if you take 2 or 3 generations back it's okay. Make sure to have over 12 GB or above memory. Thanks for the comments appreciate it sincerely.
hey guys. can somone help me how to install optix sdk? Thanks. In the video this part is skipped.
You must make sure to have Nvidia drivers, nnkit and other Nvidia dependencies installed then optix SDK becomes easier to install. If you share the error message I can provide exact help
how to render and export to mp4 file?
python: can't open file 'E:\Tutorial
gp\instant-ngp\scripts
ender.py': [Errno 2] No such file or directory
well the file your python is trying to open doesnt exist... try maybe reinstalling python
@@mootvey The code is moved from render.py to run.py (scripts/run.py) and please use --video_output with the run.py to save as mp4
The code is moved from render.py to run.py (scripts/run.py) and please use --video_output with the run.py to save as mp4
My right ear feels lonely....
what are the licenses for using this in a commercial project?
Each commercial product has its own license as well as open source project. Please consult to the product creator to learn more about the license before using the product if you have concern with the license.
Thanks for the comment, appreciate it.
i need some help setting up, do u have a discord?
Yes. You can email me directly and I will be happy to help you.
anyone seen this error?
#:/opt/instant-ngp$ sudo cmake . -B build
CMake Error at /usr/local/share/cmake-3.24/Modules/CMakeDetermineCUDACompiler.cmake:277 (message):
CMAKE_CUDA_ARCHITECTURES must be non-empty if set.
Call Stack (most recent call first):
CMakeLists.txt:11 (project)
There are some issues with cmake 3.23 so if you will use 3.22 I believe the problem will be gone.