I got this going with the help of your video. Much appreciated! I had a couple of components that needed to be installed and some errors I didn't really understand but I figured it out in the end :)
when i run interactive instant-ngp, error occured: (ngp) D:\instant-ngp\Instant-NGP-for-GTX-1000>D:\instant-ngp\Instant-NGP-for-GTX-1000\instant-ngp.exe test 14:35:25 SUCCESS Initialized CUDA 11.5. Active GPU is #0: NVIDIA GeForce GTX 1660 Ti [75] 14:35:25 INFO Loading NeRF dataset from 14:35:25 INFO test\transforms.json 14:35:25 ERROR Uncaught exception: No training images were found for NeRF training!
after the Camera match part, the json file and colmap_sparse and colmap_test not in the same folder as the image and.MOV. colmap.dp which showed in your video, i can not find it
Sounds like a path error: "No training images were found for NeRF training!" I would suggest seeing my video notes and go to the links for help where I put "Problems? For help, see the Discussion area..." - I can't provide much support for instant-ngp; I don't know much about the errors generated (it works for me).
I see that in your one upload - I'm afraid I don't know why, if the input video isn't using a fisheye. Your witch's castle doesn't seem to have the problem.
Thanks so much SIr! I've tried but it is not working on the custom dataset, i captured video from iphone 14 pro max and extracted upto 400 images and changed the dale from 16 to 32 but still it is not giving me the nerf.It's whole cloudy and i dont know, i need your help in this?
I can't really say. Some nerfs just come out full of "clouds", things converging badly. Dimly lit places seem to have this problem more, but it's inconsistent. I am much less of an expert than you might think. My advice, as always, is to check the text below this video and go try the Discord server etc. I mention there. You might also give NerfStudio a try (I still need to get that one going myself).
There's an Export Mesh at the bottom of the main overlay menu (if you don't see it, scroll the menu on the right edge). Open that section, Mesh it! and Save it! are the options. Don't expect a great mesh - quality definitely varies with how well the NeRF was created.
Really, there's little reason to build instant-ngp from scratch - just download the correct executable. I do have some build tips in the video notes for th-cam.com/video/8GbENSmdVeE/w-d-xo.html, in case they're a help.
The quality of rendering at any image resolution, no matter how much additional training is expected, to put it mildly, is disgusting. Is it possible to make a video to make 512 * 512 exclusively for viewing on a smartphone and in no case watch on a computer =) And since it comes to that, there is also Nerf-Studio, there are *.bat files to which you simply transfer the video or a folder with images, and it the whole whirlwind of operations itself produces and opens an already rendered scene.
Nerf Studio looks cool (and uses instant-ngp as one of its engines). Unfortunately, I've never been able to compile it - it balks on me having Visual Studio 2022 installed, last I tried. So, I'm glad you've gotten it to run for you. I still love what they're doing, fixing a number of typos when they first announced: github.com/nerfstudio-project/nerfstudio/pull/712 . I should give it another try, maybe it will compile for me now.
Hello! Can you please help me? I have 2 Quadro GV100, and if I want to create a 3D model using 150 photos by 8mb each, I have an error “Uncaught exception: FullyFusedMLP: insufficient shared memory available on the GPU. Reduce ‘n_neurons’ or use ‘CutlassMLP’ (better compatibility but slower) instead” This is Volta video card. Does the NGP supports Volta type of video cards ?
I've never seen that message before, but I'm not a pro when it comes to this software. I highly recommend trying the support forums I give links to in my notes for this video.
Hello, I'm starting from scratch (no programming knowledge or software installed), I tried installing all the required software but I keep seeing new requirements every time I try to run the process. I can extract the images from my video, but the process stops after that without being able to generate the camera path file. I get a series of error messages saying I'm missing the following files : boost_filsystem-vc143-mt-x64-1_77.dll, boost_program_options-vc143-mt-x64-1_77.dll, FreeImage.dll, ceres.dll. I have no idea what those are or what I should install. I'veinstalled visual Studio, the CUDA software, Anaconda, COLMAP, FFMPEG, and I don't know what else. I'd appreciate some help. Thanks
Yikes! I'm sorry, I have no idea what these problems are about. I tried searching on the issue at github.com/NVlabs/instant-ngp/issues but didn't get any hits. Personally, at this point I download the proper EXE from github.com/NVlabs/instant-ngp and it just works for me. I was also able to build it from scratch without much fuss; I guess I already had the DLL's needed. Hmmm, if I were you, I'd consider maybe trying Nerfstudio docs.nerf.studio/. Or maybe try the new hotness, 3D Gaussian Splats, github.com/graphdeco-inria/gaussian-splatting (definitely a bit of work getting that going). Easier still, Polycam and Luma are things you might check out (I haven't looked in awhile): poly.cam/tools/gaussian-splatting and lumalabs.ai/interactive-scenes - hope this helps, and sorry I can't help more on Instant NGP.
@@MrErichaines Thank you for replying! In the meantime, I discovered that those Dlls come with the Colmap software, but when using the colmap2nerf script they don't get called. I haven't had the chance to try again yet but next thing I'll do will be to just dump all the Dlls in the same folder with the script and hopefully it will work!
@@sloenmusic thanks for the followup as to what was going on - good to know, and hopefully this'll help someone else. Best of luck getting things going! Instant NGP really is fun (and fast) once you get it going.
How can I activate Instant NGP? I'm able to activate NerfStudio, which I've previously installed. Instant NGP is in my Downloads Folder - I've changed location, copied path of .exe, and it's not working. (Noob to Python)
I don't know how to hook up NGP to NerfStudio (I've tried to use NerfStudio but haven't been successful yet). You can run Instant NGP alone without it (as I do in this video). Just download the right one and run it, basically: github.com/NVlabs/instant-ngp - see the notes for this video for the basic commands you need.
Hello! Help please. What should i do? download numPy by muself? Traceback (most recent call last): File "X:\Users\Public\Documents\Instant-NGP-for-GTX-1000\scripts\colmap2nerf.py", line 16, in import numpy as np ModuleNotFoundError: No module named 'numpy'
Did you do the commands I give in the show notes, at the bottom? The "pip install -r requirements.txt" I think should help you get set up. But I'm a Python baby. See my notes for places you can get help from people who know much more than I do.
@@Nibot2023 I'll repeat my reply, in case you missed it. Me, I didn't have this problem - numpy installed OK for me. Did you do the commands I give in the show notes, at the bottom? The "pip install -r requirements.txt" I think should help you get set up. But I'm a Python lightweight. See my notes for this video for places you can get help from people who know much more than I do.
@@MrErichaines I saw it. I get this. >>> pip install -r requirements.txt File "", line 1 pip install -r requirements.txt ^^^^^^^ SyntaxError: invalid syntax >>> Hence why I asked. I know nothing about python lol. I will keep digging.
@@Nibot2023 Huh. Yeah, I know very little about Python, too. Is "pip" installed? Try typing just 'pip" and see if you get a long Usage message. You do need to run this line from in the instant-ngp directory itself. For Windows, you could also try "py -m ensurepip --upgrade" to make sure pip's there and the latest. pip.pypa.io/en/stable/installation/
hi, i have a error when i run "python D:\python\instant-ngp\scripts\colmap2nerf.py --video_in 1.MOV --video_fps 2 --run_colmap --overwrite" 1.MOV: No such file or directory
thank you, i replaced"video_in 1.MOV" with "video_in D:\python\instant-ngp\scripts\1.mov". it's working but now is have a new error when run #Camera match part,"F1107 14:03:41.730525 9384 cache.h:132] Check failed: max_num_elems > 0 (0 vs. 0)" @@MrErichaines
@@xiaochenyuan-jg1zv Googling, I found this: github.com/colmap/colmap/issues/1195 - see if that helps. Honestly, I am not a COLMAP expert, I just use a few basic commands.
I couldn't really say. It varies from country to country and depends on the status of the model being recreated. A good place to start: en.wikipedia.org/wiki/Threshold_of_originality#Reproductions
@@MrErichaines Thanks for the reply. Is it correct that this software itself does not have a license like vs code or ue5, and as long as the rights of the subject can be cleared, it can be used for free and commercial use?
@@たかしたかし-d7s Read the license: github.com/NVlabs/instant-ngp/blob/master/LICENSE.txt - Section 3.3 looks to me like it can only be used non-commercially
Is there any way to crop the nerf so that only the background remains and we crop/delete the center? In other words remove the center item and leave the background.
Not with this software. The whole topic of editing NeRFs scenes is definitely an area some researchers have tackled - things are evolving quite rapidly, there's just a huge amount of work coming out now.
Thanks for the detailed video! I am following it step by step, but I am having some trouble during the image extraction. I get the following warning before feature extraction: "Your current options use the maximum number of threads on the machine to extract features. Exracting SIFT features on the CPU can consume a lot of RAM per thread for large images. Consider reducing the maximum image size and/or the first octave or manually limit the number of extraction threads. Ignore this warning, if your machine has sufficient memory for the current settings." and then the following error at the end: "ERROR: Not enough GPU memory to match 9964 features. Reduce the maximum number of matches. ERROR: SiftGPU not fully supported ". I am using an NVidia 2060 videocard. Experimented with different image resolutions, smaller amount of images, reduced the aabb_scale parameter, but nothing works. Would be great if you could advise me on where could the problem be?
I never ran into this error, luckily. Googling "ERROR: SiftGPU not fully supported", there are a few hits. There's an error logged on COLMAP itself here github.com/colmap/colmap/issues/1019 - There seems to be some way to compile and run COLMAP without the GPU (takes much longer, I would guess, but that should solve it). Poking a bit more, I see on colmap.github.io/faq.html?highlight=sift#available-functionality-without-gpu-cuda some info on how to run without SiftGPU and how to reduce memory needed. This one groups.google.com/g/colmap/c/vP0EVupIJeI also seemed relevant, mentioning doing what you're doing already: "I don't think your card model is the problem, but it has "only" 6Gb of memory. According to the size of your images, and the maximum features to be found, you may have reached the limits of the GPU memory especially for the allocation of the matrix used in the second step of feature matching. Have a look at the max_image_size, max_num_features, and max_num_matches parameters in feature extraction and feature mapping steps. Lowering them may help." You might try poking at the other pages Google finds.
hi, when I run instant-ngp, it's working but it's not working on the GPU and it's quit long. I have an RTX 2080, I use the specific instant npg for RTX 2k series. Do you have an idea how I can figure this out ?
I am afraid I don't know. Please look at the notes for this video (click "Show more"), as I give a few websites where you can post your question and get help.
That doesn't ring a bell for me. I have seen things get a little out of sync if you click the box for looping animation, so I tend to click that as the last thing before making an animation. In the video's notes section I point to some places where you can get more support. Please do give them a try - reporting bugs helps the software get better, for sure.
Ha, and sorry, I know what you mean. Really, the stuff in the video description is pretty vital! I just looked now to see if there was some easy way for me to add "hey, look at the notes below" in the video, but I couldn't find anything in their editing tools. Frustrating!
Cool vid, but I have a problem. When typing "conda activate ngp", I get an error message : "EnvironmentNameNotFound: Could not find conda environment: ngp" Any ideas, please ?
Right, you have to make your own ngp environment. See the notes below this video. At the very bottom you'll see how to create this ngp environment, it's just a few lines that you need to run just once. After that you can do "conda activate ngp" any time you start a new Conda session.
That's easier still: just put all your pictures in the directory where the video's images get extracted to, e.g., "Chesterwood\images" is where all the JPEGs get put in the video's example.
Yes, and that's even easier: you don't need to do the step "Pull images from movie" shown in the notes below the video. You already have the images. Just put your images in a subdirectory called "images" and run the "camera match" command in the directory above it. You can see this process directly in my older video: th-cam.com/video/8GbENSmdVeE/w-d-xo.html
@@DaSmokeDaddy I'd like to, but wasn't able to get it compiled on my system when I tried in October. I should try again - instant-ngp is one of a number of systems it manages.
@@DaSmokeDaddy Like I said before, the colmap2nerf.py process just makes a directory of images that it extracts from the video file. So, if you instead have a set of images, in your imagination act like you just extracted these from a movie. Images are images. So, whatever you're doing once you extract these images from a video, just do that same thing with a separate set of images. Note, also, that in this video's notes I list a few places for further help with instant-ngp. Like I say, I'm just an intermediate user at best. You're using the software in a way I'm not familiar with. So I'm not sure I can be of much more help.
Brilliand tutorial, followed all up to render video, my version didnt have that option, i could set all the cameras but didnt have that section ? any ideas, is there a switch, cheers
The "one weird trick": make sure to resize the "Camera path" UI window. A lot. And even close the "Path Manipulation" section by clicking on the little arrow to the left of it. You should then see a green "Render video" button at the bottom, with various options below it.
Is there a video how to set it up? I downloaded the files and the zip for 4000 card.. i did not get the beginning how to set up anaconda... thanks for the video btw. it is very appreciated. 🙂 subbed
I just downloaded "Instant-NGP-for-RTX-3000-and-4000.zip" and instant-ngp.exe is in directory Instant-NGP-for-RTX-3000-and-4000 once you unzip it. Your system may be hiding suffixes, in which case "instant-ngp" is the file you want.
@Eric Haines yeah I recognized that I looked in the wrong folder. My bad. Do you know how to activate dlss? I get an error that Vulkan is missing. But I downloaded Vulkan SDK.. I don't know how to get it ready
@@pandadayi Vulkan SDK should be enough, I'd hope. Maybe reboot, to make sure everything is properly linked? DLSS is an option that should be visible in instant-ngp, find it under the Rendering options in the main UI window (you may have to resize this window). Turning it on doesn't change the demo fox's look much, but it definitely has an effect on other scenes. And, IIRC, DLSS is just used for interactive viewing - you really don't need it for making animations, as that does enough supersampling on its own that DLSS is not useful. (I could be wrong.)
@@MrErichaines yeah, i compiled instant-ngp again and this time it recognized vulkan and now i can use DLSS. looks much better now, but i am also not shure if it is only for preview. The better the input the better the results, so i have to experiment with some cameras and lenses.
Just like the last one, this is not about animation. The title is misleading. if you've found this looking for how to animate a nerf, you're at the wrong spot. This is just about camera movement. I appreciate the effort and thank you for the video, but an animation requires something to be animated. There is nothing animated here, it's only camera movement. Have my Like regardless.
Hi! Thanks for this excellent helpful video. You guys are the best. I got a question. When run python C:..\scripts\colmap2nerf.py --colmap_matcher exhaustive --run_colmap --aabb_scale 16 --overwrite, I am getting the following error: ==== running: C:...\COLMAP.bat feature_extractor --ImageReader.camera_model OPENCV --ImageReader.camera_params "" --SiftExtraction.estimate_affine_shape=true --SiftExtraction.domain_size_pooling=true --ImageReader.single_camera 1 --database_path colmap.db --image_path "images" [option_manager.cc:811] Check failed: ExistsDir(*image_path) ERROR: Invalid options provided. FATAL: command failed Does anyone know how this could be resolved?
Googling, it sounds like it's this error: github.com/bycloudai/instant-ngp-Windows/issues/5 - check your directory structure, I think. Maybe if you explicitly do this it will work, e.g., --images full-path-to-your-images-directory
@@MrErichaines Thanks very much, dear Eric! It worked! This is really amazing! I was wondering if you could also let me know what the output types of these amazing models are (before rendering)? Are they 3D point clouds (location plus (x,y,z) RGB values)? Are they location plus a vlaue for Opacity?
@@Sina.Davari whew, good! Thanks for letting me know. The model is sort of voxels with little neural nets at each that take in a ray direction and location and return an RGBA, for each voxel along the ray. At least, that's my hand-wavy understanding. The paper itself, as well as the SIGGRAPH 2024 NeRF course notes, are the best guides.
Always love seeing some progress like this. Well done!
I got this going with the help of your video. Much appreciated! I had a couple of components that needed to be installed and some errors I didn't really understand but I figured it out in the end :)
Yeah, it's a bit of a minefield - I'm glad I could offer a little help
This is an excellent video, many thanks for taking the time to make this and documenting it so well in the comments :-)
Exciting work, fantastic, unbelievable!!!
Great video. Thanks Eric!
Thanks a lot for video, you're the best
when i run interactive instant-ngp, error occured:
(ngp) D:\instant-ngp\Instant-NGP-for-GTX-1000>D:\instant-ngp\Instant-NGP-for-GTX-1000\instant-ngp.exe test
14:35:25 SUCCESS Initialized CUDA 11.5. Active GPU is #0: NVIDIA GeForce GTX 1660 Ti [75]
14:35:25 INFO Loading NeRF dataset from
14:35:25 INFO test\transforms.json
14:35:25 ERROR Uncaught exception: No training images were found for NeRF training!
after the Camera match part, the json file and colmap_sparse and colmap_test not in the same folder as the image and.MOV. colmap.dp which showed in your video, i can not find it
Sounds like a path error: "No training images were found for NeRF training!" I would suggest seeing my video notes and go to the links for help where I put "Problems? For help, see the Discussion area..." - I can't provide much support for instant-ngp; I don't know much about the errors generated (it works for me).
wow! this is incredible. So if I take photos of a subject from different angles then essentially I can turn that into a video.
Exactly, if all goes well (right now you sometimes get "fluffies"). And the tech will just get better as more research is done and put out there.
@@MrErichaines i wonder if one can extract the tracking data and then use that in say blender to place a 3D object in scene.
Thanks for making this!
Kind words, thanks!
my video outputs have a weird distortion effect like its using a fisheye lens, do you know how to fix? you can see it in my uploads
I see that in your one upload - I'm afraid I don't know why, if the input video isn't using a fisheye. Your witch's castle doesn't seem to have the problem.
Thanks so much SIr!
I've tried but it is not working on the custom dataset, i captured video from iphone 14 pro max and extracted upto 400 images and changed the dale from 16 to 32 but still it is not giving me the nerf.It's whole cloudy and i dont know, i need your help in this?
I can't really say. Some nerfs just come out full of "clouds", things converging badly. Dimly lit places seem to have this problem more, but it's inconsistent. I am much less of an expert than you might think. My advice, as always, is to check the text below this video and go try the Discord server etc. I mention there. You might also give NerfStudio a try (I still need to get that one going myself).
@@MrErichaines right Sir!
I'll try using images, then inform accordingly,
Thanks
Can I export OBJ file from this?
Thank you in advance
There's an Export Mesh at the bottom of the main overlay menu (if you don't see it, scroll the menu on the right edge). Open that section, Mesh it! and Save it! are the options. Don't expect a great mesh - quality definitely varies with how well the NeRF was created.
Fantastic Video. Thanks
Can you upload a video showing how to build new instant-ngp instance
Really, there's little reason to build instant-ngp from scratch - just download the correct executable. I do have some build tips in the video notes for th-cam.com/video/8GbENSmdVeE/w-d-xo.html, in case they're a help.
any experience doing this with insta360 footage?
I haven't tried. Googling about, I do see some others have worked on this. Give it a look: github.com/NVlabs/instant-ngp/discussions/1200
The quality of rendering at any image resolution, no matter how much additional training is expected, to put it mildly, is disgusting. Is it possible to make a video to make 512 * 512 exclusively for viewing on a smartphone and in no case watch on a computer =) And since it comes to that, there is also Nerf-Studio, there are *.bat files to which you simply transfer the video or a folder with images, and it the whole whirlwind of operations itself produces and opens an already rendered scene.
Nerf Studio looks cool (and uses instant-ngp as one of its engines). Unfortunately, I've never been able to compile it - it balks on me having Visual Studio 2022 installed, last I tried. So, I'm glad you've gotten it to run for you. I still love what they're doing, fixing a number of typos when they first announced: github.com/nerfstudio-project/nerfstudio/pull/712 . I should give it another try, maybe it will compile for me now.
Hello! Can you please help me? I have 2 Quadro GV100, and if I want to create a 3D model using 150 photos by 8mb each, I have an error
“Uncaught exception: FullyFusedMLP: insufficient shared memory available on the GPU. Reduce ‘n_neurons’ or use ‘CutlassMLP’ (better compatibility but slower) instead”
This is Volta video card. Does the NGP supports Volta type of video cards ?
I've never seen that message before, but I'm not a pro when it comes to this software. I highly recommend trying the support forums I give links to in my notes for this video.
@@MrErichaines thank you man
Hello, I'm starting from scratch (no programming knowledge or software installed), I tried installing all the required software but I keep seeing new requirements every time I try to run the process. I can extract the images from my video, but the process stops after that without being able to generate the camera path file. I get a series of error messages saying I'm missing the following files : boost_filsystem-vc143-mt-x64-1_77.dll, boost_program_options-vc143-mt-x64-1_77.dll, FreeImage.dll, ceres.dll. I have no idea what those are or what I should install. I'veinstalled visual Studio, the CUDA software, Anaconda, COLMAP, FFMPEG, and I don't know what else.
I'd appreciate some help. Thanks
Yikes! I'm sorry, I have no idea what these problems are about. I tried searching on the issue at github.com/NVlabs/instant-ngp/issues but didn't get any hits. Personally, at this point I download the proper EXE from github.com/NVlabs/instant-ngp and it just works for me. I was also able to build it from scratch without much fuss; I guess I already had the DLL's needed. Hmmm, if I were you, I'd consider maybe trying Nerfstudio docs.nerf.studio/. Or maybe try the new hotness, 3D Gaussian Splats, github.com/graphdeco-inria/gaussian-splatting (definitely a bit of work getting that going). Easier still, Polycam and Luma are things you might check out (I haven't looked in awhile): poly.cam/tools/gaussian-splatting and lumalabs.ai/interactive-scenes - hope this helps, and sorry I can't help more on Instant NGP.
@@MrErichaines Thank you for replying! In the meantime, I discovered that those Dlls come with the Colmap software, but when using the colmap2nerf script they don't get called. I haven't had the chance to try again yet but next thing I'll do will be to just dump all the Dlls in the same folder with the script and hopefully it will work!
@@sloenmusic thanks for the followup as to what was going on - good to know, and hopefully this'll help someone else. Best of luck getting things going! Instant NGP really is fun (and fast) once you get it going.
How can I activate Instant NGP? I'm able to activate NerfStudio, which I've previously installed. Instant NGP is in my Downloads Folder - I've changed location, copied path of .exe, and it's not working. (Noob to Python)
I don't know how to hook up NGP to NerfStudio (I've tried to use NerfStudio but haven't been successful yet). You can run Instant NGP alone without it (as I do in this video). Just download the right one and run it, basically: github.com/NVlabs/instant-ngp - see the notes for this video for the basic commands you need.
Thanks@@MrErichaines
Hello! Help please. What should i do? download numPy by muself?
Traceback (most recent call last):
File "X:\Users\Public\Documents\Instant-NGP-for-GTX-1000\scripts\colmap2nerf.py", line 16, in
import numpy as np
ModuleNotFoundError: No module named 'numpy'
Did you do the commands I give in the show notes, at the bottom? The "pip install -r requirements.txt" I think should help you get set up. But I'm a Python baby. See my notes for places you can get help from people who know much more than I do.
I am getting the same issue. Did you resolve this?
@@Nibot2023 I'll repeat my reply, in case you missed it. Me, I didn't have this problem - numpy installed OK for me. Did you do the commands I give in the show notes, at the bottom? The "pip install -r requirements.txt" I think should help you get set up. But I'm a Python lightweight. See my notes for this video for places you can get help from people who know much more than I do.
@@MrErichaines I saw it. I get this. >>> pip install -r requirements.txt
File "", line 1
pip install -r requirements.txt
^^^^^^^
SyntaxError: invalid syntax
>>>
Hence why I asked. I know nothing about python lol. I will keep digging.
@@Nibot2023 Huh. Yeah, I know very little about Python, too. Is "pip" installed? Try typing just 'pip" and see if you get a long Usage message. You do need to run this line from in the instant-ngp directory itself. For Windows, you could also try "py -m ensurepip --upgrade" to make sure pip's there and the latest. pip.pypa.io/en/stable/installation/
hi, i have a error when i run "python D:\python\instant-ngp\scripts\colmap2nerf.py --video_in 1.MOV --video_fps 2 --run_colmap --overwrite" 1.MOV: No such file or directory
I'm afraid I don't know, I've never run into that error. The obvious thing: is 1.MOV in the directory? Googling, I didn't see much else.
thank you, i replaced"video_in 1.MOV" with "video_in D:\python\instant-ngp\scripts\1.mov". it's working but now is have a new error when run #Camera match part,"F1107 14:03:41.730525 9384 cache.h:132] Check failed: max_num_elems > 0 (0 vs. 0)"
@@MrErichaines
@@xiaochenyuan-jg1zv Googling, I found this: github.com/colmap/colmap/issues/1195 - see if that helps. Honestly, I am not a COLMAP expert, I just use a few basic commands.
What about the licensing of the videos and models made with this? Can they be used for commercial purposes?
I couldn't really say. It varies from country to country and depends on the status of the model being recreated. A good place to start: en.wikipedia.org/wiki/Threshold_of_originality#Reproductions
@@MrErichaines Thanks for the reply. Is it correct that this software itself does not have a license like vs code or ue5, and as long as the rights of the subject can be cleared, it can be used for free and commercial use?
@@たかしたかし-d7s Read the license: github.com/NVlabs/instant-ngp/blob/master/LICENSE.txt - Section 3.3 looks to me like it can only be used non-commercially
@@MrErichaines I understand. Thank you kindly.
Does anyone know if this can produce a simple 3D model as well? As example an stl or obj file for 3D printing
As I recall, there's an OBJ export function in the program.
Why nvidia hasn't though about apple users , some of your tech new producs doesn't run on mac , for instance nvidia broadcast and rtx?
Is there any way to crop the nerf so that only the background remains and we crop/delete the center? In other words remove the center item and leave the background.
Not with this software. The whole topic of editing NeRFs scenes is definitely an area some researchers have tackled - things are evolving quite rapidly, there's just a huge amount of work coming out now.
Thanks for the detailed video! I am following it step by step, but I am having some trouble during the image extraction. I get the following warning before feature extraction: "Your current options use the maximum number of threads on the machine to extract features. Exracting SIFT features on the CPU can consume a lot of RAM per thread for large images. Consider reducing the maximum image size and/or the first octave or manually limit the number of extraction threads. Ignore this warning, if your machine has sufficient memory for the current settings." and then the following error at the end: "ERROR: Not enough GPU memory to match 9964 features. Reduce the maximum number of matches.
ERROR: SiftGPU not fully supported ".
I am using an NVidia 2060 videocard. Experimented with different image resolutions, smaller amount of images, reduced the aabb_scale parameter, but nothing works.
Would be great if you could advise me on where could the problem be?
I never ran into this error, luckily. Googling "ERROR: SiftGPU not fully supported", there are a few hits. There's an error logged on COLMAP itself here github.com/colmap/colmap/issues/1019 - There seems to be some way to compile and run COLMAP without the GPU (takes much longer, I would guess, but that should solve it). Poking a bit more, I see on colmap.github.io/faq.html?highlight=sift#available-functionality-without-gpu-cuda some info on how to run without SiftGPU and how to reduce memory needed.
This one groups.google.com/g/colmap/c/vP0EVupIJeI also seemed relevant, mentioning doing what you're doing already: "I don't think your card model is the problem, but it has "only" 6Gb of memory. According to the size of your images, and the maximum features to be found, you may have reached the limits of the GPU memory especially for the allocation of the matrix used in the second step of feature matching. Have a look at the max_image_size, max_num_features, and max_num_matches parameters in feature extraction
and feature mapping steps. Lowering them may help."
You might try poking at the other pages Google finds.
@@MrErichaines wow thanks for the quick and detailed reply! will be digging in soon
hi, when I run instant-ngp, it's working but it's not working on the GPU and it's quit long. I have an RTX 2080, I use the specific instant npg for RTX 2k series. Do you have an idea how I can figure this out ?
I am afraid I don't know. Please look at the notes for this video (click "Show more"), as I give a few websites where you can post your question and get help.
When I add my camera, it places it somewhere else, any idea why ?
That doesn't ring a bell for me. I have seen things get a little out of sync if you click the box for looping animation, so I tend to click that as the last thing before making an animation. In the video's notes section I point to some places where you can get more support. Please do give them a try - reporting bugs helps the software get better, for sure.
There needs to be a big, red, pinned comment for idiots like me, that says "READ THE DESCRIPTION, especially the end"
Ha, and sorry, I know what you mean. Really, the stuff in the video description is pretty vital! I just looked now to see if there was some easy way for me to add "hey, look at the notes below" in the video, but I couldn't find anything in their editing tools. Frustrating!
Cool vid, but I have a problem. When typing "conda activate ngp", I get an error message :
"EnvironmentNameNotFound: Could not find conda environment: ngp"
Any ideas, please ?
Right, you have to make your own ngp environment. See the notes below this video. At the very bottom you'll see how to create this ngp environment, it's just a few lines that you need to run just once. After that you can do "conda activate ngp" any time you start a new Conda session.
@@MrErichaines hey, thanks a lot. Will try that, once I get home. Cheers !
so how does one do it with pictures only?
That's easier still: just put all your pictures in the directory where the video's images get extracted to, e.g., "Chesterwood\images" is where all the JPEGs get put in the video's example.
Is it possible to import a NERF file into UNREAL5?
What are the output formats of NERF?
Googling, I see there's a Luma AI plugin. Right now there's no standard NeRF file format that I know of.
Can Instant NGP handle 'single' images instead of a video? If so, what is the process?
Yes, and that's even easier: you don't need to do the step "Pull images from movie" shown in the notes below the video. You already have the images. Just put your images in a subdirectory called "images" and run the "camera match" command in the directory above it. You can see this process directly in my older video: th-cam.com/video/8GbENSmdVeE/w-d-xo.html
@@MrErichaines Thanks again. Have you experimented with Nerfstudio?
@@DaSmokeDaddy I'd like to, but wasn't able to get it compiled on my system when I tried in October. I should try again - instant-ngp is one of a number of systems it manages.
@@DaSmokeDaddy Like I said before, the colmap2nerf.py process just makes a directory of images that it extracts from the video file. So, if you instead have a set of images, in your imagination act like you just extracted these from a movie. Images are images. So, whatever you're doing once you extract these images from a video, just do that same thing with a separate set of images. Note, also, that in this video's notes I list a few places for further help with instant-ngp. Like I say, I'm just an intermediate user at best. You're using the software in a way I'm not familiar with. So I'm not sure I can be of much more help.
@@MrErichaines Thanks for replying. I figured it out. My command prompt was incorrect when using my own images.
Brilliand tutorial, followed all up to render video, my version didnt have that option, i could set all the cameras but didnt have that section ? any ideas, is there a switch, cheers
The "one weird trick": make sure to resize the "Camera path" UI window. A lot. And even close the "Path Manipulation" section by clicking on the little arrow to the left of it. You should then see a green "Render video" button at the bottom, with various options below it.
Can we extract 3d model?
There's an option to perform marching cubes and get the export, but it's pretty noisy and nothing special.
hello，how can i export the 3d model to comsol or other software?
I don't know, but if you ask on the Discord server, Reddit, etc. listed in the video's notes, you may get an answer.
Is there a video how to set it up? I downloaded the files and the zip for 4000 card.. i did not get the beginning how to set up anaconda...
thanks for the video btw. it is very appreciated. 🙂 subbed
I just downloaded "Instant-NGP-for-RTX-3000-and-4000.zip" and instant-ngp.exe is in directory Instant-NGP-for-RTX-3000-and-4000 once you unzip it. Your system may be hiding suffixes, in which case "instant-ngp" is the file you want.
@Eric Haines yeah I recognized that I looked in the wrong folder. My bad. Do you know how to activate dlss? I get an error that Vulkan is missing. But I downloaded Vulkan SDK.. I don't know how to get it ready
@@pandadayi Vulkan SDK should be enough, I'd hope. Maybe reboot, to make sure everything is properly linked? DLSS is an option that should be visible in instant-ngp, find it under the Rendering options in the main UI window (you may have to resize this window). Turning it on doesn't change the demo fox's look much, but it definitely has an effect on other scenes. And, IIRC, DLSS is just used for interactive viewing - you really don't need it for making animations, as that does enough supersampling on its own that DLSS is not useful. (I could be wrong.)
@@MrErichaines yeah, i compiled instant-ngp again and this time it recognized vulkan and now i can use DLSS. looks much better now, but i am also not shure if it is only for preview. The better the input the better the results, so i have to experiment with some cameras and lenses.
Great!!! make a video with METASHAPE too. :)
Just like the last one, this is not about animation. The title is misleading. if you've found this looking for how to animate a nerf, you're at the wrong spot. This is just about camera movement. I appreciate the effort and thank you for the video, but an animation requires something to be animated. There is nothing animated here, it's only camera movement. Have my Like regardless.
Good point! Changed to "movie."
@@MrErichaines Thanks here as well!
Hi! Thanks for this excellent helpful video. You guys are the best. I got a question. When run python C:..\scripts\colmap2nerf.py --colmap_matcher exhaustive --run_colmap --aabb_scale 16 --overwrite, I am getting the following error:
==== running: C:...\COLMAP.bat feature_extractor --ImageReader.camera_model OPENCV --ImageReader.camera_params "" --SiftExtraction.estimate_affine_shape=true --SiftExtraction.domain_size_pooling=true --ImageReader.single_camera 1 --database_path colmap.db --image_path "images"
[option_manager.cc:811] Check failed: ExistsDir(*image_path)
ERROR: Invalid options provided.
FATAL: command failed
Does anyone know how this could be resolved?
Googling, it sounds like it's this error: github.com/bycloudai/instant-ngp-Windows/issues/5 - check your directory structure, I think. Maybe if you explicitly do this it will work, e.g., --images full-path-to-your-images-directory
@@MrErichaines Thanks very much, dear Eric! It worked! This is really amazing! I was wondering if you could also let me know what the output types of these amazing models are (before rendering)? Are they 3D point clouds (location plus (x,y,z) RGB values)? Are they location plus a vlaue for Opacity?
@@Sina.Davari whew, good! Thanks for letting me know. The model is sort of voxels with little neural nets at each that take in a ray direction and location and return an RGBA, for each voxel along the ray. At least, that's my hand-wavy understanding. The paper itself, as well as the SIGGRAPH 2024 NeRF course notes, are the best guides.
@@MrErichaines Great! Thank you so much, dear Eric. I really appreciate it.💐
any chance to export this as a mesh or point cloud or so?
ok I see the mesh it function as obj.. but how to map the textures on the mesh? hmm