3D Gaussian Splatting full tutorial, crop, transform, mesh, import into UE5 & Collision
ฝัง
- เผยแพร่เมื่อ 11 มิ.ย. 2024
- A full in depth tutorial showing you how to create Gaussian Splats without common commercial tools and without having to pay or upload your images and data onto third party sites.
Instead I use repo-sam.inria.fr/fungraph/3d... to create a splat using python libraries, then show you how to use PlayCanvas and CloudCompare to edit, trim, crop, rotate, and mesh your splat, then Blender to decimate and import into unreal collisions using the free XV3DGS plugin.
You will need a powerful GPU and some basic knowledge of Python and terminal
Please comment on any issues ! Thanks for watching
Chapters:
Start 00:00
Making 3DGS with Python 01:26
Colmap and my dataset 04:01
Train Images 05:16
PlayCanvas Super-Splat 06:25
Unreal Plugin 08:49
CloudCompare 10:22
Meshing 13:13
Collision inside UE5 15:27
Links:
www.danielgm.net/cc/
github.com/xverse-engine/XV3D...
github.com/bycloud-AI/gaussia...
www.unrealengine.com/marketpl...
playcanvas.com/super-splat
www.unrealengine.com/marketpl...
#unrealengine
#gaussiansplatting
#tutorial
You`re a hero. It is very hard to find stuff in one place. Looking at gaussian splat as an actual working in-engine thing. I want to try using them with PCG , Thanks!
Thank you for putting together this walkthrough with all the details!
Thanks for this. PS. there is an easier way to setup the collision for this; in Unreal, open the static mesh (not the blueprint) and under collision select "use complex as simple"
Great video. That thing how you build the collision model was pretty smart. Thanks for the tip.
That was an excellent overview. Thanks a bunch!
Thanks a lot for the great tutorial 👍 . Also, in the awful native viewer of Gaussian splatting repo, if you use drop down to select trackball instead of FPS you can rotate, pan zoom etc but it is still quite awful
omg, that is pretty cool!
great video! Thank you
Amazing video !
thank you so much 👏
Gosh, what a god! Love you man
Great video kam , really great detail .
Tx buddy
Amazing thanks so much
Hey man, I recently saw your video on the pulse sensor. I wanted to ask you if you could do more tutorials with the serial comms plugin! Stuff like motion tracking via acceleration sensors is something that I think many people could be interested in :)
I love your videos man, keep it up!
thanks for tuto. 5.26 I made some tries with native resolution, but it was always crashing before the end of the train
Thank you so much.
Thanks! Amazing
nice tuotorial
Thanks!
Much appreciated
Cool!
This is very interesting! How does it compare to regular photogrammetry? Does it give more detailed meshes?
Outstanding! But as you said, gaussian or nerf without collision automation are useless or hard to work like you genius did. 😮
Hi there, really enjoying your tutorials, thank you. Is it possible i could contact you to ask some career advice?
hey s1 else got this error when giving the training command:
ModuleNotFoundError: No module named 'diff_gaussian_rasterization',
the rasterization folder is there under .git but i dont know how to resolve this error
Wow. But could you give an advice. What if I want to use mesh for generating shadow from GS splats? And what is there a simple way to texture mesh with splats projection to change lighting? Thank you )
Hi, could you advise me what to do if I get this information?:
(gaussian-splatting) C:\Users\agnie\pinokio\api\gaussian-splatting-Windows.git>python train.py -s input_data/fern
Traceback (most recent call last):
File "train.py", line 16, in
from gaussian_renderer import render, network_gui
File "C:\Users\agnie\pinokio\api\gaussian-splatting-Windows.git\gaussian_renderer\__init__.py", line 14, in
from diff_gaussian_rasterization import GaussianRasterizationSettings, GaussianRasterizer
ModuleNotFoundError: No module named 'diff_gaussian_rasterization'
Hello and thanks for your contribution.
To elaborate the model, I wonder how powerful the Home Computer needs to be? Thank you.
i sed it in the video but heres the just.
As far as the PC RAM probly not that important but i'd bet it would eat 16gig easy if anything is processed on the cpu. I would recomend myself a min of 32gig ram better if 64 and up so your not limited to just doing the one thing at a time.
Now for the important bit.
For the easiest install and setup.
You NEED an NVIDA card with 24 GigaBytes of Video memory. You can get away with less Video ram but it will lemit how detaild and large your splats will be and effect the speed of generation if the system has to swap data in and out of the card wich i dont think it can do. yet anyway.
Good luck.
I understand you have the invisible mesh, but is the visible object a point cloud running in the game live? Like could this run point cloud environments in VR if I put down an invisible collidable floor?
Hi, nice video. You export in format.stl, but its possible export the textures? the color data? like the old scan softwares?
im interested in this as well because if you could it would increase the performance for VR
Is there any update that simplify this or a equivalent workflow or app can do the same?
I believe that the repo you use (inria) has limitations of its own i.e. the software cannot be used for commercial purposes without a license from inria.
Switch from FPS to Trackball to pan. Also there are some keys like u,i,o or p work for panning and rotating? I never use them to be honest. Trackball works fine
Thank you I never bothered doing any looking into this just went with CC and playcanavs that are great for working with .ply and 3dgs ply
It it possible to turn those splats into *textured* meshes? I think I'd probably prefer working with them that way.
That's awesome. I didn't know you could mesh 3DGS. Can you export any textures with that mesh or is it too much to ask? 😂
As I understand it each point has a colour so it will be per polygon texture. I know blender allows you to bake these baked textures to a 2d image but I have not tried with this workflow and would not expect a high res texture
I was looking for that aswell lol
@@MrKams1 Interesting, I`m thinking in use that workflow to 3d modeling, like using a 3d in place to have as a reference, maybe can work even better that just some images , lets see lol
You should look into UCX collisions instead of complex ones
i wanted a complex one in this case as i wanted the projectiles i fired to go through the car door
Having issues with Pinokio closing itself after a few seconds of being open. Is this an issue with the latest dev build of windows 11?
seems like its stopping at installing something, sees simple but have tried running as admin, also see what it is installing and try manually. They have a discord afaik, give it a go?
I want to capture a room from the inside to watch it later in VR. But 3D Gaussian splatting seems to work best when capturing objects from the outside. My results with Polycam were truly ABYSMAL. How to do it?
Depends on the room and camera really. If there is lots of glass and windows it will fail. If it's empty model it, if it's full of objects any occluded areas behind objects will be hard to capture using anything other than lidar
Is that the death of 3d scanners in general?
I would not have thought so, 3D scanners still preferable for scanning smaller precision parts for CAD/manufacture etc, also lidar methods still preferable (with photogrammetry) for larger areas. These methods are simply faster (and cheaper) for VFX/Game/cinematic scans on a budget
Not by a long shot. :) There's no measure of scale in the splats just an estimation. Even with a reference the scale is going to have a huge range of error compared to a scanner.
Some of the highest end ones can get into the micrometer range and still be a few inches away from the surface it's scanning.
Promo_SM 💔