Convert single image to 3D model with AI in ComfyUI with CRM, for GPU & CPU
ฝัง
- เผยแพร่เมื่อ 16 มี.ค. 2024
- In this tutorial I walk you through a basic CRM workflow in ComfyUI. CRM stands for Convolutional Reconstruction Models and is now usable with ComfyUI. With CRM you can create simple 3D models from single images. It is usable with GPU and CPU. It is quite hungry on the RAM.
Be sure to update your ComfyUI to the newest version and install the needed models and custom nodes which are linked below.
The installation steps for the CRM nodes are described on their Github page.
Please comment if you have questions or tell me your suggestions for future videos.
CRM Models:
huggingface.co/Zhengyi/CRM/tr...
Custom Nodes:
github.com/flowtyone/ComfyUI-...
github.com/cubiq/ComfyUI_esse...
More infos on CRM:
ml.cs.tsinghua.edu.cn/~zhengy...
Connect with me:
/ _neuron_ai
/ _neuron_ai
The equipment that I use (affiliate links):
GPU: ASUS ROG STRIX RTX 4090 amzn.to/42ptUTi
#comfyui #stablecascade #stablediffusion #tutorial #animatediff #automatic1111 #aiart #ai #inpaint - วิทยาศาสตร์และเทคโนโลยี
thanks for the excellent video. please consider making a video on how to use CRM without comfyui, as the CRM nodes available are being unreliable to some people. i for one had success with following your tutorial but then the next day the workflow was giving me the nvdiffrast not found error which many people face and there is no consistent solution for.
Great idea. Will look into this. Its a pitty that CRM is not nicely implemented so far.
Have you run into the error about tensors? "Sizes of tensors must match except in dimension 3. Expected size 256 but got size 64 for tensor number 1 in the list." Have seen this particular issue across several different models and unsure how to resolve it. I have input a 256x256 image but unsure if that is related.
There's a lot of info packed in your videos and have watched a few. A quick tip, I find that I have to listen on average 1m to 2m to get an idea what the video is about, sometimes I am well into a video before I know if I have to download any custom nodes, models etc. Would you be able to front load that in your videos, just so that we can follow the video?
Thanks for watching. And thanks for your question. Great idea. I will make clear what is needed in the beginning of every video from now on.
I have an error, when the CRMPoseSampler is executed : "Error occurred when executing CRMPoseSampler:
The shape of the 2D attn_mask is torch.Size([77, 77]), but should be (1, 1)." Have you got an idea to resolve it ?
I am getting the same error
Looking about how ti increase texture resolution. It's only 256x256
I am not aware if this is possible at this early stage of development. But it would be great and there might be a way. But I am not sure how ATM.
Is there a way to export?
The readme mentions obj, but it's unclear on the method.
I am not able to check right now. But it should put a mesh file into your output folder.
Hi,good sir,do you mind send me a link of your comfyui installation guide? thank you
There is no video like this so far. I plan to make a step by step series in the future.
hello, what is the diffirence from triposr?
The main difference is that triposr uses LRM. This is a different approach then CRM. yiconghong.me/LRM/ will do a video on triposr as well. regarding the quality I am not yet sure which one is best. WIll compare them all soon.
@@neuron_ai thanks.
Getting this error when trying with CPU node
Error occurred when executing CRMModeler:
Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (when checking argument for argument grid in method wrapper_CUDA__grid_sampler_2d)
Thats odd! Do you use the CRM Modeler node and not the CRM Modeler (Cuda only) node? And in the RemBG session node choose CPU as Provider!
@@neuron_ai NOT using the cuda only node. And I was using an already made PNG cutout of a cat, was not removing backgrounds at that moment.
Hey, I have RTX 3070, 32GB of RAM and 8GB of VRAM, and when I run the node « CRM modeler » I have error message « CUDA : out of memory » is this normal. When I put 6 step the error is always here
Sometimes other apps use lots of vram too. so be sure not using other apps at the same time. sometimes this happens to me with workflows that I had no problems with before. sometimes restarting comfy or even the pc can help as well.
Has anyone had the this error? ModuleNotFoundError: No module named 'kiui'
did you try to install it with pip install kiui
@neuron_ai yes, first thing I tried!
Does this work w AMD GPUs or NVDA only?
I dont know. Seems the github is only talking about cuda it might only be usable on nvidia gpu and cpu.
Can you please help me fix this error? When my ComfyUI loads the CRM Pose Sample nodes, it gives the following error "Error occurred when executing CRMPoseSampler:
module 'open_clip' has no attribute 'create_model_and_transforms'
File "C:\Users\Admin\pinokio\api\comfyui.git\app\execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "C:\Users\Admin\pinokio\api\comfyui.git\app\execution.py", line 81, print get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "C:\Users\Admin\pinokio\api\comfyui.git\app\execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "C:\Users\Admin\pinokio\api\comfyui.git\app\custom_nodes\ComfyUI-Flowty-CRM\__init__.py", line 77, print sample
stage1_model = instantiate_from_config(OmegaConf.load(stage1_model_config.config).model)
File "C:\Users\Admin\pinokio\api\comfyui.git\app\custom_nodes\ComfyUI-Flowty-CRM\crmlib\imagedream\ldm\util.py", line 90, in instantiate_from_config
return get_obj_from_str(config["target"])(**config.get("params", dict()))
File "C:\Users\Admin\pinokio\api\comfyui.git\app\custom_nodes\ComfyUI-Flowty-CRM\crmlib\imagedream\ldm\interface.py", line 51, prints __init__
self.clip_model = instantiate_from_config(clip_config)
File "C:\Users\Admin\pinokio\api\comfyui.git\app\custom_nodes\ComfyUI-Flowty-CRM\crmlib\imagedream\ldm\util.py", line 90, in instantiate_from_config
return get_obj_from_str(config["target"])(**config.get("params", dict()))
File "C:\Users\Admin\pinokio\api\comfyui.git\app\custom_nodes\ComfyUI-Flowty-CRM\crmlib\imagedream\ldm\modules\encoders\modules.py", line 190, prints __init__
model, _, preprocess = open_clip.create_model_and_transforms"
It could be lots of things. did yoiu try to use my worklfow which is linked in the description? how did you install the crm custom node?
I installed and followed your instructions. I also use your process but when running to nodes "CRMPoseSampler"
"Error occurred when executing CRMPoseSampler:
module 'open_clip' has no attribute 'create_model_and_transforms'
Or is it because my pixel-diffusion.th model is corrupted?
Hey! thanks a lot for your video. I have followed it very carefully, but I have encountered an error just in the CRM modeler step. The Comfyui gives me this error:
"Error occurred when executing CRMModelerCuda:
No module named 'nvdiffrast'"
I have searched the way to install that missing module but I have not found any clear way. Could you give me some clue? thanks in advance
Thanks! Did you try to install nvdiffrast with "pip install nvdiffrast" ?
@@neuron_ai Hey! thanks for your quick response! :D
I have tried just now with "pip install nvdiffras" in a cmd window, but that gives me this other error "ERROR: Could not find a version that satisfies the requirement nvdiffrast (from versions: none)
ERROR: No matching distribution found for nvdiffrast"
...I`m using Python 3.11.8. Maybe I need some other pack or component?
Did you manage to get it working? I'm getting the same error with a 4080 on Win 10. I also installed the CUDA Toolkit, but still nothing
@@TheValkire Nop, sorry. I have left it for later because no luck in my reseach by the moment
i somehow alleviated this error but at the same time encountered "No module named nvdiffrast.torch" which is a different error i am still trying to fix