Important BUG FIXES: I have updated the model list for all those who have received error messages (the new models are green). Download these models and add them to the folders. Let me know in the comments to this post if you still have any issues! For those who don't see the manager I have added another download link. Try to downloading / installing the manager from this link. Thanks for all your feedback and help with troubleshooting!
Error occurred when executing ACN_AdvancedControlNetApply: 'NoneType' object has no attribute 'copy' File "C:\Users\misha\Downloads\ComfyUI_windows_portable\ComfyUI\execution.py", line 152, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\misha\Downloads\ComfyUI_windows_portable\ComfyUI\execution.py", line 82, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\misha\Downloads\ComfyUI_windows_portable\ComfyUI\execution.py", line 75, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\misha\Downloads\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Advanced-ControlNet\adv_control odes.py", line 173, in apply_controlnet c_net = convert_to_advanced(control_net.copy()).set_cond_hint(control_hint, strength, (start_percent, end_percent)) ^^^^^^^^^^^^^^^^ i still get the error
Hey there, followed your advice, but i still get this error Error occurred when executing ControlNetLoaderAdvanced: Weights only load failed. Re-running `torch.load` with `weights_only` set to `False` will likely succeed, but it can result in arbitrary code execution.Do it only if you get the file from a trusted source. WeightsUnpickler error: Unsupported operand 118 File "C:\Users\tompi\Documents\ComfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 152, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\tompi\Documents\ComfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 82, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\tompi\Documents\ComfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 75, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\tompi\Documents\ComfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Advanced-ControlNet\adv_control odes.py", line 90, in load_controlnet controlnet = load_controlnet(controlnet_path, timestep_keyframe) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\tompi\Documents\ComfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Advanced-ControlNet\adv_control\control.py", line 512, in load_controlnet controlnet_data = comfy.utils.load_torch_file(ckpt_path, safe_load=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\tompi\Documents\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\utils.py", line 20, in load_torch_file pl_sd = torch.load(ckpt, map_location=device, weights_only=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\tompi\Documents\ComfyUI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\serialization.py", line 1039, in load raise pickle.UnpicklingError(UNSAFE_MESSAGE + str(e)) from None
@@tompilot4574 Can you go into the ComfyUI manager, install custom nodes and check if "ComfyUI's ControlNet Auxiliary Preprocessors" is installed? Then go to manager "Update All", restart and try again.
@@mickmumpitz hey there, thanks for the support , i updated the node you said, because it was already installed using the manager, but now i still get this error Requested to load SD1ClipModel Loading 1 new model ERROR:root:!!! Exception during processing !!! ERROR:root:Traceback (most recent call last): File "C:\Users\tompi\Documents\ComfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 152, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\tompi\Documents\ComfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 82, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\tompi\Documents\ComfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 75, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\tompi\Documents\ComfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Advanced-ControlNet\adv_control odes.py", line 90, in load_controlnet controlnet = load_controlnet(controlnet_path, timestep_keyframe) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\tompi\Documents\ComfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Advanced-ControlNet\adv_control\control.py", line 512, in load_controlnet controlnet_data = comfy.utils.load_torch_file(ckpt_path, safe_load=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\tompi\Documents\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\utils.py", line 20, in load_torch_file pl_sd = torch.load(ckpt, map_location=device, weights_only=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\tompi\Documents\ComfyUI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\serialization.py", line 1039, in load raise pickle.UnpicklingError(UNSAFE_MESSAGE + str(e)) from None _pickle.UnpicklingError: Weights only load failed. Re-running `torch.load` with `weights_only` set to `False` will likely succeed, but it can result in arbitrary code execution.Do it only if you get the file from a trusted source. WeightsUnpickler error: Unsupported operand 118 Prompt executed in 43.01 seconds
@@mickmumpitz Hey there ! i found a fix maybe? so Instead of having the "control_v11p_sd15_canny" in the box that sais "Load Advanced Control Net Model" i instead put the "control_v11f1p_sd15_depth" one inside this box and now its working... But you had the canny one in the box. I hope me choosing the depth one will not affect my output differntly than yours? is this the right thing to have here? because its the only thing that works in the box the canny one you had there doesnt work for me
Man you really have the best channel for this type of content everyone here just marketing a ai tools costing a lot of money and not usable things too you are the best ♥️
wow, this is an AMAZING tutorial. Very well explained, with workflow and list of models to download (plus their path position in the folder which more often than not isn't included, yet so important!). THANKS SO MUCH
it is a great tut to be sure - one of the most concise out there - but some of those linked models appear to be pickle tensors which are not as safe as safetensors?
This is the smoothest AI animation I've ever seen. You're amazing man. I have suggestion for the next video: 1. how to make animation based on your face, but the output is a different face (an anime girl face with horns for example) 2. How to capture the AI animation of just the actor (without background) by using green screen. thnks again btw.
You're amazing! Been looking for days now for something that's coherent without being completely overwhelming and this seems like a great guide! Thanks so much for the workflow...can't wait to try it out. Would you happen to a workflow version or have advice for how one might use a lora stack as well as facedetailer with your workflow? Thanks again!
Wow, what a wonderful tutorial! Do you know of some way to do prompt schedualing with this workflow? I've never been able to find a way to do so with SD1.5 animatediff.
Unfortunately, it doesn't work. I always get Error occurred when executing IPAdapterApply: Error(s) in loading state_dict for Resampler: size mismatch for proj_in.weight: copying a param with shape torch.Size([768, 1280]) from checkpoint, the shape in current model is torch.Size([768, 1024]). Doesn't really matter what kind of resolution my video has, it always halts on this step.
Can you download the new ClipVision (I updated the link in the drive document) Model and try it out again? It should work now! Make sure the new one is selected in the "Load CLIP Vision" node in the IPAdapter setup.
Thank you! I have a pretty old GPU (for this kind of stuff) with only 8 GB of VRAM (NVIDIA GeForce RTX 2070 SUPER). Video generation for the examples took between 2 and 15 minutes.
I suspect it wont be too long before we start to see Ai able to completely change a game's graphics on the fly, similar to how we can currently use a shader in Reshade for simple effects. We'll be able to completely change the look of a game in realtime, bring an old game up to modern graphical standards (or surpass), or retheme etc. Anyone want to take a guess at when we'll have consumer grade dedicated Ai chips that will perform at the level required for very low latency processing on a local machine? Perhaps it will initially be a cloud based service.
Hello thank you so much for the workflow it's working great i'm wondering if you can add an upscale step to fix some problems like eyes not showing correctly ?
hey mick i am able convert this perfectly can but i can only get 14 frames and if i increase the frames it gives me memory error i have ryzen 5 1500x 16gb ram rtx 2060 6gb if you dont mind can you tell me how i can convert atleast 30sec to 1 min clip in single shot it will be really helpful
Hey Mick I am able to run the whole thing but I am going through problems 1) sometimes it gives an error memory ran out so can you tell me what config I need to run this workflow 2)the results I am getting are not the same as the video you shared secondly I am getting only 1 second of the video I don't know why 3) I think you didn't update the adapter models and directory after the update (not confirmed about this) any help would be appreciated hoping for a reply 😀 I think you should make a second part of video since many things are changed
Error occurred when executing InsightFaceLoader: No module named 'insightface' could this be issue this is with just the IP adapter going to try a runpod to see If it solve the issue
Great tutorial thanks for sharing. Did you try using multiple ip adapters with attention masks? I figured the ip adapter plus model is great for anything else than the plus face or full face just for the face itself.
fantastic stuff. you're doing some great work. Thank you for your tut! one question some of your models used are pickletensors? I assume that using the safetensor versions / alternatives should be fine?
I've got IPAdapter error, workflow not usable. Checked the comments, they are mostly very vague. I think the reason is the wrong VAE model or one of the controlnets, I noticed that list of nodes is different, not the same as in your worlflow. Even comparing it to video, some names mismatch.
@@mickmumpitz Hi, thanks and great tutorial :))) but it gives me error, ("ClipVision model not found.") , maybe it should be renamed to some other name?
This workflow is very nice! Do you have any recommendations on extra tactics to help the eyes be more consistent? I feel like I'm getting a lot of eye drift throughout my animations, and turning on the eye and pupil settings in the face mesh setup you have really just messes up the output. Maybe another set of control nets or is there a way to run multiple masks -- one for the mouth, and one for the eyes?
Great video but i tried to download the tools and install it but the manager button left of the screen does not appear.. I do not have invidia card. so I click on run_cpu.bat but it opens but the manager do not appear. i tried several times. I tried also the new link you gave us but it does not make the manager appear. can you help me? thank you
Tried it but keep on getting this error: Error occurred when executing IPAdapterApply: Error(s) in loading state_dict for Resampler: size mismatch for proj_in.weight: copying a param with shape torch.Size([768, 1280]) from checkpoint, the shape in current model is torch.Size([768, 1024]). Any help please.
@@mickmumpitz I tried the new Clip Vision model and now I get this error: Error occurred when executing ACN_AdvancedControlNetApply: 'NoneType' object has no attribute 'copy'
@@magneticgrid7998 I updated the list again. Can you try it with the new ControlNet models that you'll find there? Thank you so much for helping me fix this!
Not working( What resolution for uploaded video? Error occurred when executing IPAdapterApply: Error(s) in loading state_dict for Resampler: size mismatch for proj_in.weight: copying a param with shape torch.Size([768, 1280]) from checkpoint, the shape in current model is torch.Size([768, 1024]).
Sorry if you are seeing multiple replies from me here. Its a checkpoint and clip vision model training size mismatch. Not to do with the video size you are using. I think my last posts were rejected or something. Use this Clip Vision model instead. You can find it on hugging face: IPAdapter_image_encoder_sd15
@@SimonePixel Thank you for your reply! I change it and get this message Error occurred when executing ADE_LoadAnimateDiffModel: 'control_sd15_canny.pth' is not a valid SD1.5 nor SDXL motion module - contained 0 downblocks.
this is super powerful was able to get fixed yeah wrong control net it looks really good all 3 on to doing stuff for my sons birthday but I notice lip sync is off I will review the video see if you mention but anyone know what is good to like possible get lips put them on it in realtime also was thinking be fun me walk around and attempt to talk script then AI voice change on it for a series start getting experimental with it
Hi, I like your workflow and your video but I need help understanding how to generate longer videos of 30 seconds or 1 minute, 5 minutes help me please
Error occurred when executing IPAdapterUnifiedLoader: ClipVision model not found. File "E:\comfy ui ew_ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 151, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "E:\comfy ui ew_ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 81, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "E:\comfy ui ew_ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 74, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "E:\comfy ui ew_ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_IPAdapter_plus\IPAdapterPlus.py", line 506, in load_models raise Exception("ClipVision model not found.")
Hey man, thanks for this incredible content, I'd love that so much, with my computer I can't handle with this much, my GPU is a GTX 1660Ti, 16GB ram, and I5 7th gen, but I bought a new PC with a RTX 4070 Super, how is the configuration of you PC? and, this workflow still work?
Great workflow. Is it possible to disconnect the character's look from the video, like using a video of you but prompt "Disney Princess" or is it always connected to the look of the person in the video?
it shows an error when I press queue prompt. How can I fix it? My VRAM is 8GB, everything has been put to the folder including the IPadapter. Error occurred when executing IPAdapterApply: Error(s) in loading state_dict for Resampler: size mismatch for proj_in.weight: copying a param with shape torch.Size([768, 1280]) from checkpoint, the shape in current model is torch.Size([768, 1024]).
Hi new sub here, your tutorial is very good, one question pls, what if i want the person in the video generated to have a different clothing, image style and background from another reference image but still has the same movement as the person in the video uploaded, like if i want to Animate a person from an image to move like the person in the uploaded video and also in a different background, pls do you understand 😢
Hey, I get this error: "No faces detected in controlnet image for Mediapipe face annotator." I made sure I installed the latest models on your word document. please advise!
Error occurred when executing IPAdapterApply: Error(s) in loading state_dict for Resampler: size mismatch for proj_in.weight: copying a param with shape torch.Size([768, 1280]) from checkpoint, the shape in current model is torch.Size([768, 1024]). does anyone know how I can fix this
Hi! it's working! Thanks.. But i want to ask about what if i have 1min up to 2min video? i understand i need to set the skip_first_frame.. but you said the frame_load_cap need's to be set in 15? how can i do batch by batch? can you enlighten me please?
Hello, I had a problem with a part and it gives me this error When loading the graph, the following node types were not found: Integer IPAdapterApply Nodes that have failed to load will show as red on the graph.
Thank you so much for this grat workflow; unfortu ately I have the same syntax error each time I try to run this workflow:(( SyntaxError: Unexpected non-whitespace character after JSON at position 4 (line 1 column 5) Do you have a solution for this ? Thanks
control_v2p_sd15_mediapipe_face, there is 4 files with the same name but different extensions, which extension should i download and put ComfyUI_windows_portable\ComfyUI\models\controlnet ???
quick question, in the load ip adapter, there is a "model name" where do i have to put the models to be shown and where do i download them? im using the version 3 of your WorkFlow
When loading the graph, the following node types were not found: Integer IPAdapterApply Nodes that have failed to load will show as red on the graph. I have this issue, i didn't find them in the manager...any help please?
Hi, I've got "Error occurred when executing KSamplerAdvanced: mixed dtype (CPU): expect parameter to have scalar type of Float" How can I fix this? Thank you!
Hi there :) Not yet dived into Your stuff, but I already ofcourse know that is great. Just wanted as also Blender Artist, and Cinematic Creator, etc for You to know that I really appreciate Your effort, knowledge and the will to share. I would totally use it, test it, and ask later maybe We can create something. One more time THX, too much things around Comfy and this channels make it into My direction
![Generate ENTIRE 3D SETS For Your Projects FAST! [FREE Blender + SDXL/FLUX Workflow]](http://i.ytimg.com/vi/U_NHijmIF3E/mqdefault.jpg)
Important BUG FIXES: I have updated the model list for all those who have received error messages (the new models are green). Download these models and add them to the folders. Let me know in the comments to this post if you still have any issues!
For those who don't see the manager I have added another download link. Try to downloading / installing the manager from this link.
Thanks for all your feedback and help with troubleshooting!
Error occurred when executing ACN_AdvancedControlNetApply:
'NoneType' object has no attribute 'copy'
File "C:\Users\misha\Downloads\ComfyUI_windows_portable\ComfyUI\execution.py", line 152, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\misha\Downloads\ComfyUI_windows_portable\ComfyUI\execution.py", line 82, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\misha\Downloads\ComfyUI_windows_portable\ComfyUI\execution.py", line 75, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\misha\Downloads\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Advanced-ControlNet\adv_control
odes.py", line 173, in apply_controlnet
c_net = convert_to_advanced(control_net.copy()).set_cond_hint(control_hint, strength, (start_percent, end_percent))
^^^^^^^^^^^^^^^^ i still get the error
Hey there, followed your advice, but i still get this error
Error occurred when executing ControlNetLoaderAdvanced:
Weights only load failed. Re-running `torch.load` with `weights_only` set to `False` will likely succeed, but it can result in arbitrary code execution.Do it only if you get the file from a trusted source. WeightsUnpickler error: Unsupported operand 118
File "C:\Users\tompi\Documents\ComfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 152, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\tompi\Documents\ComfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 82, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\tompi\Documents\ComfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 75, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\tompi\Documents\ComfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Advanced-ControlNet\adv_control
odes.py", line 90, in load_controlnet
controlnet = load_controlnet(controlnet_path, timestep_keyframe)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\tompi\Documents\ComfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Advanced-ControlNet\adv_control\control.py", line 512, in load_controlnet
controlnet_data = comfy.utils.load_torch_file(ckpt_path, safe_load=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\tompi\Documents\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\utils.py", line 20, in load_torch_file
pl_sd = torch.load(ckpt, map_location=device, weights_only=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\tompi\Documents\ComfyUI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\serialization.py", line 1039, in load
raise pickle.UnpicklingError(UNSAFE_MESSAGE + str(e)) from None
@@tompilot4574 Can you go into the ComfyUI manager, install custom nodes and check if "ComfyUI's ControlNet Auxiliary Preprocessors" is installed? Then go to manager "Update All", restart and try again.
@@mickmumpitz hey there, thanks for the support , i updated the node you said, because it was already installed using the manager, but now i still get this error
Requested to load SD1ClipModel
Loading 1 new model
ERROR:root:!!! Exception during processing !!!
ERROR:root:Traceback (most recent call last):
File "C:\Users\tompi\Documents\ComfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 152, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\tompi\Documents\ComfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 82, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\tompi\Documents\ComfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 75, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\tompi\Documents\ComfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Advanced-ControlNet\adv_control
odes.py", line 90, in load_controlnet
controlnet = load_controlnet(controlnet_path, timestep_keyframe)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\tompi\Documents\ComfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Advanced-ControlNet\adv_control\control.py", line 512, in load_controlnet
controlnet_data = comfy.utils.load_torch_file(ckpt_path, safe_load=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\tompi\Documents\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\utils.py", line 20, in load_torch_file
pl_sd = torch.load(ckpt, map_location=device, weights_only=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\tompi\Documents\ComfyUI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\serialization.py", line 1039, in load
raise pickle.UnpicklingError(UNSAFE_MESSAGE + str(e)) from None
_pickle.UnpicklingError: Weights only load failed. Re-running `torch.load` with `weights_only` set to `False` will likely succeed, but it can result in arbitrary code execution.Do it only if you get the file from a trusted source. WeightsUnpickler error: Unsupported operand 118
Prompt executed in 43.01 seconds
@@mickmumpitz Hey there ! i found a fix maybe? so Instead of having the "control_v11p_sd15_canny" in the box that sais "Load Advanced Control Net Model" i instead put the "control_v11f1p_sd15_depth" one inside this box and now its working... But you had the canny one in the box. I hope me choosing the depth one will not affect my output differntly than yours? is this the right thing to have here? because its the only thing that works in the box the canny one you had there doesnt work for me
A brilliant guide as always! Clear, concise... perfect. Thanks for pushing things forward as you always do, keep up the great work.
Man you really have the best channel for this type of content everyone here just marketing a ai tools costing a lot of money and not usable things too you are the best ♥️
I love how you've implemented these things, explained these things, and that you're willing to share them with us in the first place! ^‿^
you are my favorite AI TH-camr
Would love a comfy UI course for begginers
Are there any?
What are you looking to do with ComfyUI as a beginner?
That's what this is tbh ^^ But maybe start with image generation rather than videos
Thanks for the shout out! This is a brilliant approach to lip sync!!
Thank you! This would not have been possible without your fantastic work!
bro, you give every possible detail, you`re amazing!!
wow, this is an AMAZING tutorial. Very well explained, with workflow and list of models to download (plus their path position in the folder which more often than not isn't included, yet so important!). THANKS SO MUCH
it is a great tut to be sure - one of the most concise out there - but some of those linked models appear to be pickle tensors which are not as safe as safetensors?
ty, one of the few that is not selling web ai services, subscribed to see what more you can do
You are so good at making videos. And teaching us how to do it. Thank you for always sharing nice stuff with your audience.
Without any doubt, you are my favorite AI TH-camr.
thank u so much for Workflow sharing as free
😍
THIS IS WHAT I'VE BEEN LOOKING FOR! THANKS A LOT!!!!!
Thanks so much for sharing this!
very crisp and on point . thanks for video really great explanation
Thank you very much for the video!
absolutely phenomenal guide - the best!
Yet again amazing work from you!❤
awesome, second times I stumble upon your video. great content.
Great video explaining all the steps involved. Keep up the great work.
This is the smoothest AI animation I've ever seen. You're amazing man.
I have suggestion for the next video:
1. how to make animation based on your face, but the output is a different face (an anime girl face with horns for example)
2. How to capture the AI animation of just the actor (without background) by using green screen.
thnks again btw.
Stunning content !
This is great! 🙌🏻
You are a king 🙌
Thank you so much, this is insane 🔥🔥🔥
First (from playlist) 🙋♂ Great channel, keep it up!
the animatediff models got renamed. So if you load them you have to either rename them in the workflow or in the folders
Can you please make a video on how to upscale and enhance a video( i.e. add new details) like krea ai in stable diffusion?
You're amazing! Been looking for days now for something that's coherent without being completely overwhelming and this seems like a great guide! Thanks so much for the workflow...can't wait to try it out. Would you happen to a workflow version or have advice for how one might use a lora stack as well as facedetailer with your workflow? Thanks again!
Wow, what a wonderful tutorial! Do you know of some way to do prompt schedualing with this workflow? I've never been able to find a way to do so with SD1.5 animatediff.
hey bro good work. thank you!! 👍👌
thanks for share!
Is there a way to turn yourself in a specific character with a character sheet instead of a prompt
Unfortunately, it doesn't work. I always get Error occurred when executing IPAdapterApply:
Error(s) in loading state_dict for Resampler:
size mismatch for proj_in.weight: copying a param with shape torch.Size([768, 1280]) from checkpoint, the shape in current model is torch.Size([768, 1024]).
Doesn't really matter what kind of resolution my video has, it always halts on this step.
Can you download the new ClipVision (I updated the link in the drive document) Model and try it out again? It should work now! Make sure the new one is selected in the "Load CLIP Vision" node in the IPAdapter setup.
@@mickmumpitzThanks, man! Now it works like a charm!
@@AlexanderKitchenkoHi can you please tell me which pc setup are you using as I am going to buy a new computer to run this workflow.. thanks
An additionnal step to correct the eyes would be neat. I wonder if there's any tip out there
Try different words for the prompting, try using some more detailed descriptions
Amazing thanks a lot
This is so good..subscribed..by the way what GPU are you using? and how long it took ?
Thank you! I have a pretty old GPU (for this kind of stuff) with only 8 GB of VRAM (NVIDIA GeForce RTX 2070 SUPER). Video generation for the examples took between 2 and 15 minutes.
I suspect it wont be too long before we start to see Ai able to completely change a game's graphics on the fly, similar to how we can currently use a shader in Reshade for simple effects. We'll be able to completely change the look of a game in realtime, bring an old game up to modern graphical standards (or surpass), or retheme etc. Anyone want to take a guess at when we'll have consumer grade dedicated Ai chips that will perform at the level required for very low latency processing on a local machine? Perhaps it will initially be a cloud based service.
Yo this cool!
🔥 thanks
Looks swesome!
Hello thank you so much for the workflow it's working great i'm wondering if you can add an upscale step to fix some problems like eyes not showing correctly ?
Thank you..❤❤❤❤❤
hey mick i am able convert this perfectly can but i can only get 14 frames and if i increase the frames it gives me memory error
i have ryzen 5 1500x
16gb ram
rtx 2060 6gb
if you dont mind can you tell me how i can convert atleast 30sec to 1 min clip in single shot
it will be really helpful
Hey Mick I am able to run the whole thing but I am going through problems
1) sometimes it gives an error memory ran out so can you tell me what config I need to run this workflow
2)the results I am getting are not the same as the video you shared secondly I am getting only 1 second of the video I don't know why
3) I think you didn't update the adapter models and directory after the update (not confirmed about this)
any help would be appreciated hoping for a reply 😀
I think you should make a second part of video since many things are changed
Error occurred when executing InsightFaceLoader:
No module named 'insightface'
could this be issue this is with just the IP adapter going to try a runpod to see If it solve the issue
Great tutorial thanks for sharing. Did you try using multiple ip adapters with attention masks? I figured the ip adapter plus model is great for anything else than the plus face or full face just for the face itself.
fantastic stuff. you're doing some great work. Thank you for your tut! one question some of your models used are pickletensors? I assume that using the safetensor versions / alternatives should be fine?
I've got IPAdapter error, workflow not usable. Checked the comments, they are mostly very vague. I think the reason is the wrong VAE model or one of the controlnets, I noticed that list of nodes is different, not the same as in your worlflow. Even comparing it to video, some names mismatch.
Could you please try to download the CLIP VISION model again (there is now a new link in the document!) and replace it?
@@mickmumpitz Hi, thanks and great tutorial :))) but it gives me error, ("ClipVision model not found.") , maybe it should be renamed to some other name?
Wow!
This workflow is very nice! Do you have any recommendations on extra tactics to help the eyes be more consistent? I feel like I'm getting a lot of eye drift throughout my animations, and turning on the eye and pupil settings in the face mesh setup you have really just messes up the output. Maybe another set of control nets or is there a way to run multiple masks -- one for the mouth, and one for the eyes?
Hi I have a question please. Is this video too old to use or do U have a new/better version of the Idea of d workflow? I find great what U do really
thank you boss
Thank you, love your work :), btw Could you please tell me which model checkpoint or lora did you used for Zelda BoTW look? :)
Great video but i tried to download the tools and install it but the manager button left of the screen does not appear.. I do not have invidia card. so I click on run_cpu.bat but it opens but the manager do not appear. i tried several times. I tried also the new link you gave us but it does not make the manager appear. can you help me? thank you
i have same issue how to fix it
Error occurred when executing IPAdapterUnifiedLoader:
ClipVision model not found. 😭
Tried it but keep on getting this error:
Error occurred when executing IPAdapterApply:
Error(s) in loading state_dict for Resampler:
size mismatch for proj_in.weight: copying a param with shape torch.Size([768, 1280]) from checkpoint, the shape in current model is torch.Size([768, 1024]).
Any help please.
Can you try downloading the new Clip Vision model (I updated it in the drive document), switch it out in the folder and try running it again?
@@mickmumpitz I tried the new Clip Vision model and now I get this error:
Error occurred when executing ACN_AdvancedControlNetApply:
'NoneType' object has no attribute 'copy'
@@magneticgrid7998 I updated the list again. Can you try it with the new ControlNet models that you'll find there? Thank you so much for helping me fix this!
Not working( What resolution for uploaded video?
Error occurred when executing IPAdapterApply:
Error(s) in loading state_dict for Resampler:
size mismatch for proj_in.weight: copying a param with shape torch.Size([768, 1280]) from checkpoint, the shape in current model is torch.Size([768, 1024]).
Sorry if you are seeing multiple replies from me here. Its a checkpoint and clip vision model training size mismatch. Not to do with the video size you are using. I think my last posts were rejected or something. Use this Clip Vision model instead. You can find it on hugging face: IPAdapter_image_encoder_sd15
@@SimonePixel Thank you for your reply! I change it and get this message
Error occurred when executing ADE_LoadAnimateDiffModel:
'control_sd15_canny.pth' is not a valid SD1.5 nor SDXL motion module - contained 0 downblocks.
this is super powerful was able to get fixed yeah wrong control net it looks really good all 3 on to doing stuff for my sons birthday but I notice lip sync is off I will review the video see if you mention but anyone know what is good to like possible get lips put them on it in realtime also was thinking be fun me walk around and attempt to talk script then AI voice change on it for a series start getting experimental with it
NIce tutorial, How can I get video by frames in output?
bro your latest version is also not working..plz support (there are errors related to clip vision and ipadapter)
Hi, I like your workflow and your video but I need help understanding how to generate longer videos of 30 seconds or 1 minute, 5 minutes help me please
Were you able to get this workflow working?
Error occurred when executing IPAdapterUnifiedLoader:
ClipVision model not found.
File "E:\comfy ui
ew_ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\comfy ui
ew_ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\comfy ui
ew_ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\comfy ui
ew_ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_IPAdapter_plus\IPAdapterPlus.py", line 506, in load_models
raise Exception("ClipVision model not found.")
How can I solve this problem🥲
MediaPipe-FaceMeshPreprocessor
No module named 'mediapipe' : I got this error please help
Hey man, thanks for this incredible content, I'd love that so much, with my computer I can't handle with this much, my GPU is a GTX 1660Ti, 16GB ram, and I5 7th gen, but I bought a new PC with a RTX 4070 Super, how is the configuration of you PC? and, this workflow still work?
Great workflow. Is it possible to disconnect the character's look from the video, like using a video of you but prompt "Disney Princess" or is it always connected to the look of the person in the video?
it shows an error when I press queue prompt. How can I fix it?
My VRAM is 8GB, everything has been put to the folder including the IPadapter.
Error occurred when executing IPAdapterApply:
Error(s) in loading state_dict for Resampler:
size mismatch for proj_in.weight: copying a param with shape torch.Size([768, 1280]) from checkpoint, the shape in current model is torch.Size([768, 1024]).
Got the same error
Hi new sub here, your tutorial is very good, one question pls, what if i want the person in the video generated to have a different clothing, image style and background from another reference image but still has the same movement as the person in the video uploaded, like if i want to Animate a person from an image to move like the person in the uploaded video and also in a different background, pls do you understand 😢
Hey, I get this error: "No faces detected in controlnet image for Mediapipe face annotator."
I made sure I installed the latest models on your word document.
please advise!
I'm running this with 2060 Super 8GB and 16GB RAM on my system. I always end up facing out of memory 😢
Great tutorial man, subbed as well. Just a question, can this convert longer length videos too like 3 4 min?
Thank you! It is possible, but of course it takes quite a long time if you don't have a good graphics card.
How much time it will take to render 1 min long clip
Nice
Pls help me urgent
Error occurred when executing IPAdapterAdvanced:
Missing CLIPVision model.
File "D:\ComfyUI_windows_portable\ComfyUI\execution.py", line 317, in execute
output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_windows_portable\ComfyUI\execution.py", line 192, in get_output_data
return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_windows_portable\ComfyUI\execution.py", line 169, in _map_node_over_list
process_inputs(input_dict, i)
File "D:\ComfyUI_windows_portable\ComfyUI\execution.py", line 158, in process_inputs
results.append(getattr(obj, func)(**inputs))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_IPAdapter_plus\IPAdapterPlus.py", line 767, in apply_ipadapter
raise Exception("Missing CLIPVision model.")
Error occurred when executing IPAdapterApply:
Error(s) in loading state_dict for Resampler:
size mismatch for proj_in.weight: copying a param with shape torch.Size([768, 1280]) from checkpoint, the shape in current model is torch.Size([768, 1024]).
does anyone know how I can fix this
It’s the wrong vae model. Look through the comments for a pointer towards the right one.
Son of a gun! Pew, pew! 💥🔫
Awesome! Thanks a lot to share your knowledge with the community
Haha yeah me too! Just go to settings (gear symbol next to the queue symbol) and set "Link Render Mode to "straight"
Hi! it's working! Thanks.. But i want to ask about what if i have 1min up to 2min video? i understand i need to set the skip_first_frame.. but you said the frame_load_cap need's to be set in 15? how can i do batch by batch? can you enlighten me please?
SyntaxError:Unexpected non-whitespace character after JSON at position 4 (line column 5 ....sir how can i fix it 😢😢😢😢
SyntaxError: Unexpected non-whitespace character after JSON at position 4 (line 1 column 5) HOW TO FIX THIS SIR THANK YOU
Would this work on something not filmed in real life? I mean using comfyUi on top of an animated film or game cutscene etc?
Hello, I had a problem with a part and it gives me this error
When loading the graph, the following node types were not found:
Integer
IPAdapterApply
Nodes that have failed to load will show as red on the graph.
1:52 it doesnt show up the button Manager here, do you know how to solve it ?
I'm having the same issue, the manager button is missing. But, I'll try it again and see what happens...
drage the icon(6 dots) on the left of "Queue size:0", now u can find "Manager" button...have fun...
amazing video, can you tell me the ps1 style model name please ' ?
I combined the two Lora’s ICBIN64 and ps1graphicsRW to get this look
Can we use something like this in Google colab?
Thank you so much for this grat workflow; unfortu ately I have the same syntax error each time I try to run this workflow:((
SyntaxError: Unexpected non-whitespace character after JSON at position 4 (line 1 column 5)
Do you have a solution for this ?
Thanks
would it make sense to add a loras or hypernetworks to this workflow as well? if so at what point would I add them?
Hmm can i run that in google colab?
control_v2p_sd15_mediapipe_face, there is 4 files with the same name but different extensions, which extension should i download and put ComfyUI_windows_portable\ComfyUI\models\controlnet ???
quick question, in the load ip adapter, there is a "model name" where do i have to put the models to be shown and where do i download them? im using the version 3 of your WorkFlow
why when a run "run_cpu" a don't have comfyUI manager ?
When loading the graph, the following node types were not found:
Integer
IPAdapterApply
Nodes that have failed to load will show as red on the graph.
I have this issue, i didn't find them in the manager...any help please?
is there no way yet to fix the goofy cross eyed look the ai does with the eyes?
You were able to get this workflow to work?
Can it be used for images not a video?
For tgose who hasn't a gpu pc, can we use this in google colab?
Hi, I've got
"Error occurred when executing KSamplerAdvanced:
mixed dtype (CPU): expect parameter to have scalar type of Float"
How can I fix this? Thank you!
Hi there :)
Not yet dived into Your stuff, but I already ofcourse know that is great. Just wanted as also Blender Artist, and Cinematic Creator, etc for You to know that I really appreciate Your effort, knowledge and the will to share. I would totally use it, test it, and ask later maybe We can create something. One more time THX, too much things around Comfy and this channels make it into My direction
Bro my IPAdapter is updated and now showing red colour, through which node I should replace the old IPAdapter Apply node?
Make sure to download the newest version of the workflow! (v03)
workflow show integer missing error ,how solved it?
Download the newest version