In general, watching similar YT tutorials by other people feels like mugging up each setting. Example: "Increase this, decrease that, keep this 0 or 1 and you'll do just fine." But that is not the case with you! I like the way you naturally interpret how the proportions of different settings affects the image and know the best blend to mix them in to obtain the desired results. Not only do you understand the 'how', but you are also able to 'communicate' it to your audience. I can watch this video four times, and still learn something new from it.
I know how much goes into the IPAdapter and making these tutorials! Really appreciate all of your efforts, because of your work I’ve been able to easily inject my own art into workflows with beautiful results. 😊
For the Encode IPAdapter Image node, I've seen other ComfyUI nodes that have a variable number of inputs. I.e., when you use up image-slot-2, then image-slot-3 appears, then image-slot-4 appears, etc. This is a more flexible than having a fixed number of inputs for your node.
to do that you have to install an additional javascript in a location outside the extension directory which is an anti-pattern. When comfyui will support adding scripts from inside the extension I'll surely add the option
It's so good that you made another video, the techniques are exactly what I need to learn and apply. I love your videos, thank you. I've had no issue following along but there is an issue I'm having with 'ip-adapter_sd15_light.bin' not showing up in the 'Load IP Adapter Model' node list. I have 6 out of 7 listed in the drop-down. It seems bizarre that only this one is not showing up. At first, I thought maybe I wouldn't use it and not worry but it will be interesting to use if I can identify the problem with my installation in the windows portable.
Great content. Some tutorials focusing just on IPAdapter + Animatediff + control net would be amazing. You touched on it a little . The ability to use 1 ref image to train style on a video sequence is of great appeal. Cheers
amazing! can't overstate how helpful your tutorials and tools! are. so many lightbulb moments in my attempt to get comfy with comfy. If you have a Patreon or similar, I'd be happy to support!
Great tutorial. If you're looking for ideas for future tutorials, where would we add LoRAs in the workflow? Can we add control net(s) and if so where in the stream. Keep up the great work.
The output of IPAdapter rendering is poor. Is there any suggestion you can give? I tried using LANCZOS but did not really help. Details are lust and bit blurred. A video about keep up the quality of rendered images would be very appreciated. I'm using 1.5 realistic vision because it delivers great results for me. Thanks for all your work. Its great.
Hey Mateo can you do a tutorial on swapping heads with IpAdapter, I have a clothes swapping workflow and I'm guessing you can modify i'm just not sure how it would be done?
Say you want to make a face model but for aliens instead of humans. In that case training would give far better results. Or say you want to describe a Picasso or a very specific art style.
thanks for all the work and knowledge shared here >> one question, where did u find this Image crop node which seems very useful avoiding to go back to an external app do to the same action >
Could you please clarify the controlNet setup? You say at 12.34 you say that you are: "loading a series of images that I resized, and then passed through a lineart preprocessor...". Could you please explain what those images are? Thanks :)
I took a video of myself saying "hello", I passed the first 32 frames of that video to a lineart preprocessor so to have the "skeleton" for the animation.
it already works with SSD-1B but it really burns the images. You have to set the CFG to like 2-2.5, but it's fast indeed. LCM is not supported yet (and don't think it will any time soon). On a side note they just released SDXL Face Plus!
Hi Matteo! As always, amazing video tutorial! In the last chapter (AnimateDiff) in the yellow-brown node called (load images upload) ,in the controlnet section, you have loaded 16 images? You don't show them in the video, what kind of images are they?
Thanks for making this tutorial, I have a question though, in my case, I am getting attn_mask (one more input) in the apply ipadaptor, any ideas, why so? Has something changed in the IPadaptar nodes and what it is expecting as input?
it is called FULL because it takes the whole 256 vector space to describe the face. As a comparison the plus models use 16 tokens. The result should be a very detailed face.
I'm having trouble when I use your canny workflow. It is telling me the size of tensor a must match the size of tensor b when it tries to apply the "KSampler" node. I've deleted and re-applied the "Apply IPAdapter" node, same issue persists. I used your other non-canny workflow, just with regular ip adapter and it works. It's something with the canny node that's messing it up and I don't know what it is.
I have an error, using the "from encoded". Error occurred when executing IPAdapterApplyEncoded: Error(s) in loading state_dict for Resampler: size mismatch for proj_in.weight: copying a param with shape torch.Size([768, 1280]) from checkpoint, the shape in current model is torch.Size([768, 1024]). Do you know what could it be? Thanks :). Amazing tool.
Any particular reason so insert the IPAdapter AFTER the AnimatedDiff Loader? Is the order even relevant? I'll try to test it the other way around as well, to see if 'adapting the style' before AnimDiff does it's magic makes any difference.
It shouldn't make a difference as everything happens in the KSampler anyway. Technically speaking IPAdapter patches the attention of the main unet so I'd put it at the end.
When i try loading the weighted workflow I'm getting this error: Error occurred when executing IPAdapterApplyEncoded: Error(s) in loading state_dict for Resampler: size mismatch for proj_in.weight: copying a param with shape torch.Size([768, 1280]) from checkpoint, the shape in current model is torch.Size([768, 1024]). File "C:\Users\Shadow\Desktop\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 152, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) File "C:\Users\Shadow\Desktop\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 82, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) File "C:\Users\Shadow\Desktop\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 75, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) File "C:\Users\Shadow\Desktop\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_IPAdapter_plus\IPAdapterPlus.py", line 338, in apply_ipadapter self.ipadapter = IPA( File "C:\Users\Shadow\Desktop\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_IPAdapter_plus\IPAdapterPlus.py", line 179, in __init__ self.image_proj_model.load_state_dict(ipadapter_model["image_proj"]) File "C:\Users\Shadow\Desktop\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\python_embeded\lib\site-packages\torch n\modules\module.py", line 2041, in load_state_dict raise RuntimeError('Error(s) in loading state_dict for {}: \t{}'.format(
thank you , second thing , both 'encode ipadapter iamge' and 'apply ipadapter' methods not working , should i use 512square also for the output? i use other resolution for output, if resolution not problem i should start find why it's pretend to not working at all, it loads and the out put is only likke t2i at the end, whole ipadapter pretend to not work with me , without any error, it just the out put exact same like t2i like there's no any ipadapter ..@@joeduffy52
the model is trained on A LOT of data so there's a good change it would work, at least for the standard stuff. If you need something very specific you'll have to train it
In general, watching similar YT tutorials by other people feels like mugging up each setting. Example: "Increase this, decrease that, keep this 0 or 1 and you'll do just fine." But that is not the case with you!
I like the way you naturally interpret how the proportions of different settings affects the image and know the best blend to mix them in to obtain the desired results. Not only do you understand the 'how', but you are also able to 'communicate' it to your audience. I can watch this video four times, and still learn something new from it.
Cool - given you have done this recently - how are you getting the "Apply IPAdapter" control? The one shown here that is.
I know how much goes into the IPAdapter and making these tutorials! Really appreciate all of your efforts, because of your work I’ve been able to easily inject my own art into workflows with beautiful results. 😊
agee! brilliant work
FANTASTIC video! Thank you, I am going to be using this technique quite a bit just to see what I can generate. Thank you again for all your hard work.
Thank you for all your work on this node!
For the Encode IPAdapter Image node, I've seen other ComfyUI nodes that have a variable number of inputs. I.e., when you use up image-slot-2, then image-slot-3 appears, then image-slot-4 appears, etc. This is a more flexible than having a fixed number of inputs for your node.
to do that you have to install an additional javascript in a location outside the extension directory which is an anti-pattern. When comfyui will support adding scripts from inside the extension I'll surely add the option
IPAdapter is one of the most useful things I've started using in A1111. My mind is always blown by how well it blends images together.
Just discovered your previous tutorial and saw you just released this video, amazing work! Thank you so much, this helps me a ton
Excellent breakdown of IPAdapter! 💎📺 Subscribed to your channel, much appreciated! 🙌🙏
Great job with the demo of time-stamp as well. Think we chatted on element as well about it.
fantastic. I dont know if some one more tell you before, but it look when talk about face, that you return a mirror pose of the original.
you are the best creator on the space. really!
training ipadapter for specific weird concepts will be amazing, waiting for it
It's so good that you made another video, the techniques are exactly what I need to learn and apply. I love your videos, thank you. I've had no issue following along but there is an issue I'm having with 'ip-adapter_sd15_light.bin' not showing up in the 'Load IP Adapter Model' node list. I have 6 out of 7 listed in the drop-down. It seems bizarre that only this one is not showing up. At first, I thought maybe I wouldn't use it and not worry but it will be interesting to use if I can identify the problem with my installation in the windows portable.
Update: renaming filetypes from .pth to .bin solved the issue.
Hi , nice to hear your voice again .. Always waiting for your tutorials, they are nicely explained and very informative .. 😊
This is the best video about IPAdapter I've ever seen. Thank you.
Really loved the vigorous experimentation - so many teaching moments. Thank you.
I really appreciate the effort you put into these videos! Topics flow together well, and a ton of good information is held within.
The IPAdapter for AnimateDiff is amazing! Thank you for the great suggestion!
Great content. Some tutorials focusing just on IPAdapter + Animatediff + control net would be amazing. You touched on it a little . The ability to use 1 ref image to train style on a video sequence is of great appeal. Cheers
amazing! can't overstate how helpful your tutorials and tools! are. so many lightbulb moments in my attempt to get comfy with comfy. If you have a Patreon or similar, I'd be happy to support!
that's amazing!
Thank you for your great tutorials once again!
thank you very much..
Oooh timestepping looks fun I learn something new everyday with AI. Thanks.
God Bless You Matteo. thanks for your efforts and pure contents.
Every minute in your videos.... PURE GOLD!! Many thanks and all my best wishes! ❤️🇲🇽❤️
you rule! thank you very much for all the effort !
Its very useful and inspirational, thank you very very much!
Thanks for doing these, excellent tutorials, well put together
Amazing stuff! Thank you for another guide.
Thanks a lot for this! I have a very simple question as to how to stop the Comfy UI server and restart it other than doing it manually
Amazing!
Please more of these tutorials! Grazie!
The best tool for SD!! Great work!!
Great tutorial. If you're looking for ideas for future tutorials, where would we add LoRAs in the workflow? Can we add control net(s) and if so where in the stream. Keep up the great work.
LoRAs and contronet are working very good with IPAdapter
Thank you very much! Very useful explanations!!!
The output of IPAdapter rendering is poor. Is there any suggestion you can give? I tried using LANCZOS but did not really help. Details are lust and bit blurred. A video about keep up the quality of rendered images would be very appreciated. I'm using 1.5 realistic vision because it delivers great results for me.
Thanks for all your work. Its great.
awesome 🙂
thank you for your amazing work on this!
Another great video, thank you very much
Great work, thank you so much!
❤❤ thank you for this amazing video
Nice work, Thanks.
thankyou, you rock!
Thank you very mutch
Hey Mateo can you do a tutorial on swapping heads with IpAdapter, I have a clothes swapping workflow and I'm guessing you can modify i'm just not sure how it would be done?
please add a link to the workflows in the description, it would make things much easier...
What would 'training the IP adapter' involve? Make it better at mapping images to tokens in more specific ways?
Say you want to make a face model but for aliens instead of humans. In that case training would give far better results. Or say you want to describe a Picasso or a very specific art style.
Hello, have you any idea why the node is red? It result loaded, no failed, but the node of IpAdapter plus is always red, even after a refresh...
thanks for all the work and knowledge shared here >> one question, where did u find this Image crop node which seems very useful avoiding to go back to an external app do to the same action >
since this came out is there an update to Appl,y IPAdapter from Encoded? I get an error on this node every time
Hey, great video! Any tips on how to increase speed of animate diff using a video and 1 controlnet input on a rtx4050 with 6gb of vram?
Could you please clarify the controlNet setup? You say at 12.34 you say that you are: "loading a series of images that I resized, and then passed through a lineart preprocessor...". Could you please explain what those images are? Thanks :)
I took a video of myself saying "hello", I passed the first 32 frames of that video to a lineart preprocessor so to have the "skeleton" for the animation.
I enjoyed the video. I wonder, what changes will happen if I raise the noise value?
Thanks for your work. It is a blast working with IPAdapter in ComfyUI! Are you planning a IPAdapter model for the SSD-1B model too?
it already works with SSD-1B but it really burns the images. You have to set the CFG to like 2-2.5, but it's fast indeed.
LCM is not supported yet (and don't think it will any time soon).
On a side note they just released SDXL Face Plus!
@@latentvision Oh didn't know. I will give it a try thanks.
Hi Matteo! As always, amazing video tutorial! In the last chapter (AnimateDiff) in the yellow-brown node called (load images upload) ,in the controlnet section, you have loaded 16 images? You don't show them in the video, what kind of images are they?
Thanks! it's line-art of a short video of me greeting at the camera!
Love IPadapter! Also does your extension works with IPadapter XL as well ?
just saw the rest of the video :)
Thanks for making this tutorial, I have a question though, in my case, I am getting attn_mask (one more input) in the apply ipadaptor, any ideas, why so? Has something changed in the IPadaptar nodes and what it is expecting as input?
this technology moves fast! have a look at my other videos! 😄there's one about attention masking
Thank you very much , I am looking at one by one. Thanks again for great videos @@latentvision
Little question, can you maybe say something about "full face" adapter?
it is called FULL because it takes the whole 256 vector space to describe the face. As a comparison the plus models use 16 tokens. The result should be a very detailed face.
I'm having trouble when I use your canny workflow. It is telling me the size of tensor a must match the size of tensor b when it tries to apply the "KSampler" node. I've deleted and re-applied the "Apply IPAdapter" node, same issue persists. I used your other non-canny workflow, just with regular ip adapter and it works. It's something with the canny node that's messing it up and I don't know what it is.
Hi, great video, Thanks!
What is the proper configuration to make it run with SD XL?
hi matteo, amazing work, thanks so much. is there any way to schedule the ip adapters for video? kinda like prompt scheduling?
well there's time stepping, I believe that's what you asking. It's a new feature that was present at the time of shooting this video but it is now.
Thanks a lot for your work ! I have only one available image slot on my Encode IPAdapter Image node, don't understand why
UP: after updating Comfy + extensions, it's all good !
I have an error, using the "from encoded". Error occurred when executing IPAdapterApplyEncoded: Error(s) in loading state_dict for Resampler:
size mismatch for proj_in.weight: copying a param with shape torch.Size([768, 1280]) from checkpoint, the shape in current model is torch.Size([768, 1024]).
Do you know what could it be? Thanks :). Amazing tool.
you are using the wrong image encoder
@@latentvision No, I realized, that I didnt activate (plus-true) in the encode. Funziona, grazie mille.
Extremely good Respect 👍
is there any demos about advance ipadapter features？
check my more recent tutorials!
Thank you!
Any particular reason so insert the IPAdapter AFTER the AnimatedDiff Loader? Is the order even relevant? I'll try to test it the other way around as well, to see if 'adapting the style' before AnimDiff does it's magic makes any difference.
It shouldn't make a difference as everything happens in the KSampler anyway. Technically speaking IPAdapter patches the attention of the main unet so I'd put it at the end.
I love you
damn... offer me dinner first at least
When i try loading the weighted workflow I'm getting this error:
Error occurred when executing IPAdapterApplyEncoded:
Error(s) in loading state_dict for Resampler:
size mismatch for proj_in.weight: copying a param with shape torch.Size([768, 1280]) from checkpoint, the shape in current model is torch.Size([768, 1024]).
File "C:\Users\Shadow\Desktop\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 152, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "C:\Users\Shadow\Desktop\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 82, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "C:\Users\Shadow\Desktop\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 75, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "C:\Users\Shadow\Desktop\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_IPAdapter_plus\IPAdapterPlus.py", line 338, in apply_ipadapter
self.ipadapter = IPA(
File "C:\Users\Shadow\Desktop\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_IPAdapter_plus\IPAdapterPlus.py", line 179, in __init__
self.image_proj_model.load_state_dict(ipadapter_model["image_proj"])
File "C:\Users\Shadow\Desktop\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\python_embeded\lib\site-packages\torch
n\modules\module.py", line 2041, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:
\t{}'.format(
Hi good tuto but waht about ratio ? it's only square image or can we create portrait or lansdcsape ratio ?
you can do any size you want. only the reference image is cropped to a square. but your generated image can have any aspect ratio
I'd love to meet you, I'm also italian
you probably wouldn't like me 😄
Where is the workflow of used in videos
what vga do you use?
4090 but the generation is almost always sped up
@@latentvision thank you!!!!!!
sorry, where can i find 'batch images' node ? from which node packs i can install it , i don't have this node , how to get it ?
Should be at Add Node>image>Batch Images
If you don't find it, you maybe need to update Comfy. It's part of the main build not a Custom Node AFAIK.
thank you , second thing , both 'encode ipadapter iamge' and 'apply ipadapter' methods not working , should i use 512square also for the output? i use other resolution for output, if resolution not problem i should start find why it's pretend to not working at all, it loads and the out put is only likke t2i at the end, whole ipadapter pretend to not work with me , without any error, it just the out put exact same like t2i like there's no any ipadapter ..@@joeduffy52
i am using the CPU , no GPU here , just testing and practicing , it just worked 'encode ipadapter iamge' after a restart os ! then stopped again !
What if anyone would want to make something other than faces? Say cars, interior designs, parfume bottles, buildings or something like that?!
the model is trained on A LOT of data so there's a good change it would work, at least for the standard stuff. If you need something very specific you'll have to train it
Good work !
I didn't find the encode IPAdapter Image, is it obsolete?
Where I can get that IPAdapter_image_encoder_sd15?
I am looking that for ages anyone please help