I am so thankful for your channel. I have watched countless yourtube video about comfy ui, but yours are definitely one of the clearest with deep understanding of the subject. I hardly leave comment EVER, but i felt the need to write this one. I was watching and rewatching your video and follow along. Its so much fun. Thank you so much!
Please continue doing more informative videos like you always do, everyone else just shows prepared workflows but you actually show how to build them. Can't thank you enough.
The result looks pretty amazing. Could you maybe do one tutorial where there is combination with control net (not sure if that possible) just so we can also control the position of the characters.
Great stuff, as always! One thing: the two girls were supposed to be "shopping" and the cat/tiger were supposed to be "playing". The subjects transferred properly (clean separation) but there's no trace of either "shopping" or "playing" in the result.
Grande matt3o, i tuoi nodi sono uno spettacolo! 😁👍 Questo workflow soprattutto sembra assolutamente interessante e malleabile alle problematiche delle varie generazioni
Thanks again for great effort and explanation Matteo. You are amazing! Quick question: Is it possible to use controlnets with IPAdapter Regional Conditioning?
Once again, it's simply wonderful! During a few tests, I noticed that the: "RGB mask from node", needs very bright colors to work. A slightly darker green and it no longer has any effect. Everything else produced cool results on the first try. Thanks for all the work! And I'm just about to follow your ComfyUI app tutorial video to make one myself.
@@Showdonttell-hq1dk using black or white and the threshold you can technically get any color. But you can probably better use the node Mask From Segmentation
u should be proud of your work. thanks for all you do. Was working on my video workflow with masking ipadaptors for multiple people... this will SOOOOOO make things easier.
What a cool update! This looks useful for controlling character clothing, hairstyle and that kind of thing, using reference images. Also, if you compose a 3D scene in Unreal Engine, it can output a segmented object map as colors, which could make this very powerful. You could link prompts and reference images to objects in the scene and then diffuse multiple camera angles from your scene, without any further setup.
Wonderful. May I ask how I can insert FaceID into this workflow? Right now I get this error: Error occurred when executing IPAdapterFromParams: InsightFace: No face detected.
Great workflow. Can you add edge control and depth to the process? I tried it but failed. Can you help me? I'm not sure how the nodes are supposed to be connected, it doesn't seem to work.
This is so helpful! I’m using closeup selfies of three people to create composite band photos for promotion, and this simplifies the workflow immensely. Question: Do you have any tips to go from three headshots to a composite image which shows three people full length, head to toe? Adding that to the prompts hasn’t worked very well so far, and I’m not sure if adding separate openpose figures for each person would be the way to go? Any advice would be most appreciated!
that has to be done in multiple passes. there are many ways you can approach that... it's hard to give you advice on such complex matter on a YT comment
@@latentvisionI understand. But “multiple passes” gives me an idea anyway. So probably I should generate bodies for each person first, and only then combine the three.
Hi Matteo, thanks for your contributions to the community. I am trying to use Attention Masking in the process of compositing product photos. The idea is to be able to define in which zone of the image each element is located. For example, in a photo of a wine, define the location of the bottle and the location of the props, such as a wine glass, a bunch of grapes, a corkscrew, etc. But I tried the Attention Masking technique and it is not giving me good results in SDXL. Is it only for Sd1.5? Do you think it is a good technique for this kind of compositions for product photography or do you think there is another better technique? Thanks in advance for your help! 😃😃😃
this is complex to answer in a YT comment. depends on the size of the props. You probably need to upscale the image and work with either inpainting or regional prompting. Try to ask on my discord server
This workflow is amazing, thank you for sharing! I have been trying to get it to work with two characters generated from two LORAs. The LORAs have been trained on XL so they are expecting to make 1024x1024 images. I have made my whole image larger so that the mask areas are 1024x1024, but still everything is coming out kind of wonky. Have any of you explored a solution for generating two characters from separate LORAs in a single image?
@@latentvision Thanks so much for reply!! I know I can replace the IPadapter Unified loader with FaceID unified loader in this workflow, but I don't know how to receive images and adjust the v2 weight or choose a weight type while using regional conditioning for FaceID, in other word, I don't know how to create an equivalent "IPadapter FaceID Regional Conditioning" node with existing nodes.
Hello Mateo! Excellent workflow :) Consultation: To solve the proportion issues, that the cat is the size of a cat and that the tiger is the size of a tiger, the best solution would be to edit the size of the masks? Thanks
no, if you need precise sizing you need a controlnet probably. To install the essentials use the Manager or download the zip and unzip it into the custom_nodes directory
how would you go about adding some overlap like the girl and dragon example from the beginning of the video where they are touching? Or does this process have enough leeway to let them interact?
The masks can overlap, if the description is good enough the characters can interact. SD is not very good at "interactions" but standard stuff works (hugging, boxing, cheek-to-cheek, etc...). On top you can use controlnets
Hi. Thanks for your work. I was wondering. Is there any IPAdapter node that will be linked to AnimateDiff? And this node will work only in a certain frame.That is, if I connect 2 input images, from 0 to 100 frame one image affects the generation, and from 101 frame the second input image affects the generation. But it would be quite nice if from frame 90 to 110 these images are blended.
Very good stuff but hard to use. Thank you for this tutorial. I hope SD3 will better understand prompts and IPAdapter will be supported on SD3 as well. ... But SD3 is now a paid/API , so sad for "free opensource"
Hi great stuff, is there anyway to do this kind of attention masking with loras, so I can apply separate loras to separate masks? There's a few things kicking around but nothing seems to work all that well.
@@latentvision hmm this is why I have been struggling there are some applications nodes for it but from the stuff I've found haven't had much luck yet, might you be able to help me out or do a lil digging maybe you can pull of some more magic! 😄
@@thomasmiller7678 But can't you just use the concerned LoRAs in a separate workflow to generate the images you like, then bring them here, apply conditioning and combine?
@@latentvision sorry, i meant animation with ip-adapter, there are many workflows out there most famous animatediff and ip-adapter i just though yours is defiantly going to be better
If i have a strong color distribution in my reference style image the result seems to put the colors in the same areas as a resulting image. Is there a way around this? (ipadapterPlus with strong setting and style transfer)
@@latentvision I was talking about certain parts at the macro color placement of the style reference image. If for example the upper part of the reference image has an orange leather texture then in the resulting image it is also more likely to have an orange background or orange "parts" in this upper area of the image.
@@latentvision Thank you for the reply. I do not know if it is possible but perhaps you could code a weight_type option for the style transfer as in the ipAdapter Advanced note has a weight_type option. You could select style transfer as the first weight_type and then there is a subcategory (weight_type2) how this style transfer is applied: linear, ease in, ease out... But this is just a rough guess.
@@latentvision maybe my weights/start/end are not right, I always have trouble make a known character interact with another known character or a known item. "Known" in this case means using IPAdapter. Most of the example I saw is 2 characters/subjects standing beside each other, not interacting, so I wonder.
@@latentvision I tested it in many cases, the interaction works pretty well; a girl holding an apple, a girl holding a teddy bear... all works well. With 2 girls holding hands, bleeding happens time to time, negative prompts are not always applicable; can the regional conditioning accept a negative image?
thanks, for style someone , is there benefit to use ipadapter v2 combine with instantId or just ipadapter v2 face id is enough ? if padapter v2 combine with instantId get more better result , any tutorial for that ? another is does a casual&normal camera taken photo of a person can get a fantasy result use above method ?
@@latentvision thanks, for point2 , can normal human photo from phone camera can be transfer into a style masterpiece using comfyui ? i don't find video in youtube talk about that
@@latentvision below are my case : i want to take my child born photo into a t shirt. but this photo is taken from very long time ago , and the quality is bad , especially the face got a bit vague , anyway its my memory. can i using comfy ui transfer this vague photo into a picture that reserve the pose and face of my child , improved the quality and with T-Shirt art style which suitable for print to t shirt，and it should reserve my child's face and body pose that i can regonize. how can i do with that using comfyui ?
@@user-rx8wb4dn9c it is possible using a combination of techniques but it's impossible to give you a walk-through in an youtube comment... it highly depends on the conditions of the original picture
The wizard says this isn't magic but creates pure magic anyway.
Any sufficiently advanced technology is indistinguishable from magic
6:05 "oh shit, this is so cool!" - nothing to add.
I am so thankful for your channel. I have watched countless yourtube video about comfy ui, but yours are definitely one of the clearest with deep understanding of the subject. I hardly leave comment EVER, but i felt the need to write this one. I was watching and rewatching your video and follow along. Its so much fun. Thank you so much!
I think it was a blessing that I found your channel. These workflows spark my creativity so much.
Please continue doing more informative videos like you always do, everyone else just shows prepared workflows but you actually show how to build them. Can't thank you enough.
Awesome as always, Thank you for sharing those amazing nodes with us.
Would love a video of you going into depth on the development and programming side of things!
maaaaybe 😄
The result looks pretty amazing. Could you maybe do one tutorial where there is combination with control net (not sure if that possible) just so we can also control the position of the characters.
Mr. Spinelli always delivering magic!! Thanks and keep the superb work 👏👏🙌🙌
Amazing stuff as always Matt3o! Can't wait for the next IPAv2 stuff you got going on!
Grazie Matteo stai facendo un buon lavoro
Great stuff, as always! One thing: the two girls were supposed to be "shopping" and the cat/tiger were supposed to be "playing". The subjects transferred properly (clean separation) but there's no trace of either "shopping" or "playing" in the result.
the first word in all prompts is "closeup" that basically overcomes anything else in the prompt
Grande matt3o, i tuoi nodi sono uno spettacolo! 😁👍
Questo workflow soprattutto sembra assolutamente interessante e malleabile alle problematiche delle varie generazioni
Great video as always. A large number of interesting things. Always thank you for your time in the community.
Thanks again for the great work.
This is going to be powerful. Good work Mat3o!
Thanks again for great effort and explanation Matteo. You are amazing!
Quick question: Is it possible to use controlnets with IPAdapter Regional Conditioning?
yes! absolutely!
Wow, this is so insanely cool. I can’t wait to play with it, Matteo.
the logic you've used in making these nodes makes it so much easier! thank you!
Thanks for all your hard work and genius!
just doing my part
An instamazing day when Maestro Latente spills his magical brilliance!!
most usefull videos about comfyui, thank you SO MUCH, im excited af about how cool it is
This is an amazing workflow! I wish we could animate it.
Angel Mateo came down from latent space again to teach the world his magic
it's gonna help million times on my personal project!! thank you
Amazing work! improved the usability of the IPadapter!
Brilliant stuff, thanks again, Matteo. Can't wait for the FaceID nodes to be brought to this workflow.
Thank you, thank you, thank you.
Great work, my friend.
Another master piece of art.
Really apreciated.
glad to help
Once again, it's simply wonderful! During a few tests, I noticed that the: "RGB mask from node", needs very bright colors to work. A slightly darker green and it no longer has any effect. Everything else produced cool results on the first try. Thanks for all the work! And I'm just about to follow your ComfyUI app tutorial video to make one myself.
you can set thresholds for each color, you can technically grab any shade
@@latentvision Of course I tried that. But it worked wonderfully with bright colors. It's no big deal. As I said, thanks for the great work! :)
@@Showdonttell-hq1dk using black or white and the threshold you can technically get any color. But you can probably better use the node Mask From Segmentation
u should be proud of your work. thanks for all you do. Was working on my video workflow with masking ipadaptors for multiple people... this will SOOOOOO make things easier.
"oh shit, this is so cool"
Amazing, thanks again for all your hard work!
Nice! More nodes to play with!
Great video! Thank you for all your hard work!
Genius. Fact. Again.
Thank you for everything!
8:01 "This is not magic," says the wizard
I swear it is not :P
I learned something new, happy, I want to ask when can we do a tutorial on replacing furniture indoors or something like that?
yeah that would be very interesting... I'll think about it
You rock!!!!! Thanks for the amazing job!!!!
What a cool update! This looks useful for controlling character clothing, hairstyle and that kind of thing, using reference images. Also, if you compose a 3D scene in Unreal Engine, it can output a segmented object map as colors, which could make this very powerful. You could link prompts and reference images to objects in the scene and then diffuse multiple camera angles from your scene, without any further setup.
what a legend
really cool
I really love your work
Wonderful. May I ask how I can insert FaceID into this workflow? Right now I get this error: Error occurred when executing IPAdapterFromParams: InsightFace: No face detected.
"the cat got tigerized"
Excellent! Thank you for your work and for sharing :)
Thank you very much man!
ahh nice tutorial, I like it very thanks
Great workflow. Can you add edge control and depth to the process? I tried it but failed. Can you help me? I'm not sure how the nodes are supposed to be connected, it doesn't seem to work.
Sei un grande!
really really cool addition. thanks
Thank you!
Great work as always🎉
That was awesome
Continue creating magic senpai!! ❤️
Incredible
Awesome stuff, as always
Just wow! Thanks a lot again!
This is so helpful! I’m using closeup selfies of three people to create composite band photos for promotion, and this simplifies the workflow immensely. Question: Do you have any tips to go from three headshots to a composite image which shows three people full length, head to toe? Adding that to the prompts hasn’t worked very well so far, and I’m not sure if adding separate openpose figures for each person would be the way to go? Any advice would be most appreciated!
that has to be done in multiple passes. there are many ways you can approach that... it's hard to give you advice on such complex matter on a YT comment
@@latentvisionI understand. But “multiple passes” gives me an idea anyway. So probably I should generate bodies for each person first, and only then combine the three.
Hi Matteo, thanks for your contributions to the community.
I am trying to use Attention Masking in the process of compositing product photos. The idea is to be able to define in which zone of the image each element is located. For example, in a photo of a wine, define the location of the bottle and the location of the props, such as a wine glass, a bunch of grapes, a corkscrew, etc. But I tried the Attention Masking technique and it is not giving me good results in SDXL. Is it only for Sd1.5? Do you think it is a good technique for this kind of compositions for product photography or do you think there is another better technique? Thanks in advance for your help! 😃😃😃
this is complex to answer in a YT comment. depends on the size of the props. You probably need to upscale the image and work with either inpainting or regional prompting. Try to ask on my discord server
thanks,and i think you should put the "pill" word in the title :)
6:04 Yes, yes it is.
:)
This workflow is amazing, thank you for sharing! I have been trying to get it to work with two characters generated from two LORAs. The LORAs have been trained on XL so they are expecting to make 1024x1024 images. I have made my whole image larger so that the mask areas are 1024x1024, but still everything is coming out kind of wonky. Have any of you explored a solution for generating two characters from separate LORAs in a single image?
love.... thank you !!!
❤❤❤ as always best tutorial
감사합니다.
Molto bene... molto bene )
Great job done! I'm wondering if theres any workflow using faceid series IPadapter with regional prompting...
it totally works there's nothing special to do just use the FaceID models
@@latentvision Thanks so much for reply!! I know I can replace the IPadapter Unified loader with FaceID unified loader in this workflow, but I don't know how to receive images and adjust the v2 weight or choose a weight type while using regional conditioning for FaceID, in other word, I don't know how to create an equivalent "IPadapter FaceID Regional Conditioning" node with existing nodes.
amazing
Hey matt3o, great stuff! I'm trying to replicate this with SDXL models but getting a blank output. Any ideas?
This video is great, but I follow the video, why does the portrait not look like the original picture
Would this work with Animatediff? Perhaps for segmenting different clothing styles in a fashion film.
attention masking absolutely works with animatediff
Why does it show ClipVision model not found when I use it?
awesome
👏👏👏
oh shit, this is so cool.... :)
Absolutely incredible. Like always. BUT: Cat and tiger are not "playing". Please fix this as soon as possible.
(just kidding, of course.) 😅
Awesome sir :)
Thanks
What file is the ipadpt in the embedded group read by the author and how to edit it
Why doesn’t it work so well with the SDXL model? In my case, the result is one girl with different styles on two sides of the head.
try to use bigger masks, try different checkpoints, use controlnets
Hello Mateo! Excellent workflow :)
Consultation: To solve the proportion issues, that the cat is the size of a cat and that the tiger is the size of a tiger, the best solution would be to edit the size of the masks?
Thanks
no, if you need precise sizing you need a controlnet probably. To install the essentials use the Manager or download the zip and unzip it into the custom_nodes directory
how would you go about adding some overlap like the girl and dragon example from the beginning of the video where they are touching? Or does this process have enough leeway to let them interact?
The masks can overlap, if the description is good enough the characters can interact. SD is not very good at "interactions" but standard stuff works (hugging, boxing, cheek-to-cheek, etc...). On top you can use controlnets
Thanks a lot! Looking forward to more content!@@latentvision
Hi. Thanks for your work. I was wondering. Is there any IPAdapter node that will be linked to AnimateDiff? And this node will work only in a certain frame.That is, if I connect 2 input images, from 0 to 100 frame one image affects the generation, and from 101 frame the second input image affects the generation. But it would be quite nice if from frame 90 to 110 these images are blended.
yes I'm working on that
@@latentvision Thank you. I've added AnimateDiff and 2CN to your workflow. And it's working well.
Very good stuff but hard to use. Thank you for this tutorial. I hope SD3 will better understand prompts and IPAdapter will be supported on SD3 as well. ... But SD3 is now a paid/API , so sad for "free opensource"
SD3 should be really easy to guide with images... let's see when they release the weights.
Woah
My comfyui don't have 🔧 Mask From RGB/CMY/BW, only have Mask from color. Where do i find it?
you just need to upgrade the extension
Is there a way to use controlnets in each region, I was having issues with that?
yes of course! just try it
Hi great stuff, is there anyway to do this kind of attention masking with loras, so I can apply separate loras to separate masks? There's a few things kicking around but nothing seems to work all that well.
not really (it would be technically feasible probably but not easy)
@@latentvision hmm this is why I have been struggling there are some applications nodes for it but from the stuff I've found haven't had much luck yet, might you be able to help me out or do a lil digging maybe you can pull of some more magic! 😄
@@thomasmiller7678 But can't you just use the concerned LoRAs in a separate workflow to generate the images you like, then bring them here, apply conditioning and combine?
Yes that is possible but it's still not a true influence like the Lora would be if it could be implemented
niubi!!
when are going to do videos with ip-adapter workflow?
not sure I understand
@@latentvision sorry, i meant animation with ip-adapter, there are many workflows out there most famous animatediff and ip-adapter i just though yours is defiantly going to be better
@@walidflux I'll try to do more animatediff tutorials, but I need to add a new node that will help with that
is there a way to use faceidv2 with this?
I will add the faceid nodes next
@@latentvision ♥
Hi, Which one is better IP adapter FaceID vs InstantID ?
the are different 😄depends on the application
If i have a strong color distribution in my reference style image the result seems to put the colors in the same areas as a resulting image. Is there a way around this? (ipadapterPlus with strong setting and style transfer)
I'd need to see. what you mean by strong color distribution?
@@latentvision I was talking about certain parts at the macro color placement of the style reference image. If for example the upper part of the reference image has an orange leather texture then in the resulting image it is also more likely to have an orange background or orange "parts" in this upper area of the image.
@@Vincent-ce7bp in that case probably your best bet is to play with "start_at" (like 0.2) and weight.
@@latentvision Thank you for the reply. I do not know if it is possible but perhaps you could code a weight_type option for the style transfer as in the ipAdapter Advanced note has a weight_type option. You could select style transfer as the first weight_type and then there is a subcategory (weight_type2) how this style transfer is applied: linear, ease in, ease out... But this is just a rough guess.
is this possible to use ipadapter and attention masks for character and items interaction? like a man handing over an apple or carrying a bag
yes of course! why not?!
@@latentvision maybe my weights/start/end are not right, I always have trouble make a known character interact with another known character or a known item. "Known" in this case means using IPAdapter. Most of the example I saw is 2 characters/subjects standing beside each other, not interacting, so I wonder.
@@latentvision I tested it in many cases, the interaction works pretty well; a girl holding an apple, a girl holding a teddy bear... all works well. With 2 girls holding hands, bleeding happens time to time, negative prompts are not always applicable; can the regional conditioning accept a negative image?
thanks, for style someone , is there benefit to use ipadapter v2 combine with instantId or just ipadapter v2 face id is enough ?
if padapter v2 combine with instantId get more better result , any tutorial for that ?
another is does a casual&normal camera taken photo of a person can get a fantasy result use above method ?
yes you can combine them to get better results, but don't expect huge improvements, just a tiny bit better :)
@@latentvision thanks, for point2 , can normal human photo from phone camera can be transfer into a style masterpiece using comfyui ? i don't find video in youtube talk about that
@@user-rx8wb4dn9c depends what you are trying to do. too vague as a question, sorry
@@latentvision below are my case :
i want to take my child born photo into a t shirt.
but this photo is taken from very long time ago , and the quality is bad , especially the face got a bit vague , anyway its my memory.
can i using comfy ui transfer this vague photo into a picture that reserve the pose and face of my child , improved the quality and with T-Shirt art style which suitable for print to t shirt，and it should reserve my child's face and body pose that i can regonize.
how can i do with that using comfyui ?
@@user-rx8wb4dn9c it is possible using a combination of techniques but it's impossible to give you a walk-through in an youtube comment... it highly depends on the conditions of the original picture
Matteo, FlashFace got released, will you bring it too?
I had a look at it, it's weird cookie. 10GB model that only works with SD1.5... I don't know...
@@latentvision 🤣🤣🤣🤣 Weird cookie it was really fun! Thank you so much for answering!
Mateo, I updated all and nodes still red, am I the only one´?😭
could you combine this with Lora?
one lora per mask? no, you can't the model pipeline is only one
прекрасно но это только портретные изображения в близи, видно что на примере кота и тигра уже не знает как изобразить что они играют