Attention Masking with IPAdapter and ComfyUI
ฝัง
- เผยแพร่เมื่อ 22 ก.ค. 2024
- Exciting new feature for the IPAdapter extesion: it's now possible to mask part of the composition to affect only a certain area... And you can use multiple masks for a perfect result. This and much more directly from the developer of the ComfyUI Extension.
You can find a few workflows in the examples directory of the ComfyUI_IPAdapter_Plus extension: github.com/cubiq/ComfyUI_IPAd...
More complex workflows (just download the json format):
Masked conditioning: pastebin.com/DjYg6jMC
Complex Masked conditioning: pastebin.com/PsD13EVK
RGB Masking: pastebin.com/jBB9cAn2
00:00 Intro
00:21 Weight types
02:51 Attention Masking
05:17 Multiple Masks
08:57 Mask Conditioning
10:48 Conclusions
🎵 Background Music
-- "Part A" by Alexander Nakarada (www.serpentsoundstudios.com)
Licensed under Creative Commons BY Attribution 4.0 License
-- "White Gold" by Karl Casey @ White Bat Audio (whitebataudio.com/)
-- "CyberPunk City" by Peritune (peritune.com/)
I want to thank everybody for taking the time to write compliments and words of encouragement. I really do appreciate all of them and they honestly motivate me in releasing more videos. Let me know if there are topics you'd like to see covered and thank you all again!
These are fantastic videos. I am wondering if there is a way to use IPadatper to only transfer color pallet or other facets of an input image (but not the whole). I think for SD 1.5 there is a way to take only the color pallet with the t2iadapters but not with SDXL - maybe you know of a way?
I don't think there's a color controlnet yet for SDXL but you can use img2img with a certain grade of success. Set the colors how you want them laid out with a paint software then use that reference image as base for your img2img with a very high denoise. That should kinda work.@@EH21UTB
👋
@LouisGedo Thank you so much for all you efford.Actually I am trying to make a flow in order to let a model wear some specific cloth provided as a picture. I tried lots of times, but the clothe weared by the model generated by the flow always different with the one I provided. I really want to understand how to do it . can you please explain about this .
I’ve finally gotten into IP Adapter. Combining it with depth and pose control nets and ReActor face swap yields incredible results. For instance, I took photos of my parents and photos of vintage Christmas cards and produced a nice vintage style Christmas card that looks like them as a gift.
Could you please share your workflow? Must appreciated
Bro I need to talk with u can u share ur insta id
You're such a great example of an open source dev. Always listening to feedback and working with the community.
Overnight Latent Vision becomes top SD instructor
IPA Plus has been the most game changing thing I've found yet. It's capabilities are insane. Can't ask for much more.
You’re now my #1 TH-camr for ComfyUI . Your channel deserves so much more subs - And it will!!!
This is just mind blowing at this point. I'm really really glad I came across you (and IPadapter) in my AI learning path!
This... IS... CRAZY !!!
Thank you so much for sharing your fantastic work !
I'm using IP+ andd I'm impressed by the results, tanks to you !
You are a legend, loving the IpAdapter and your vids are wonderful, simple in style yet so informative ...Love em..You are appreciated..
The merge of styles is insane!
You, kind Sir, are a treasure. I absolutely love IPadapter. I am very very thankful.
Brilliant - thank you for these and for sharing the workflows
Just started going through these playlist; This is truly a gem. Thanks much for sharing.
If you get a chance, do list the machine specs you use for controlNet + animateDiff +IPAdapter? It does take a long time even on a 16GB GPU. TIA
I just tested it and it's amazing! The results are impossible to obtain with a normal prompt, this bring so many possibilities. Thanks a lot for your videos!
A wizard in action! :) So this is how you are supposed to work with masks! I had to pause this video several times, so much information in such a short time, well done, very useful!
There's not that many videos yet on your channel, but every video is a diamond. One of the most interesting and valuable content I've met on youtube so far. Thank you VERY much for what you are doing.
Fantastic! A truly superb tool and fantastic explanation. Thank you!
Matteo il MAESTRO!!! Many thanks for the great latent wisdom you give us!! ❤️🇲🇽❤️
I am grateful...
that's beautiful, that's exciting.
exquisite, incredible work!
you are very honored. Thank you very much for everything, a great mind.
you have no idea how many people you're helping, making our projects more fun, easier and better!
Amazing job as usual, what a great new feature update!
This is pure (AI) magic and will help soooooo much in composition! Amazing results in your demonstration, and your tutorial is spot on, very clear to understand. Thanks so much for your hard work!
Wonderful content and wonderful tech, thanks for your hard work!
This is awesome Bravo. I tested the channel penalty this morning, it improved the result as it handles well weights more than one
Wow! You have unleashed one of the best things for Stable Diffusion since ControlNets! Thanks for developing these great tools and showing us how to use them!
Fantastic! I will have to watch this several times!
Jesus christ, ive been in love with IPAdapter since its release and this just completely unlocks all the possibilities. This is an insane value tech, thanks a lot for the contribution.
Amazing! Keep up the incredible work.
Holy! That was so impressive. Love your videos man! So much to learn and see it in action.
just to say thank you for you work and dedication !
I want thank you for your great work, not only this video, but also other videos and the comfyui_adpater_plus, which help me learn great knowledge about how to control the AI model to draw images.
thanks! And have fun with AI! 😄
This is just absolutely amazing!
omg. amazing stuff... unbeatable feature.. you are brilliant
Thank you for your work with IPAdapter. I've only been using it ever since I found your videos on it. It's so great, being able to have 1 image loras without the need to download others, and can just very quickly create them. The one Issue I've been having is it's been transferring the hair to every character design (I'm using a Harley Quinn reference I made for the adapter image and it's copying the pigtail hairstyle and face on pretty much every character lol)
Great job as always!
Fantastic work, thank you so much!
Your work is unbelievable. This is tremendous and I just can't believe we get to watch you develop in real time these tools that will revolutionize AI image generation.
That is pretty cool, thanks for the update and the tutorial
Amazing Work as always!
This is amazing. I cannot express my feeling after seeing your video. This blows my mind. ❤😊
Amazing work as always!
bravo , matteo! 10/10 questa roba! 🔥
Thank you for the work. Amazing tool
This tech is unreal! Your channel deserves to blow up. I'm loving your tutorial format.
Completely blow me away! Thanks so much for creating the Adapter and these tutorials! ✌
Superb , Awsome work
I agree with others. Mind blowing feature. Thank you! 🤯👌🤘
Truly amazing. Many thanks
Excellent work!
Mateo, this blows my mind, I'm so inspired by your great work, I can't express my gratitude enough. Your guidance hit the mark in a simple and easy to understand way which was definitely what I needed. Thanks again for the great work you do.
Hey there! I rarely comment on videos, but I feel compelled to express my appreciation for your unique approach. Unlike many creators who merely guide us to mimic their actions, you delve deeper, offering insights into the 'why' and 'how' behind each step. This approach not only makes your content easy to follow but also empowers us to adapt and understand the process, fostering genuine learning.
I've eagerly watched all your videos, and the knowledge I've gained is invaluable. Your response to a comment in another video about the tedium of covering basics resonated with me. It's true that many overlook the importance of fundamentals. Your explanations are a rare gem in a sea of superficial tutorials; they enable us to think, not just copy.
To alleviate the repetition for you, perhaps consider creating a series of detailed videos on these basic concepts. This could serve as a reference point for your viewers, and it's also a strategy that tends to be favored by the TH-cam algorithm.
What captivates me the most, however, is watching your workflow, especially in segments like the anime and pixel art transformations. Your real-time commentary adds immense value. Regardless of how many times a topic has been covered by others, your unique perspective and method of presentation make your videos a must-watch, even on well-trodden subjects."
A comment made by AI for a video about AI😅We live in weird times! I really enjoyed it thanks for the kind words
I used A.I. to touch it up but its really how i felt
This is awesome - thank you 😎
WOW ! you are the BEST ! It's just so hard just by looking at video but might be able to replicate by doing. Thanks for that man !
Really awesome stuff! Brilliant!
I used to love ipadapter now I adore it 😍😍😍
Always a treat when Mateo puts out a new video 🙌
Thanks a lot! it was very very useful!
Great Work again 💪
Wow! This is a great update!
Mateo, you are a GOD! Thank you so much!
Beautiful, simple, explosively relevant. You're inspiring.
thank you, this is amazing!
Thanks a lot for your work on this most excellent tool. This has given me so much possibilities regarding composition, and easily as well ! Less hassle and much better control/results.
Impressive stuff.
Outstanding
Gamechanger! There are many, many methods to achieve these end results; however, as we all know, they're tedious and extremely mind-numbingly time-consuming. Matteo's IPAdapter with noise method works the best. Work smarter, not harder!
amazing~!! Thank you very much !!
You blew me away with compositing using multiple masks. That is so freaking cool!
This is amazing! Thanks
Thank you so much for your insight :)
wow, great job! Thank you!
amazing tools and workflow! Insanely impressive!
Wow, just wow. So detailed explanation and so easy to follow one of the best channels on Comfyui. I am always looking forward to your videos.
I've only recently discovered your content, but I just had to tell you how blown away I am at your knowledge and skill in this field. It's just breathtaking stuff you're getting up to. Kudos to you, and all the brains that make this technology possible.
Could you imagine travelling back in time even just 20 years and showing this to people? It'd be science fiction.
Incredible, thanks a lot
this is amazing!
Sei un grande!
Beautiful, simple, explosively relevant. You're inspiring. Love you so much.
really incredible -- great demo, I can't wait to use this. I've been experimenting with some of your other tutorials to make animatediff clips. Excited to see the control I might be able to get from masking areas and being able to prompt each one.
amazing! Thank you!
good job🥰
I love this! :)
Fantastic!
.
wow, that is great🥰
Very valid concentrated information... thx a lot! - I wished all these special infos would be provided as "note-nodes" explaining things right in the json-example. It is obvious to me that your knowledge about all these components and how they work together is superior to some other "random-masters" on YT. It is difficult though... to remember all the little facts, like which model needs what other model and so on...
I hear you, the problem with comfy (and all generative ai) is that things move so fast that any documentation is outdated in 3 months (sometimes less). That would mean reviewing all the workflows every now and then and it's a huge amount of work. I'm thinking of writing some articles especially for more complex topics... but do people still read? Everything seems to be video or burst
@@latentvision You are right Matteo, thank you for your answer. I also know that my "wish" included more work for you... I didnt mean it in a demanding way, I am thankful for any piece of information on a journey into comfy UI. The problem is, that I like to understand things in depth... while it seems to me, that some are ok if if only some nice picture result in the end... and in the beginning of research it is still difficult to seperate the good from useless sources of information. Not many developpers are as good in teaching as you are, or they do not teach at all. I think, much of the information, like - basic relations - will not be outdated so quickly... once the main principles are understood... understanding coming updates can also be challenged :-)
che lavoro della madonna però, grandissimo
Very cool!
Where do you come from you hero!! Amazing explanation once again! Thank you.
thanks king
super cool!
superb
Bella scoperta, un italiano! che bello. Iscritto subito
Bro is fuggin Yoda of ComfyUI.
thanks for the lol 😆
Well i struggeled with one part.. i cant get the "Prepare image for Clipvision" Node.. What do i get Wrong?
damn I had no reason to use comfyUI until NOW, this update is crazy
🔥
How can you do style transfer in comfyUI ?
If you've ever seen a child watching a juggler throw balls, you can imagine my expression.
as long as it's not a clown!
It's more like a magician. I look forward to watching your new videos, but there are so many questions. For example, if you are almost satisfied with the composition, but would like to remove something and move something to another place. It's still a big mystery to me.
wow. just that... wow.
Amazing. Is this workflow from video available somewhere?
My Mask Conditioning was not really working. Would like to see more on that as I did not get anything similar to your results. But the rest was very useful!
you probably need to lower the IPA weight a little to get mask conditioning to work