Multi-ControlNet tutorial.
ฝัง
- เผยแพร่เมื่อ 9 ก.ค. 2024
- With the new update of ControlNet in Stable diffusion, Multi-ControlNet has been added and the possibilities are now endless. In this Stable diffusion tutorial we'll go through the new Multi-ControlNet feature.
Support me on Patreon to get access to unique perks! / sebastiankamph
Chat with me in our community discord: / discord
ControlNet tutorial and install guide
• NEW ControlNet for Sta...
Control Lights in Stable Diffusion
• Control Light in AI Im...
LIVE Pose in Stable Diffusion
• LIVE Pose in Stable Di...
My workflow to Perfect Images
• Revealing my Workflow ...
Ultimate Stable diffusion guide
• Stable diffusion tutor...
The Rise of AI Art: A Creative Revolution
• The Rise of AI Art - A...
7 Secrets to writing with ChatGPT (Don't tell your boss!)
• 7 Secrets in ChatGPT (...
Ultimate Animation guide in Stable diffusion
• Stable diffusion anima...
Dreambooth tutorial for Stable diffusion
• Dreambooth tutorial fo...
5 tricks you're not using
• Top 5 Stable diffusion...
Avoid these 7 mistakes
• Don't make these 7 mis...
How to ChatGPT. ChatGPT explained:
• How to ChatGPT? Chat G...
How to fix live render preview:
• Stable diffusion gui m...
The FREE Prompt styles I use here:
www.patreon.com/posts/sebs-hilis-79649068
Man these videos are so good. I love working with SD, and now I have found a way to control it and a channel that explains it so good! Keep the great work!
Glad you like them! Thank you for the encouraging words 😊
Thanks for introducing use of control net in your workflow. The "little" details in settings and technique all add up to..the next level
You're very welcome!
I would recommend to use control as well in img2img when you upscale so you don't need to lower the denoising as much
Man you are really good with this stuff. Keep going 🤗 and thanks for the videos.
i love your tutorial, subscribed!
Another great one!! Thanks a lot!! 😊
Thank you, your videos helped me a lot!
this video is so well explained, and your voice makes me feel of those meditation app voices.
That is really awesome. Just, amazing. Thank you for sharing this with us.
Glad you enjoyed it!
Seb always show's us the best! Love ya man!
Thank you superstar!
I can't believe the release cycle of these features. Photoshop etc. might release a new feature once a year, and now we're at one every couple of days.
I'm still trying to figure out how to re-light a real face. Do you think any of the following might work?
1) Create a Blender 3D model based on the depth-map produce by ControlNet, and actually render a new face with desired lighting.
2) Carefully create a mask in Photoshop so use with Inpaint. But I don't know if gray will count as a partial mask or as a total mask.
3) Some clever combination of ControlNet models..
Do you have any ideas or suggestions about it?
His previous video showed how to blend a light source with ControlNet. A good place to start and then you can adjust the values for more/less light. Cheers.
That’s the power of passionate open source. AI plus open source will completely change our reliance on proprietary software and tech monopolies
@LDFArt Can you dreambooth a single image? I don't want it to kind of look like me-- I want it to exactly match my face in a particular moment.
@@benjamininkorea7016 maybe you should use a beautiful face instead of.
@LDFArt Yep, that's fine, but will it allow for lighting effects? Not much point having my wife in a beautiful moonlit swordfight if she has her regular indoor face lighting, right?
Very nicely done and very informative. Have to give controlnet and the pose control a try
Thank you. ControlNet is the real deal.
Amazing
Thank you! 😊
Ty Seb , very nice infos !
My pleasure, my friend!
You are a King
And you are a superstar for being so positive 💫
wow, very cool!
Yes! That's exactly what I was realy eager for with colntrolnet)
Thanks!
Ooh if this does what I think it does I was literally just dreaming of this feature.
I used to use the 'sonar' script for this but it was kind of broken. I was trying to reverse engineer it to make it work with multiple images.
This just made my day.
Happy to hear it! Go wild with it
I was wondering when this would become a thing haha that is just amazing
bro, your puns are gold
Came for the SD knowledge. Subscribed for the pun :) Either way lots of fun.
Would love to see how you'd tackle this multi-pose approach with consistent character models.
i.e. Instead of astronaut as text prompt that it generates variants of into the open pose, having a text prompt that associates to a character which uses the open pose. Goal being to compose a comic book scene essentially.
Thanks for the sub! I think for your usecase you will need to train models for your specific characters. But it's possible.
@@sebastiankamph is it possible to make Dreambooth one of the controlnets?
Another fantastic tutorial Seb! Thank you. Just throwing out a wish here... is it possible to bring in a sequence of images into the ControlNet(via depth map, open pose, whatever the style)? Just would like to know if animation sequences can be produced with the influence input of the ControlNet in any way.
Yes, can use CN together with animation.
Loving all of your vids! Where can we find the openpose editor? I'm not finding it on HF.
Hey, check one of my previous vids for that. 1 or 2 before this one.
Great tutorial! How do I download the SwinIR_4x upscaler? I've looked for it everywhere but can't find it :(
Hey! thanks for the video. It is amazing! How do you increse the word tokens to 150 ?
Nice share :)
Hi, Kamph, do you know how to control color outputs with text2image generation, for example, i need green hat, white dress! I can not do this without inpaiting.
hi.. how to train the SD local install with a particular character so that we can use open pose editor to regenerate the same character in multiple poses and background scenarios?
wow they innovate so fast! I love open projects like this one :)
Why does your webui shows you a preview of what it's working on? mine only gives an output by the end of it, is it a toggle I need to switch? (I also miss previews in my LORA, textual inversions etc.. so probably it's a problem on my webui)
1:57 Hi, where did you get Open Pose Editor tab? I don't have one, is this a different extension?
Nevermind, I found all the answers I put on you in last hour, thank you
Nice video🥰! But I only have one controlnet to use and you have controlnet 0 and -1, do I need to download any extensions?
Thank you! Not at all, just make sure you have the latest version, then go into settings - controlnet, there you can change how many you want displayed.
I have ControlNet installed it shows in interface and works, but: it doesnt output controlnet's image (i mean scribble, depth image etc), and i also dont see ControlNet in settings. How to fix it?
Edit: to fix it just update gradio ( pip install gradio==3.16.2 )
🙌
Great breakdown of workflow to make cool compositions! You said to send to I paint in the end...is there a way to have Automatic1111 SD upscale work on each tile separately separetly than apply the controlnet image to every tile....that would replace the need for a lot of inpainting
If you solve that, let me know!
Is this available as a Colab notebook without Gradio? Can we do each of these tasks with just python code and parameters?
Hey :) Can you please share how we can get the model depth leres? Is it included when updating the extension?
Yep, just update :)
Thanks bruh,you are the grand master of SD , also you look like one of my favorite actors , John Malkovich
, do you get that a lot"" :)
Haha, not usually, but thank you 😁
Have you noticed Deforum Updated to contain ControlNet? Yeah, ControlNet+Deforum.
Yep, but it's super buggy sadly
@@sebastiankamph Indeed. My first install broke everything. I'm in the middle of a big project so I'm sticking with the pre-ControlNet version for now. Thank you so much for your videos BTW.
always an insta-click with your vids!
Thought I was the only one 🤣🤣
You guys are the real MVPs!
Still I cannot create consistent dress in my characters, There is always changes in clothes. do you have any tips for that ?
Hello. Have a way to make it reproduce a character that was generated before in SD in other composition, the same character with same clothes ?
Not really. Gotta use Dreambooth or Lora for that.
sub-a-dub-dubb. keep 'em coming ❤🔥
Will do!
Where can I have access to the prompt he uses ?
Haha my vacuum cleaner is kept in its faraday cage while not in use, i keep a tight leash on that little spy, im way more worried about the cat in the building tbh.
Jokes aside :) i love you keep doing these, and how fast!, seems like the only way to keep up with this revolution because holyy its going at mach 5 now.
I feel i barely got acquainted with controlnet, and now this opens up so many more options :)
Imagine I have to keep up making videos too! Slow down developers 😂
@@sebastiankamph Yeah haha i dont know how you do it =) But keep at it!
What are the "heely presets" are they your own custom presets or is it a library or?
Made by Hili in our Discord. Check #resources channel
What is the VRAM usage per controlnet … as for me on default settings with LowVRAM enabled it uses between 2-5 GB of VRAM and depending on my loaded model it OoM’s very frequently (safetensor pruned models)
Install --xformers easy to install theirs videos , had same problem i can now do bigger images an actually play around with controlnet
at 2:56 where you select the preprocesor and model. I am able to select the matching preprocessor, but for the Model i have no options. someone help please!
You need to download the models. See my first ControlNet tutorial.
I'm dying to try Controlnet. I just hope the models it uses aren't too heavy because I don't have any space left on my Google drive.
I don't really understand those dots supposed to help gain control over head and eyes position. Both eyes (or what I believe are eyes) have different colors, so I though that reversing (move left to right and right to left) all control bones, including limbs, I'd be able to control if the character is seen from the front OR from the back. But no matter what, with ControlNet all I get are people facing the viewer :/ Which sucks in some cases. If you manage to figure that out (if there's a way ?) I'd be happy to hear about it ;)
I think two of those are ears, or side of the head. So should work
This is amazing! I was just contemplating the other day, after reading the full ControlNet paper and understanding about 60% of it, whether you could chain ControlNet models together or if that would require specific training of combo models. I guess the answer is "yes, chaining works," and I LOVE it!
Now... Can someone with more knowledge and a better computer please train a palletizer model to force generations to match an existing color palette? It would be great for game art, or just consistent styling across generations. I know you can do this with Img2Img, but then it also retains the structure of the palette image (bands or blocks, even at 100% denoising), which isn't ideal.
I love the rate of progress on open source, public AIs!
Is anyone else having error: "RuntimeError: mat1 and mat2 shapes cannot be multiplied"?????
@@V0odo0TV Stop spamming your questions....
That's just style transfer. You can train an embedded and use it to add the color palette to any generated image using any compatible model.
@@axolotron1298 I can't train anything; my computer's hardware isn't good enough. I once tried to train a LoRA model and it was going to take 8 hours just to generate the base category images before training even started.
@@IceMetalPunkUse Google colab. I've had success training on those.
Hey there! Nice vid, as usual! I am trying to have ControlNet to work but everything I do, it ignores my ControlNet enabled instructions. Even trying the smallest thing (like your pengiun example) it always ignores the ControlNet drawings. Even with the canny bird, no "map" is displayed alongside the results and the images are nothing like they should. Am I doing anything wrong? Thanks
Sorry if it's a silly answer, but I made that mistake a couple of times. Did you tick enable under the controlnet panel in txt to img?
@@StevenHuxley Yeah I enabled it. Restarted the cmd process twice and it started working again. Go figure. :)
Are your presets available somewhere?
I've started posting them in the video description, so check one of my latest vids and it should be near the top.
When I click apply and restart UI it just goes to Page cannot be loaded... and I don't have the multi option...
Nice one. Suggestion for the mic, I think you have some clipping, limiter plugin might help with that. Or maybe you can put mic a bit further away and speak louder a bit. Note: asmr vibes
Thanks!
This joke landed well 👍👍 😉😉
Haha, wonderful, glad you liked it 😊
Rrar channel, clear and usefu
that happened faster than i expected
Is there a reason you're using batch count instead of batch size? Batch size always renders faster for me.
Probably because they keep moving them around in the ui 😂
Hey Sir, I find it that the video segmentation on your video didnt work, you may want to look into a different formatting of it to work.
PS: Nevermind I thought it was segmented but they are only videos.
I'm beginning to wonder If I come to your channel to learn cool stuff (I always do) or to get a new awful pun to repeat to others.
Why can't it be both? 😂
Hi Sebastian, what machine do you use for all these AI work? Could you share its specs? Thanks!
Ryzen 9 5900x, 32gb, rtx 3080
What is he using? I know the stable diffusion a.i but where are all these settings? Does he have stable diffusion open in something else?
Stable diffusion automatic1111, the most popular gui with the most settings and extensions.
@@sebastiankamph thank you .
@Sebastian Kamph do you have a how to video to get it all setup like you have?
Hello! What is your graphic card?
Rtx 3080
Thanks for your guidance
I will upload an image1 of a person
I want to upload an image2 of an apple
I want to make the person in the image 1 eat apple in image 2 with his left hand
Later I will upload an image3 of an orange
I want to make the same person in the image 1 hold orange in image 3 in his right hand and eat apple in image 2 with his left hand
How to do that ?
My constraints : I don’t want to install softwares on my computer. I want to achieve in some websites which freely allow me to do that
I will be highly grateful for your help
👋
Mvp supporter!
Great video. Some feedback though would be to try find a way to remove the saliva sounds when you’re talking. They’re a little horrible to listen to.
Thanks for the tip!
But no 6gb vram yet
Hard to live in the present with hardware from the past ;) It's like complaining that CyberPunk2077 doesn't run on an Intel Pentium 90. Just gotta live with your time ;) Coders aren't gonna spend their days coding for old hardware.
Bro. I've been 2 weeks without my main rig and I'm already like 2 months behind in terms of features I have to learn.
Binge when you return!
Hili presets! 🙌
They're great! Thank you as always, Mr. Rocket. You're my biggest supporter 😍 We miss you on Discord though!
@@sebastiankamph Yes, that's right, I'm a little tired of Discord right now.. but I'll be in touch more often in the future. I'm glad you put everything that's new together in stable 😁👍
Thanks you so much for all of your work but plllllssss use Nvidia Brodacast for your sound. Your mic get all of your mouth noise pls pls pls i'm turning crazy
is it free or a subscribtion based AI gen?
100% free. Running on your own machine
@@sebastiankamph Thanks a bunch man!
gpu?
Rtx 3080
That Vacuum jokes.... it really *sucks* :D
Heeeeee
Spectral Desser... Please
And a de-clicker, misophone's will have heard attack
Thanks, I'll work on it!
What the time to be alive pipi
It would be better to apply layer by layer and composite.
Can't understand
I love your videos, but the smacking sounds are killing me :)
Currently researching how to solve this. But also don't want to stop producing content in the meantime 😅
Is anyone else having issues with openpose? My prompt gives the following error: "RuntimeError: mat1 and mat2 shapes cannot be multiplied (154x1024 and 768x320)"
Is anyone else having error: "RuntimeError: mat1 and mat2 shapes cannot be multiplied"?????
Try switching to a 1.5 model
full control…. hahaha😂
Lol even 1 controlnet is not working anymore since their updates this week.
Please put a noise gate on your mic, we don’t need to hear every sound a mouth makes
They need a de-esser for "s", "t", "c" and "sh" sounds and a proper pop filter for explosives like "p", "b", or "v"
I like the ASMR effect lol
Personally I like the low key ASMR
@@Oozywolf 🤮
Let’s stay in AI zone and use izotope RX software, which have plugin do declicking
I realize this is mostly a me-problem. But, i think the compression on your mic is a bit too high, because it increases the volume of "mouth noises", as in lip-smacking and such, which i find unbearable. Probably a level of misophonia.
Check some of my later videos to see if you feel the issue has been improved. I've implemented a lot of changes to my audio mixing since this.
I love your videos sir, but that dad joke...well, it sucked all the air out of me...;-)
😂
I'm reporting you to TH-cam for that vacuum joke. It sucked.
Boom! 💥