Style Transfer Adapter for ControlNet (img2img)
ฝัง
- เผยแพร่เมื่อ 28 ก.ย. 2024
- Very cool feature for ControlNet that lets you transfer a style.
HOW TO SUPPORT MY CHANNEL
-Support me by joining my Patreon: / enigmatic_e
_________________________________________________________________________
SOCIAL MEDIA
-Join my discord: / discord
-Instagram: / enigmatic_e
-Tik Tok: / enigmatic_e
-Twitter: / 8bit_e
- Business Contact: esolomedia@gmail.com
_________________________________________________________________________
Details about Adapters
TencentARC/T2I-Adapter: T2I-Adapter (github.com)
Models
huggingface.co...
Ebstynth + SD
• Stable Diffusion + EbS...
Install SD
• Installing Stable Diff...
Install ControlNet
• New Stable Diffusion E...
![Tutorial : ToonCrafter with ComfiUI [Interpolation cartoon]](http://i.ytimg.com/vi/TShrifdcRf4/mqdefault.jpg)
![Tutorial : ToonCrafter with ComfiUI [Interpolation cartoon]](/img/tr.png)
I can not wait to apply this to my AI Animations. This is a huge game changer. Using less in the text prompt area is a step forward for us. Having two images being the only driving factors should help a ton with cohesion/consistency in animation
I agree. Definitely makes it easier in some aspects.
@2:47 thank you so much for that i was thinking of reinstalling controlnet just because of that
Guidance start is about *when* it starts to take effect.
The color adapter works for me, but the style adapter does not. The Guidance Start value doesn't change anything. The result is the same as when ControlNet is turned off. Please tell me how to fix this? Thank you!
Happen the same to me. did you figure it out?
@@miguelarce6489 the total number of tokens in prompt and negative prompt should not exceed 75
incredible amount of useful information in this video. thank YOU!!!!!!
No problem 👍🏽
Hi! I really ask for help, I'm desperate :( Clip Vision preprocessor is not displayed (automatic1111), and I can't find where to download it. What am I doing wrong?
i try use this new controlnet extension and get not style in generated result. i remove all prompts (using img2img with 3 controlnets activate: cany + hed + t2iadapter with clip_vision preprocessor), in generating process appears error: "warning: StyleAdapter and cfg/guess mode may not works due to non-batch-cond inference" and generated result appears with not affected style =( frustrating .... i try many denoising strengths in img2img and many weights on controlnet instances without success... not applying the style on final generated result =( try to enable "Enable CFG-Based guidance" in contronet setting too, and still not working =( anyone got this same issue?
I need help, having the same issue, when I generate it just generates a random image...
love your clear instructions.
I'm following along but my system seems to get stalled on the HED and clip vision controlnets. Any tips for when this happens? I keep restarting.
Am trying same steps but first only doing one control net at a time to see if it works, and then adding each controlnet after it successfully runs. So far the HED is definitely slow to run. After this will try clip vison/T21style by itself (as one control net tab).
What does your height and width look like? I ran into a similar problem and had to reduce the size to make certain parameters work. Might be that it can’t handle it
@@enigmatic_e 512x512 I found this video (the guy mentioned some issue and how they got fixed) I will watch it later, I attached link below in case of interest. . the controlnet LED tab seems to be an issue. Is there a reason you use 3 tabs of controlnet? I'm testing out one tab of clipvision/t21 alone. It's still running. th-cam.com/video/tXaQAkOgezQ/w-d-xo.html
@@enigmatic_e which video of yours do you show how to add gitpull into the code? I think it was your video? I need to access my webui-user.bat to add something to the code but I can't recall how to do that. Thanks if you have link to video you made where you showed that.
@@judgeworks3687 i think this one th-cam.com/video/qmnXBx3PcuM/w-d-xo.html
@@enigmatic_e yes this was it. Great video. I ended up uninstalling SD and re-installing from the video you sent me. Thank you!
Great video :-) Thanx for sharing
Really cool thanks for sharing! I wonder what would happen if you put 3d wireframes in the controlnet lines instead of the generated ones, could be very temporally stable
It turned Barret into President Shinra 😂
Great Video ! Trying to update my xformers, it seemed to install a later version of pytorch that no longer supports cuda, and the version of xformers you used is no longer available, how do I fix this?
I need help, when I generate the result is way different than the actual image im using.
Thank you! very interesting!
Thank you!
What should we choose as Control Type? All?
Also, noticed that generating image with txt2img controlnet with given image it takes veeeeery long time, though my machine is decent. Do you have the same?
i believe there should be a box you can check that says "Upload independent control image"
@@enigmatic_e THANK YOU!!!!
Hey I was going to tell you how to get after effects to pull in multiple png sequences and autocross fade them. You have to pull them in and make each ebsynth outfolder it's own sequence. Then right click all the sequences, create new composition, in the menu there is an option to crossfade all the imported sequences. Specify with the ebsynth settings. Voila!
Ahh ok! I will have to try this! Thank you for the info!!
any tips on how to build this with ComfyUI?
Looks like Gen-1 will have competition…
hi Sir., thanks for making those good videos on style transfer, i have a question , is there a way to change a person's outfit based on a input pattern of picture? using style transfer and inpaint? thanks in advance
this is really a cool way to get it ... thank you!did u explain the way to bring a whole img sequence into comfy to get your great 0:20 result? thx in advance!
No I haven’t. I still need to get into comfy, still haven’t tried it yet.
hi, I have created more than 70 images with stable diffusion, and I would like to know how I can transform these photos into a moving animation with the same photos, could you help me?
Other than using premier pro or any other editing software, I know there are websites that can make your image sequences into videos. I’m not sure which one is a good choice though I’ll have to look into it.
@@enigmatic_e if you create a tutorial, or find the sites you mentioned before, let me know :)
@@PlayerGamesOtaku working it right actually
Thank your very much
Nice find bro !! ....yeh consistency is still a problem with video, i find lowering the denoising strength helps but then you lose the style,..its a double edged sword ⚔️
True, it’s always the struggle.
Great video man! I been learning so much from your vids. it might be a rookie quesion but I got stuck on the third controlnet model when you choose clip_vision under the preprocessor, I dont seem to have that option. Is it becuase I have to use a certain version of SD? thanks!
Thanks! Have you updated everything? Like SD and ControlNet?
@@enigmatic_e oops. that solved it! thanks for the help! looking forward to your next vid man, great work :D
@@SHsaiko Please describe what have you done, exactly please? I have updated all requirements but still there is no clip_vision on the left drop-down menu
Have you looked at the new Ebsynth_Utility extension to A1111?
Wait what??
@@enigmatic_e new enigmatic eb synth utility video incoming
@@enigmatic_e Automates most of the process.
Ur a legend
If I run more than one ControNet tab I get the CUDA out of memory error (8GB of VRAM GPU). Any suggestions?
have you tried checking the low vram option in controlnet?
@@enigmatic_e Thank you for responding. Yes, to no avail :-(
@@j_shelby_damnwird try lowering dimensions, that might help
@@enigmatic_e Thank you. Currently trying 1024 x 768- Will give a go 768 X 512.
I definitely need to grabe me one of those fancy new GPUs :-/
@@enigmatic_e Will give it a go. Currently trying to output 1024 x 768. Maybe 768 x 512 will do the trick.
I really need to grab me one of those new fancy GPUs :-/
i dont have the clip_vision prerprocessor where do i download it ??????
Did you update?
@@enigmatic_e I also can't find it. I did git pull inside the ControlNet folder, reinstalled it 2 times and still can't find it
Ahhh yeah
This is can make video and free??? I tried disco difussion, thats like trial.
не работает!
Ebstynth + SD link
My bad, just updated link, th-cam.com/video/47HpHOLkIDo/w-d-xo.html
I am not getting great results like you out of the box, i have to play a lot of slides for starting showing, wondering what i am missing here lol
I follow the instructions from the Aitrepreneur channel and it worked for me.
happy you figure it out.
I made a post about making sure you deal with the ymal files, but I think it got deleted because it linked to Reddit. Anyway, there’s some work to be done if you haven’t.
is it from the stable diffusion reddit?
Thanks for another informative video. This style transfer feature already makes Photoshop’s Style Transfer neural filter look sad by comparison. It’s clear that Stable Diffusion’s open-source status, enabling all of these new features, is leaving MidJourney and DALL-E in the dust.
Yea, it’s getting so good!
Shows "where to download it" with a list of .pth files..
Then shows his folder where he's got safetensors and yaml files..
Great tutorial.. just making stuff up and not actually showing where or how you installed anything.
Why did you provide a Link for style Adapter, but Not for the clip_vision preprocessor?
Hello. What about style transfer in images?
I'm looking for information about this but haven't found anything.
For example, I want to make a series of images of animals. I have a photo of a tiger, a pencil drawing of a horse, a pencil drawing of a bull (but by a different artist), an ink drawing of a wolf, a watercolor drawing of a cheetah... and I want to transform them so that all these images are done in the same style, like as if they were painted by the same artist. Is there any product that can help achieve this goal?
Unfortunately this tutorial is outdated now. I havent messed around with style transfer lately so I dont know whats a good alternative at the moment.
Thanks for sharing this! Very cool that they’ve added this style option. Excited for your next video on connecting it with eb synth. I’ll watch that next and see what I can do as well.
But damn, these Davinci Deflicker/Dirt removal render times are killing me haha
I feel you on the deflicker. Sometimes stacking too many is not a good idea 😂
@@enigmatic_e have you found a good compromise? I tried just one and it wasn’t great. So I stuck with the 3 you stacked after the dirt remover. But 24 hours for a 30 second clip is unsustainable lol
@@BeatoxYT no not yet. I’m sure another faster alternative will come out soon.
where do you get clip vision even from?
if its not in the link i provided, it might appear when you update everything