SDXL ComfyUI img2img - A simple workflow for image 2 image (img2img) with the SDXL diffusion model
ฝัง
- เผยแพร่เมื่อ 28 ก.ค. 2023
- In this quick episode we do a simple workflow where we upload an image into our SDXL graph inside of ComfyUI and add additional noise to produce an altered image. I show you how to drop it into a standard workflow as well as how to adjust it to get as much difference we we would like. This is a simple graph and can be used as a point of departure for your AI art or other stable diffusion projects. Note that I don't go overboard here with the noise, and you can really push this into another realm if you add noise over 50%, so feel free to experiment!
If you are confused, check out the SDXL graph basics here: • SDXL ComfyUI Stability...
#stablediffusion #sdxl #comfyui #img2img
Grab some of the custom nodes from civit.ai: civitai.com/tag/comfyui
Here is the manager: civitai.com/models/71980/comf...
Grab the SDXL model from here (OFFICIAL): (bonus LoRA also here)
huggingface.co/stabilityai/st...
The refiner is also available here (OFFICIAL):
huggingface.co/stabilityai/st...
Additional VAE (only needed if you plan to not use the built-in version)
huggingface.co/stabilityai/sd... - ภาพยนตร์และแอนิเมชัน
i love that you build your workflows from scratch and explain each node !
Glad to hear that! I think it is important to know how it all fits together.
@@sedetweilerDid you upload the workflow json somewhere?
Exactly this.
Your tutorials are really great. They convey so much useful info in an easy to digest way. Thanks!
Glad you like them!
Thank you! Really love working this way with SD. Can really fine tune the process to the graphics card and squeeze every drop out of the hardware.
This worked great!
Now I think I know enough to tie multiple previous tutorials together, to get several advantages. I don't know how to disable and reroute certain things easily, so it looks like I'll be maintaining multiple workflows.
I just want to say something else that maybe others have missed regarding your excellent tutorials. It's not JUST that they're technically good, it's also because your voice is SO easy to listen to. You could teach all day and it wouldn't be exhausting for the student I think.
Wow, thank you!
yes he has a voice for radio@@sedetweiler
1000% agree
you just fell in love with Scott, admit it
yep
Thankful that you are repeating the first steps in these videos. Makes it really a lot easier to remember
Glad it is helpful!
Thanks again for your video series here - I think it's very important for those picking up Comfy UI - it has helped me understand workflows I've loaded up off of the example workflow site. And I really appreciate it when you zoom into a node you're doing something interesting with - too far out and it becomes a compressed blur.
Glad it was helpful!
wow ! I love your tutorials, you are the best resource on comfyUI topic. I'll grab that "manager" stuff to have custom nodes.
You are SIMPLY THE BEST !!! fluent, effortless, snappy, concise, to the point, crystal clear,... you name it, man you are a Godsend !!!!! 😘😘
I appreciate it!
Amazing to see you are back to regular uploads😊
Yes! Thank you! I just had to work that life balance thing, but I have it sorted now. Cheers!
I think this is still one of the best and easiest workflows that gives quick results, Thank you!
Glad you think so!
I would love a tutorial explaining how to create 2 independent prompts, run them together for composition. Method (1) : Using position & area numbers (such as, starting from X0-x300, y0-y200 for prompt 1, etc.) Method (2) :Allowing the mouse-drawn mask area to be fed with the prompt -- please include previews of the masks being done & a preview of the mask combination. I'd really appreciate it. ComfyUI allows for greater composition control & understanding how to do it will really lean into it's strength
Im just got into image generation and thank you for your vids. I truly appreciate you sharing your knowledge 👍
omg I've been working for hours to try to figure out how to integrate this into the refinement workflow from earlier in the playlist.
Did you get it? Just checking that you are all good now.
@@sedetweiler
Edit: I think I figured it out! I had things in the wrong order and some values were a bit wonky.
5 img2img steps, denoised to 0.80
->
3 refiner initialization
->
20 base sampling steps
->
12 final refining steps (not sure about this one)
--
I don't think so.
I started with the refiner workflow, ending with a final image. This is an exact copy of a previous tutorial and it works 100%
At the end of that workflow is a KSampler. I take that and split it off its latent.
I take that latent and pass it through an image scale to side, with the side length of the longest of height or width I've been working with.
I pass that latent to a LatentAdd and bring it into an exact copy of the img2img tutorial.
I take that LatentAdd and insert it just before the img2img KSampler.
It.. "does stuff" but I intuit it isn't correct.
I already cleaned it up with groups (which was new to me), and I'm working on cleaning it up with reroutes so I can show it. I don't like that reroutes are someone else's extension because it feels like something that can break on updates, but I hope that reroutes are popular enough to demand updates.
You're awesome scott thanks for all these tutorials
My pleasure!
Started your tutorials awesome work great teacher! Excited to see learn more!
Awesome, thank you!
Perfect Scott, thanks again
Excellent tutorial, i cant wait to see how far you can go with ComfyUI
You and me both! :-)
@@sedetweilercaan i batch modify 10 frames i have like we use to do in old stable diffusion?
The 10 frames are about the same extracted from a video and I want to batch modify them in a similar manor like in the normal stable diffusion
Can i do that with sdxl?
can you help to use controlnet on the comfyUI?
Your tutorials are amazing and incredibly informative. Really hoping you cover SDXL Lora training and maybe even animation creation using ComfyUI at some point?
Those are coming soon!
@@sedetweiler Awesome! Can't wait!
Amazing, Thank you.
Glad you liked it!
super awesome, thank you
Welcome 😊
I would like to see a second part of this video from img2img adding ksample refiner to the image to improve the composition, thanks for the video and good work
its very easy to do and gives good results!
Love it!
Great video! Can you share the json file for this work flow?
This is great! How can we generate (not process) a batch of images from img2img?
Thanks a lot! again.
many thanks mister
Yup yup!
It was nice spending the weekend with you, boss. Well, with your videos, anyway. You've got me psyched on ComfyUI.
Two things I hope to see you cover, eventually:
1.) You mentioned in another video that ComfyUI might be used for model training? Yes, please; and
2.) RE: the .yaml tweak for sharing a model folder touches on a larger issue. Play around with AI for a few days, and you end up with Oobabooga, TavernAI, Docker, Jupyter, the WSL, half a terrabyte's worth of LLMS, et al AND competing versions of Python (for X you need 3.10, for Y you need 3.11)... then there are the Condas and Cudas and-- you get the picture. As methodical and organized as you are, I'm guessing your hard drive is set up logically and efficiently. A peek into the nuts and bolts of your setup would be, as the kids say, amaze-balls...
Your pal,
-jjg
Awesome to hear buddy! 🥂
Scott: I am really enjoying your videos. You mention in your videos if we have a question to drop it in the comments. I have a more general question about Stable Diffusion, ComfyUI, Automatic111, and MidJourney. Is it best to post here or somewhere else?
Hi Scott
When I used stable difusion 1.5 pre comfyui there was an option to render next to the hirez fix for tiling .Is there any way to replicate that in comfyui as I do a lot of seamless patterns ??
What about inpainting using a mask? my current workflow relies a lot on a loop that looks like:
prompting>inpaint>img2img>inpaint>img2img>inpaint... etc.
Thanks a ton :)
You're welcome!
I have the latest Derfuu nodes but I only have Width or Height Options on "Image scale to side" and not "Longest". any ideas why?
I would just do a pull off the latest. Might be a new addition.
lol didnt realize how short todays tutorial was going to be. Noice!
On to the next!
Woot! Speed is the key, brother!
Awesome!, finally I can create my first breathtaking images on my desktop! could you please create one tutorial to explain how can we design separately the character, the background and then combine?
Yes, soon
When I followed this tutorial , after I created the primitive node and was going to connect it to the CLIPTextEncodeSDXL node as you did @0:54 I had a Text_I linker, but not the one for the text_g linker. As you do in this video. What am I doing wrong? I'm building this workflow underneath (and disconnected from) the default workflow. Could that be causing it?
Nice tutorial. What's the advantage or difference in using Primitive plus CLIPTextEncodeSDXL compared to using CLIPTextEncode(PROMPT) for your prompts?
You can use both of the CLIP models if you use the SDXL one, otherwise you are only using one.
The shortcut key for the search box in the video, I want to know it，Thx Bro！
idk if it's a stupid question to ask but I'm a newbie soooo.... Why does the SDXL CLIP encoder node have two text inputs? At first I thought it would be for both the positive and negative prompts but apparently it isn't. I don't get why you need to connect the primitive node twice lol.
There are 2 different CLIP models you can leverage. I will go into this deeper in another video but for now, they are both either positive or negative.
Would a "base" or "refiner" model be better for img2img?
I guess you can also use the scaler to upscale the final image output?
I put out a video today on that, since you can do that, but there are much better ways to get great results.
thanks a lot for this, could you do a tutorial on how identify and solve common errors? like when load image refuses to load image.
Sir is it possible to use this sdxl online? like without using the GPU since my laptop is not very powerful but I would love to dive into this stuff. I am very new to all this so forgive me for asking the obvious.
Hello brother, what is the function of the "clip space" on the menu on the right?
how do you make your projects run and compile so quickly?? I tried using a VM on run diffusion and it still takes 2-5 minutes. Do you have any other VMs you would recommend that are affordable?
thank you for the content
I wonder whether there is workflow for i2i in batch because I want create video.
In ComfyUI is it possible to change the folder the saved images go to?
How is it that all your width and height inputs say 1024, yet your output image appears to be a portrait aspect ratio?
Hi, thanks or the video, I wanna add control net into this node, use img to img to do some line art animate style but with is it possible? I tried to combine the node that you share about controlnet, but not success, thank you.
Just started to learn ComfyUI. When I select a primitive it does not have the 'string' output on it? Mine only says " connect to widget input " . Next time could you have a json file or PNG image that contains your workflow. Thanks for the tutorial
The primitive will adapt to what you link it.
@@s.anonyme6855 thanks for the tip
does gfpgan work in comfui i keep getting error ('tuple' object has no attribute 'cpu')? can you make video about best roop implementations?
Are you running from a CPU or gpu?
I'm using a CPU and haven't been able to get Roop to work in either comfy or a111.
@@justinwhite2725 gpu (amd) and roop works on both comfy and a111 the result is just realy pixulated if its an close up image so i thought gfpgan would make it better but it doesnt work
great! is there an option to use that for a image sequence in an automatic way? thx in advance!
You can just drop it into the middle of any workflow.
@@sedetweiler thank you ...
is there a way to do higher batch number than 1 at a time....i use an Efficient Loader that combines Model, Prompts, VAE, and Batch Steps in one node. When i increase the batch number to say,3, Comfy only does 1 image.
Perhaps the batch number is being bugged when part of the group? That is how I would normally do more than one per batch.
What if there's a refiner do i use the VAE of the refiner or the base model ? and which one goes to the vae decode ??
I would try them both, as each has a personality. You choose the one you feel is best for your situation.
Is there a way to add a trigger or something similar if you want to keep the image can you trigger an upscale of the image?
Yes, you can use the Image Chooser node (search for choose).
Thank you for getting back to me. I am just starting to use Comfy and Your tutorials are the first thing that have got me to see the potential instead of the confusion and the overwhelming screen. @@sedetweiler
It took me a while to figure out / learn that SDXL is different stuff with different models. Whoops!
Just in the step where you add ❝Derfuu nodes❞, it doesn't appear because when I install it, I get ❝import failed❞, what could I do?
that derfuu node, "image scale to side" whats the difference from using latent scale to side
This helps with text to video, but how do I do image to video locally? Text to video is kind of useless with LoRA involvement if we actually want good results
I take it the next one would be a ComfyUI+SDXL inpainting tutorial?
Yes, that is coming soon!
I will like to see how we can make portrait of yourself ( or other peronne) in comfy ui
Can you add a link to the manager? I cant see it on civatai
civitai.com/models/71980/comfyui-manager
could someone explain how to do this as a batch, to process a folder of images?
if i use comfyui in google colab, how can i get that Manager to work?
I don't think so. In the colab you can specify the mods you want to install, but I don't think the manager will work since a restart is required to make the new nodes show up.
I can't find your image scale to side I googled and I have checked the resources you provided none not even the manager gives me that node
Guys, I cannot find the installed custom nodes in the search field - derfuu among them :( when I install custom nodes in the Manager - via simple installation or via Github URL - it shows "To apply the installed/updated/disabled/enabled custom node, please RESTART ComfyUI. And refresh browser." But when I clikc on the RESTART button, it show only RECONNECTING... that lasts foerver. If I click CLOSE and refresh the browser, a blank page appears instaed of the COMFY interface - and I have to restart the SDXL. But it won't change anything - the node still won't appear :(((( Can anyone help me?
is there a good workflow for controlnet in CUI?
oh i see someone already asked... so - that tut next? :)
also also, are any of the custom nodes on CivitAi "questionable"? lots of flavors, but some look kinda sketchy.
Is this really image to image or is it how to use an image already generated with comfyUI in a workflow? For example, with stable diffusion, you can use an image as the input and then create from that image.
ty vm
You are welcome!
Can you please tell me how to save the image generated here?
Use the Save Image node at the end instead of the Image Preview.
@@sedetweiler how to switch to save image node? And Where it will be saved?
the node Utils folder is not apper
what about img2img with sdxl base + sdxl refined?
Sure! That works great!
@@sedetweiler It's confusing to put it all together properly. For base+refiner I use advanced KSampler, so I guess that I have to pass image latent to base KSampler, and then put start step > 0. This will be equivalent to denoise value.
That is correct, but I am trying to keep things a bit simpler as it is still early on in the series. You are ahead of the curve my friend!
thanks you so much from ukrainian soldier
Wishing you the best!
@@sedetweiler thank you, friend, your videos are very informative and powerful. I have already worked for 16 months at the front in the infantry.. now I am trying to grasp new technologies, although I suffer from digital critinism)
stupid question, how can i remove a slot from a node?
Depends. Which slot?
@@sedetweiler i Made a " target height" Slot wrongly in the sdxl..Text..node
@@mikebert so is it a nested node or did you code one?
@@sedetweiler its the one from your Tutorial, the second after the Text primitive. (I dont remember the exact Name). I solved it by making a new node
Thats nice but you skipped refiner which is a bummer, without it results are meh, why did you do that? How i can connect refiner now ?
It just made it easier for this video. You can always add in the refiner.
whats ur gpu?
lost it when you click Manager button which I don't have
There is a link in the description to go download that from civit.
👋
how come nobody takes comfyUI to the web - rather than needing a kickass gpu?
A service like midjourney, but comfyUI - or does it exist already?
I don't understand this compulsion to speak and point and click so rapidly. Not everyone is as familiar with this system as you are. For those of us brand spanking new to this interface, this tutorial borders on unintelligible. PLEASE, slow down and explain a little more about what you're doing and WHY. You speak and click WAY too fast. And I'm a native English speaker with six months experience with Stable Diffusion on another UI. Subtitles aren't available here for some reason.
In other words, follow the basic tenet of teaching: 1) tell them what you're going to tell them, 2) tell them, 3) tell them what you told them. It works. I taught hi tech subjects to adults for years.
Then, the tutorials will be of lasting benefit to everyone, rather than an exercise in noob frustration.
why are people, faces and hands still so f.cked up in SDXL? Even good 1.5 checkpoints generate much better results.. I really don't get into SD >1.5
Seems like the vae is struggling a bit with small faces, closeups looks insanely good
Hi do you have a video on this but using a second image as what you want the first to look like for that kind of look? so image to image style to image I guess? thanks
Hello great video GG so interesting @0:51' you add "primitive" where you find "logic" ? in my COMFYui i have not "logic" neither "_classifier" what custom node did you have added ?
In other hand my own "utils + primitive" dont look like the same of yours STRING + Text BOX -- mine is "connect to widget input" without text box
Thank you for your sharing your play list about COMFYui & stable are master class congratulations
Team Shibuntu