ComfyUI: Scaling-UP Image Restoration, SUPIR (Workflow Tutorial)
ฝัง
- เผยแพร่เมื่อ 30 พ.ค. 2024
- This tutorial focuses on SUPIR for ComfyUI, some core concepts and upscaling techniques used with SUPIR. Image restoration, enhancement, and some mixed techniques are used with the workflow to achieve the desired results.
------------------------
JSON File (TH-cam Membership): / @controlaltai
SUPIR Comfy UI: github.com/kijai/ComfyUI-SUPIR
SUPIR GitHub: github.com/Fanghua-Yu/SUPIR
SUPIR Model Downloads: huggingface.co/camenduru/SUPI...
Gemini Pro API: aistudio.google.com/app/apikey
------------------------
TimeStamps:
0:00 Intro.
00:51 Requirements.
03:40 SUPIR Nodes.
17:02 Correct Image Resolution.
24:31 Understanding Tiling.
29:57 Workflow Organization.
33:33 Workflow Summary.
36:26 Image Restoration & Upscaling.
40:12 Image Enhancement & Upscaling.
44:26 Mixing Techniques. - แนวปฏิบัติและการใช้ชีวิต
@21:29 Slight oversight. Connect the Width and the Height to the downscale from the Image Resize at the bottom (divisible by 32 group) and NOT THE ORIGINAL. Thanks to the user "dr.gurn420" for correctly catching this oversight. The oversight would affect only if Downscaling an non 32 divisible image. Would have no effect if using the rinse and repeat (upscale downscale method) as shown in the video.
Do you mean from the "Divisible by 64" group? I'm reading what you wrote here as exactly what is already happening in the video.
Additionally, are you supposed to connect the image output from the image crop in the "divisible by 64" group and apply that to the image resize image input in the "upscale factor" group? or are you supposed to use the original image as seen in the video? I dont understand why you wouldn't always use the cropped image especially if your upscale factor group is already changing the size to the cropped image heights & widths
Please Check the video further. All output from the image resize is connected to the switch which passes on to the upscale. If you are doing directly connect from the bottom group to upscale factor.
@@KQforever Yes divisible by 64 group (as per that section of the video), When making the video it I was still testing it was 64, then in between I had to switch to 32, finally went with 32. if it works with 32, it will work with 64.
Great tutorial. Appreciate the fact that you didn't just dictate what needs to be done but explained as many relevant parameters are possible, going into the "why" of them.
Thank you!
Very cool to see your workflow and learn about new nodes
It was a very long video, but it was a class that got me hooked one by one. I think I'll be able to try many things thanks to this kind, detailed explanation. thank you!😍
I think this is probably the most useful ComfyUI tutorial that I have ever watched (and I have watched tons). I am gathering together a suite of workflows that let me blend 3D generated product designs with AI actors and backgrounds and this workflow allows me to easily add a convincing level of blended detail. Most of all I was able to understand what I am doing thanks to your efforts.
Best TH-cam Tutorial for Photos... Thank you very much!
Thanks for all the effort you put into explaining every step!
This video is awesome. I'm brand new to all of this but hoping to be able to use this to upscale my landscape photography for larger print sizes. Hopefully I'll be able to figure all of this out.
Awesome tutorial, very detailed, many thanks for explaining all the parameters, that's very valuable!
Glad it was helpful!
This is great. Thank you!
This is one of the most difficult tutorials I have ever heard, almost did not understand, I think I will re-watch several times, hoping to finally understand, thank you very much
www.youtube.com/@stephantual
I agree with you. This is God level
hey - great video!
Hey, Thanks! Okay, you got the oversight accurately. Let me explain what I was thinking when I did that. If the image is divisible by 32, the crop will not do anything. If not, it will crop the image to the nearest number, which is divisible by 32.
For the downscale, you are correct. The image resolution is from the original, but the image source is from the cropped. The correct way would be to connect the cropped width and height to the downscale input instead from the original.
Why it doesn't matter. Because if you have a high res image, we are not going to use SUPIR for restoration. The image is already high res and proper.
The downscale method here is a rinse-and-repeat method, meaning you take a low-res image and correct it by making it divisible by 32, which is done during upscale, we then use that to downscale, upscale again (rinse and repeat). The divisibility by 32 is fixed during the first upscale pass. Therefor calculation will always be correct in the downscale in the second and subsequent passes, whether width and height are coming from the crop output or the new original.
This is an oversight from my side, as the downscale was never meant to be used for directly inputting a 4 K image, for example, and downscaling to upscale. It was meant for low-res, to upscale through multiple passes. The first pass would correct the divisibility by 32.
So, you are the first person to get this oversight correctly. To answer your query, the 100% correct way would be to take the width and height values of the downscale from the cropped output.
I got the oversight; however, I did not correct it since I only use the downscale for the rinse and repeat, so the res is always divisible by 32, no matter if it is coming from the original or crop output.
(by image crop output I mean, the width and height from the image resize node in the bottom group)
I have pinned a message mentioning the correction for the oversight.
Great nice work, I have a problem with the Gemini module, I have the api-key I put it in the json file as you explained but I have an error.
Thanks! Gemini will give error on any photo with a face. They have blocked people photos. So even if you upload on gemini pro in browser, it gives error. Everything else works.
What AI voice do you use?
O Brasil manda um abraço! But 2 hours trying to do your job, which is wonderful, but there are a lot of details. Could you not make the Workflow .json file available?
JSon is made available for paid channel members. However you can replicate everything shown. Video has everything shown how to create the workflow.
TNX! Great tutorial, but there is no way to get JSON File, cuz TH-cam membership not allowed in certain countries(. Any chance to get json from ur tutorial?
I show everything in the tutorial video. You can just follow that and make the workflow yourself.
@@controlaltai Yup, TNX again)
@@Gmlt3000 Definity unfollowed. People are taking this "workflow" too far as if they are the people that MADE the software. peace.
As a newbie to comfyui, I'm overwhelmed both in the complexity and the amazing results. Do I understand it all? No but I would love to play around with it. I have 64GB RAM and a 4090 gpu with 24GB VRAM. To download your workflow, I have to be a paid subscriber?
Hi, yes for the json, however nothing is hidden behind a paywall in regards to the tutorial. I always showcase everything in the video, showing how to build the entire workflow. But do appreciate any support given. 👍🙏
@@controlaltaiI see. How does your workflow differ from other Supir workflows available? do they have the divisible by 8 and 32 nodes etc that you have? ar they dynamic like yours?
@gohan2091 You can check out the SUPIR workflow by Dev himself, who implemented this, you will get it under custom node, SUPIR, examples folder. The SUPIR upscale group in the video that’s what the basic workflow consist off, everything else is additional and done by me.
Also the settings used for the sampler are very different. The workflow will give consistent accurate results, with these settings. The only limiting factor I see is the hardware. If I had better hardware I would just use full tiling for every image and be done with it.
The workflow has dynamic automation for practical use. For example, you don’t want to calculate every image manually, different tile setting is a unique approach, ability to switch things on the fly and so on….I cannot speak about other SUPIR workflows. I have only seen the default given by the dev.
Edit: divisibility by 8 is taken care of by comfy, not 32. The issue comes to convert to 8/32 both. You can always manually convert crop outside comfy. Images generated using Stable diffusion won't have any compatibility issues.
@@controlaltaiWe appear to have the same hardware. It's the best we can get at the moment (without spending tens of thousands) I must admit, I am not fully understand the 3 different modes but I will re-watch your video later. I am using the dev workflow now, cannot even figure out how to increase the size but it's definitely improving quality. I am having to guess settings though like s_chum, s_noise, DPMPP eta etc. I guess with your workflow, I would still have to guess the settings?
@@gohan2091 I do explain the settings in detail, it’s complicated. If you want consistent, high accuracy result, try with the settings I used, changes in the setting mostly result in creative output, which personally, should be avoided for upscaling. But yeah, for example if you want some color change or add more details etc, you have play around with it.
The tiles matter, in terms of vram usage. Typically close to original, will always give best quality, but above 2k it’s very difficult on 24gb vram. Secondly, on that setting it takes more time, if doing in batches, then maybe a middle ground would be a good balance on speed and quality.
It just good to have flexibility in the workflow.
It might be better to do an upscale or downscale image the first, and then resize the image to a multiple of 32?
The multiple of 32 applies only if it's not a multiple of 32. Technically it's doesn't matter. You would want the image to be as close to the original as possible. There is no image upscaling before conversion otherwise supir gives error. Downscale is case to case bases. Once the image is a multiple of 32. The entire multiple of 32 process is ignore and nothing in terms of resolution happens to the image.
I agree with you @andrewq7125 . This workflow makes the assumption that you don't start with a downscale. So as long as you start with an upscale it works fine.
I have a partially blurry image I wanted to try this workflow on with a size of 3168 (w) by 4752 (h). The workflow cropped the image down to 3168 by 3168 but then when downscaled 4x it becomes 792 by 792 which cannot be divided by 32. If I changed the crop factor to being divisible by 128 instead of 32 it worked fine. It just doesn't make sense to have to manually calculate those things in advance considering how automated this workflow is, so best to change the workflow to start with the downscale at the very least, bypass it if not downscaling and then crop and upscale.
That being said, it's a good tutorial though. As a beginner there's alot I've learned from this tutorial
Hi, I saw the oversight some days back, as pointed out by another user, and provided a solution in a pinned comment. If you start downscaling from the beginning, you just have to make one change.
"@21:29 Slight oversight. Connect the Width and the Height to the downscale from the Image Resize at the bottom (divisible by 32 group) and NOT THE ORIGINAL. Thanks to the user "dr.gurn420" for correctly catching this oversight. The oversight would affect only if Downscaling an non 32 divisible image. Would have no effect if using the rinse and repeat (upscale downscale method) as shown in the video."
I hope with this change, the workflow is now fully automated, even in the case of downscale first.
Sorry for the off-topic question, but I was hoping to rely on your knowledge and expertise. Do you know any xl models that have a strong understanding of various backgrounds and environments, such as the interiors of public restrooms, gyms, or trains
Ermm sorry, I am not aware on any such specifically trained models. However I am sure you will find something at civil ai. The only way is to download and test with your prompts as they don't say on what dataset the model is trained in.
@@controlaltai 👍 so seems like the best approach is to experiment with different models and find the ones that work well..in my testing even SD3 doesn't seem to have a perfect grasp of room structures, so it might be challenging with XL but yeah I'll see what I can find. Thanks!
Hello, I would like to inquire whether it is possible to attempt running this workflow with a 6G graphics card and 32G of memory.
Hi, no this workflow won't work with a 6Gb VRAM. I recommend 12 to 16 GB at least.
@@controlaltai Yes, I tried this workflow, but it didn't work out, haha... Hello, I would like to ask if you have considered developing a workflow that involves sketching, coloring, and assigning material textures to objects. From a product design perspective, this kind of workflow aligns with the product design process and is highly practical, but I am unsure if current technology supports it to achieve ideal results. If it can be implemented, I believe many people would enjoy it.
Great Tutorial iam thankful
Somehow in the node ' image resize' i don't have the 'multiple_of', its missing idk why.
Thank you! That image resize is from comfy essentials custom node, there are multiple image resizes look for the one with the wrench icon.
@@controlaltai iam sorry but its still missing i updated several times and days passed.
The ' multiple _of' is only missing, somehow.
I use now the size of the 'Supir Denoiser' preview to be able to get the nearest size that is devisible by 32.
If you know other solution to work with 'multiple of' you can tell me, please.
:)
Other solutions are complicated to explain in a comment. You have to do a whole math conversion set of nodes that check divisibility of 32 then send the image width and height to inputs of the image resize. I still think you are using the wrong image resize node. There are several. Enable tags to confirm which image resize node are you using in comfy.
@@controlaltaiSorry, whats the meaning of : enable tags to confirm.
I didn't understand how to enable tags, whats that.
By the way i have other question.
I did exactly the same workflow with the same reference image and seed and all things. Except the fp-8 unet , i have it on false becauss i have only 8000vram and 32 Gg system Ram.
Now the question is that i get other output than you got. The face is someone else, its not bad but its different than yours. Is it normal if this happens?
Go to settings and enable badges, this way you can see what custom node does the node belong to. It will show the custom node name on the to right. Okay so, about the different face, that's normal. This is because the image I used has a massive data loss, generative ai is what the name says it is. With every seed you may get a different face. Also using fp8 means less precission. The adding of new data won't be the same as fp 16 or fp32. People normally mis understand the word restore and think the ai would know exactly what the face looked like. Thats un true cause the AI sees what you see just bit more, the word restore comes since it maintains the structure of everything else, like it gets that there is an emblem of sort on the top of the hat, there is a strap for the hat, etc. So its completely normal to get a different face for such a massively lost image.
Hello and thanks for the video! I'm a newbie of ComfyUi and I am looking for a workflow img2img that let me "simply" improve realistic detail skin face (texture, pores, etc). Your workflow looks so great but to hard for me :) (my fault!)
Do you think I can find what I'm looking for?
Thanks again!
Hi, yeah don't use this workflow, this one is complicated and its not image 2 image. Check out this Ultimate SD Upscale tutorial which allows you to do image 2 image in Comfy easily. th-cam.com/video/NDw9AjjW1t4/w-d-xo.html
@@controlaltai Thanks for replying! I saw the tutorial and indeed is easy, but my problem remain, the img generated is still unrealistic (ok, I know fro the tutorial you used an art image) maybe I have to look for some upscaler that gives more realistic appearance to the skin? If it exist :)
Thanks!
@@giusparsifal skin is different, and realistic skin is very different, along with the upscale a specific checkpoint would be needed to add details only to skin, plus some Lora’s to make the skin natural. AI has a habit of making the skin like plastic. Civit AI would be a good resource for the checkpoint and Lora. Juggernaut, RealVisXL are two very good checkpoints for skin. Check the trained settings for steps and cfg. Hope this helps.
@@controlaltai yes, I generally use epicrealism as checkpoint too, the problem, for me, is create a workflow to get what I wish...
Thanks anyway, you're very kind!
SUPIT e Gemini ConfyUI is not available on ConfyUI Manager (there's no install button). I tried to install using git clone but without sucess. Someone can help?
Its SUPIR, try searching again. I can find it in manager. What environment are you running comfy in?
@@controlaltai Sorry, typing error. Now it works!, python is 3.11 (in the zip file ComfyUI is cu121); Congratulations for your explanation and patience.
Great and Thank You!
Hi, i need help, i have problem in 23:09 . when i connect "IMAGE" to "get image size" the line to original image disconnected. i cant double line like in the video. same problem when i connected "crop image output" to image resize downscale node. how to get double line like that??
Hi, are you taking about the switch ? Inputs? For that just drag and drop the outputs it will keep adding inputs to that node automatically. That's switch any node from impact pack
@@controlaltai No, in 23:09 you can connect multiple lines to a single node, "get image size" from upscale factor got two line, but mine gets disconnected and only allows a single line.
Get image size from comfy essential has a single image input, where am I connecting two inputs there. The switch any node has multiple inputs and only a single input goes to get image size. The get image size has two outputs. Please clarify or email a screenshot circling what you mean exactly.
@@controlaltai Thank you, sorry its my misunderstanding the node, still new here. i follow your workflow tutorial till the end and its WORK now!. Subscribed!
hi, after duplicating all your workflow, i can use restore dpm/edm sampler ,but when i use tiled restore dpm/edm sampler it caused error'
The size of tensor a (48) must match the size of tensor b (128) at non-singleton dimension 3', how can i fix this? many ths
Hi, When using tiled restore, ensure that the tile size is default, this won't cause the error. Do not use the standard or full tile size with tile restore, for some images the number is not divisible correctly.
@@controlaltai cool😀, but cause another error'Error occurred when executing ColorMatch:
stack expects a non-empty TensorList'🙏
Do one thing close and restart command prompt and run tiles fresh. Make sure the color match nodes source is correct. I never encountered that color error. Use the default one. Let me know.
@@controlaltai i tried several times, it occurs ocasionally, dont know why
Well only way to know for sure is to check the workflow. And the image. Sendnit to me via email will have a look mail @ controlaltai. com without spaces
I use a tiles node, but the generated image is still in tiles and will not be merged into a single image. What is the reason for this? Is the size of the segmented image appropriate?
Has the process completed at the end, what is your tile size and stride value, along with vram and system ram?
@@controlaltai 4090 graphics card, 64G memory,
I uploaded a workflow video on my homepage, th-cam.com/video/ApgTbjfFsag/w-d-xo.html Can you take a look and help me solve the problem?
I have tried sizes over 1280 * 1280, Supir cannot repair images. If it is an image above 1280, how should we handle it? thanks
The video does not show the last part. Do one thing send me your workflow, because obviously there is something wrong, I need to check your workflow on my system. Images above 1280to 1280 should not be a problem., if you have 24gb vram you can upscale to 2k from that resolution. mail @ controlaltai . com (without spaces).
This was a lot like this mathclass meme, starts out simple, get distracted for a second and suddely the whole board is full, luckily this is a video, pause rewind and replay. Phew that was a lot. What if your old images are discolored or faded? Like an old polaroid that turned very red over the years
For old images discoloration it's a bit more complicated. The first thing is to get it to restore to some degree, 1k. Then you have to use another workflow probably the ControlNet re color one.
th-cam.com/video/hTSt0yCY-oE/w-d-xo.htmlsi=bF4qIFUg3K7lnMrk
The simplest trick would be to make the image black and white and then add color again.
Finally use SUPIR again to upscale for 2k or higher.
For cracks and other stuff you can use Fooocus inpainting as shown here...
th-cam.com/video/wEd1wPlCBaQ/w-d-xo.htmlsi=yQJibUveCYe_oaFl
great video! is the workflow available anywhere? :)
Thank you!! The json is made available for paid channel members. But you can create the workflow yourself as well. Nothing is hidden in the video, as in everything is shown on how to build it.
Hello Friend. Thanks for such a detailed video. This is just a masterpiece. But there is an incomprehensible moment. We are going to engage in an increase. But you have a group to reduce the image (downscale). I reviewed the video several times, but no longer understand what the meaning is in this group? Why do we reduce the image? What is the role of this group? What will happen if you remove it?
Hi, please rephrase the exact name of the group as shown in the video. Is it downscale or are you talking about 4x upscale?
@@controlaltai Sorry friend, that's google auto-translate. I'm from Russia and I use a translator. I stand corrected. I'm interested specifically in the group with the image reduction. timecode 34:13
@video-sketches Hi, check this 44:31 on why downscale is there. Basically, you can upscale to add details downscale then upscale again, to get desired results.
Good afternoon, how can I find json?the link does not work, the white screen please help!it is very necessary(((
Hello, What link doesn't work? JSON is for channel members. And what does white screen help mean. Elaborate
@@controlaltaiI joined a TH-cam community, but nothing happens!!!
@@controlaltaia blank screen, dots and a cross...empty...
Well I cannot help with that. Check with TH-cam support if memberships is supported in your country. As far as I can see you are not a member, maybe a subscriber. Contact TH-cam support for further help.
I don't know what I did wrong, but I get very poor results. Nearly imperceivable difference even after upscale, copy, paste and upscale from 512x512 all the way up to 9kx9k using all of your suggestions. I tried on different images also with very little change. I noticed that there were some parts edited out of the video where things were changed suddenly without explanation.
and yes, I did back up and slow the video down to try to understand better!
This depends on your image, the sudden cut parts are to streamline the video, nothing regarding the workflow is actually cut. When you are using a 512 image, is the image clear or in poor not recognizable quality, that would determine the upscale factor. If the image in clear, try upscale from 512 to 1500 custom. Check, then proceed.
Imperceivable difference comes with very clear images using 1x upscale. Unless I see your image I cannot tell you what is going wrong.
You might want to send the image to me via email and I will have a go with it, reply back with the results. (mail @ controlaltai . com) without spaces.
I sent you 2 of my test images. 1 lowres photo and 1 512x512 AI generated @@controlaltai
Yup received and replied via email.
@@controlaltai thanks again 😊👍
There is no Manager button on my ComfyUI, please help.
You have to install the manager manually
@@controlaltai how to install it?
Supir video 😂
Well, it didn't work for me. Some images got sharp but most of them got noisy.
What's the upscale factor? For good enough images (very clear) 2x upscale will work, 1x spoils it, 1.5x little better....
Hey, how can i get json file? Thx!
Hi, Check the members post in community. You should have access to it.
@@controlaltai Got it Gaurav Seth! Thanks for your work!
How do you fix a damaged photo
Thats complicated to explain in a comment. I do explain it in the video. If you have any specific questions, feel free to ask.
@@controlaltai i would absolutely love if you could make a tutorial of how to remove bends scratches and such, if possible 🤩
A tutorial is not warranted for that, as it’s dependant on the image. But basically using fooocus in paint technique, masking, cssr or SUPIR bend and scratches can be removed. The workflows depends on the image, hence the tutorial won’t matter, as what will work for a single image won’t work for any other.
@@controlaltai okay i saw it a while ago in a tutorial in stable diffusion automatic1111 tho.. But They did use inpainted and it actually worked for a couple of pictures.. Maybe i should see if i can find that again and see if i can replicate in comfyui..
@@supernielsen1223 check out the refinement method in the yolo world in paint out paint workflow. I don’t have a cssr workflow but SUPIR or cssr both should work for restoring.
why not just autocrop the image with a-amod64?
I don’t know what that is, but am learning new things every day. If you could elaborate…
@@controlaltai Sorry, I guess % is more common for Modulo. So the function above should be "a-(a%64)" where a is your input resolution, which allows you to get the nearest resolution divisible by 64 (or 32 etc. as needed).
To elaborate, assuming an image with a width of 550px, the function above would be 550-(550%64) = 550 - (38) = 512.
Regardless, thanks for the elaborate Tutorial.
@@thevoid6756 Thanks for the explanation. Yup I am aware of this.
Resolved
There is no image to seed node.
@@controlaltai my bad, I've edited the comment above!
how to download json? i am the member
JSon is only made available for paid channel members who monetarily support the channel. You can, however, just watch the entire tutorial and create the workflow. Everything is shown in the video. We make it a point with every tutorial that the entire workflow can be created for free and not hide the creation process behind the paywall.
Error 400 - Invalid API :(
I refreshed it and it worked
How to use "instruct pix 2 pix" & "SDXS" in comfyui?
Tutorial for Pix2Pix will be out next week....:)
those images to start with of the policeman were awful. If you're going to grab attention you need to start with something good. I doubt that policeman actually looked anything like that final image.
The thing is... People request these kinda things.. Often another family member or such will tell you how close it is.. It could be way off it could be pretty close... No way to tell really..
@@supernielsen1223 That's exactly why it was a terrible showcase. I am interested in a process to restore some detail, not to completely reimagine a photo. After seeing that horrible example at the start, I stopped watching this video. Waste of time.
@@supernielsen1223 It's very easy to tell with the policeman, lol. The original image is of a dorky, bewildered English-looking guy. The after image is of a less bewildered, somewhat middle-eastern man. It's not just hallucinating new detail; it actually completely changed the obvious remaining detail from the original image.
“Restoration” - means they just do img2img, and final image has nothing common with old image. But who cares on youtube, where viewers have -10 IQ level.
Please don't call people on TH-cam with 10 IQ level. Most of the viewers are smart business professionals. On the technical level I don't want to get to an argument cause such a statement would not been made in the first place. Technically every pixel has data and if you read the research paper you would know how the model is trained and what actually happens. So yeah details are added to the police officer, if you notice the hat and everything else the whole image is indeed restored from broken pixel. It did not put a party hat or a different hat design.
Restoration means taken 5% pixels even 5% and building on that. If the image is that broken even a human won't be able to restore from that level of data.
I never comment on opinions however Channel viewers should not be insulted for no reason.
Thank you for your opinion and offcourse your are entitled to your views regarding the AI tech. It's a free world.
My mans just said that about watching youtube... while watching youtube 😂 bro what does that make your iq? 5??
Please don't use word "restore" when you generate new data from nothing.
Grandpa is now an AI model.
jupp, its a replace generator with a guide.
Am I correct in understanding that this workflow does not use the juggernaut model? I saw different tutorials and they used juggernaut sdxl model everywhere
Yes, it uses RealVis_XL version 4 (baked vae), normal one.
Specs:
4070ti 12gb
32gb ram
i9
I got this error when running it
Error occurred when executing SUPIR_first_stage:
Allocation on device 0 would exceed allowed memory. (out of memory)
Currently allocated : 2.47 GiB
Requested : 4.39 GiB
Device limit : 11.99 GiB
Free (according to CUDA): 0 bytes
PyTorch limit (set by user-supplied memory fraction)
: 17179869184.00 GiB
Switch to default tile, use custom resolution, don't upscale 2x. Check original image size. If higher than 1024 then downscale to 1024. One of the above issues, low vram/system ram