SDXL LORA STYLE Training! Get THE PERFECT RESULTS!
ฝัง
- เผยแพร่เมื่อ 22 ก.ค. 2024
- In this video, I'll show you how to train LORA STYLE SDXL 1.0 using YOUR OWN IMAGES! I spend hundreds of hours testing, experimenting, and hundreds of dollars in cloud computing training to bring you the ultimate LORA STYLE training guide for complete beginners and experts alike for Stable Diffusion. SDXL is incredibly easy to train as long as you know what you are doing with the right training parameters. After this video, you’ll have the required knowledge to train any style you want with the Kohya GUI tool.
What do you think of SDXL Style LORA? Let me know in the comments!
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
SOCIAL MEDIA LINKS!
✨ Support my work on Patreon: / aitrepreneur
⚔️ Join the Discord server: bit.ly/aitdiscord
🧠 My Second Channel THE MAKER LAIR: bit.ly/themakerlair
📧 Business Contact: theaitrepreneur@gmail.com
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
✨ PATREON LINK: / aitrepreneur
Previous Lora video: • ULTIMATE SDXL LORA Tra...
Kohya ss GUI: github.com/bmaltais/kohya_ss
SDXL: huggingface.co/stabilityai/st...
Jup1t3R!
scale_parameter=False relative_step=False warmup_init=False
Runpod: bit.ly/runpodAi
Runpod Template: runpod.io/gsc?template=ya6013...
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
►► My PC & Favorite Gear:
i9-12900K: amzn.to/3L03tLG
RTX 3090 Gigabyte Vision OC : amzn.to/40ANaue
SAMSUNG 980 PRO SSD 2TB PCIe NVMe: amzn.to/3oBR0WO
Kingston FURY Beast 64GB 3200MHz DDR4 : amzn.to/3osdZ6z
iCUE 4000X - White: amzn.to/40y9BAk
ASRock Z690 DDR4 : amzn.to/3Amcxph
Corsair RM850 - White : amzn.to/3NbXlm2
Corsair iCUE SP120 : amzn.to/43WR9nW
Noctua NH-D15 chromax.Black : amzn.to/3H7qQSa
EDUP PCIe WiFi 6E Card Bluetooth : amzn.to/40t5Lsk
Recording Gear:
Rode PodMic : amzn.to/43ZvYlm
Rode AI-1 USB Audio Interface : amzn.to/3N6ybFk
Rode WS2 Microphone Pop Filter : amzn.to/3oIo9Qw
Elgato Wave Mic Arm : amzn.to/3LosH7D
Stagg XLR Cable - Black - 6M : amzn.to/3L5Fuue
FetHead Microphone Preamp : amzn.to/41TWQ4o
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
Special thanks to Royal Emperor:
- TNSEE
-Gluthoric
Thank you so much for your support on Patreon! You are truly a glory to behold! Your generosity is immense, and it means the world to me. Thank you for helping me keep the lights on and the content flowing. Thank you very much!
#stablediffusion #sdxl #lora #aitools #texttoimage #ai
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
WATCH MY MOST POPULAR VIDEOS:
RECOMMENDED WATCHING - All LLM & ChatGPT Video:
►► • CHATGPT
RECOMMENDED WATCHING - My "Tutorial" Playlist:
►► bit.ly/TuTPlaylist
Disclosure: Bear in mind that some of the links in this post are affiliate links and if you go through them to make a purchase I will earn a commission. Keep in mind that I link these companies and their products because of their quality and not because of the commission I receive from your purchases. The decision is yours, and whether or not you decide to buy something is completely up to you.
Timestamps:
00:00:00 Train SDXL with LORA
00:00:57 What is LORA?
00:01:11 Koya SS GUI
00:01:53 Image Dataset preparation
00:04:45 Find good video game images on Steam
00:05:32 Upscale images sharper with Stable Diffusion
00:07:50 Make sure images are not too big for LORA Training
00:08:31 SDXL LORA Training Folder Creation
00:09:45 Instance Prompt
00:10:30 Find the right style for LORA Training
00:12:48 Class Prompt
00:13:00 LORA Repeats Training Steps
00:14:21 Regularization images
00:17:39 Training Data
00:18:18 LORA Image Captioning
00:22:52 How to replace a word in multiple text files on Windows
00:26:00 The right SDXL STYLE LORA Training Parameters
00:26:28 LORA Type Standard vs LoCON or LoHA LyCORIS
00:26:49 Train Batch size
00:27:44 Number of Epoch Training
00:28:54 BF16 & FP16 & Cache
00:29:27 The RIGHT Learning Rate for SDXL STYLE LORA
00:30:11 SDXL Image Resolution & Buckets
00:30:55 LORA Network Rank & Network alpha
00:31:40 Gradient Accumulate Steps LORA
00:32:21 Noise Offset LORA
00:33:14 LORA Training & VRAM Usage
00:33:38 GPU Solutions & RunPod
00:34:02 Runpod Setup for LORA Training
00:34:41 Runpod Template LORA Update
00:36:43 How to select the right LORA Model?
00:38:43 Chose the final Model
00:41:37 Conclusion on LORA STYLE Training
HELLO HUMANS! Thank you for watching & do NOT forget to LIKE and SUBSCRIBE For More Ai Updates. Thx
👋
How about reviewing oobabooga's new TTS extension?
Thank you very much! You have the only video on TH-cam about training LoRa for style, not for character!!! 👍🙏😻
Pateron supporter and was one of your followers asking for this for a few months. I finally had time this weekend to explore this, and just wanted to say a huge thanks! I finally have a lora that doesn't look so nasty and overtrained. I had improved results already using your character training from awhile back, but this really nailed it in, and the tip about using a famous artist helped. I actually found using a famous art movement (abstract expressionism) helped it to understand my work. This is massive, thanks so much for all the videos you have posted this year, for stable diffusion that have been game changing for me.
thanks for the great content. you now are the first creator I support on Patreon.
thanks a lot again for this so great tutorial and the patreon you made in conjuction ! Great job Dude !
Finally, a video on style LoRA! Very thankful for it. My VRAM isn't so good that I can train/generate stuff on SDXL, but this guide is so general that I can rely on it even for stuff based on SD1.5. I suppose the only difference would be that I'll still be stuck on 512 x 512 resolution and some other minor differences. Thank you very much!
Finally you back ❤
I am ;)
replaying in background to increase stats :) good to see you posting video once again. Hopefuly you were able to catch up few things when you were waiting for training results :)
Thanks yeah a little bit ;)
Hey! Thanks for all of your knowledge and support always. I have a quick question. What app do you use for your notes (the app where you save the sample photos of your epochs to compare and choose the best one ?
Thank you for another excellent video. One question I still have is how can I change the trigger word(s)? Do they change when you replace the subject in the .txt files? Or is it hard coupled to the instance prompt? In your previous video (the character one) you had to prompt "sascha luss" to create the Milly character, which my brain doesn't really like 🙂
Thanks for the update
😉
what is that tool he's using where he's searching for what prompts sdxl understands? the "something" studio by stability ai
Is there anyway to enque trainings in khoya? So that you could set a few up and have them train overnight.
Useful information. Thank you.
Glad it was helpful!
When you run a test on all of the models to choose which you like, have you run that a few times to see if your best model stayed the best?
Please do another simplified video on how to locally install vicuna, previous videos seem to have become outdated.
What if I want to train a LoRA to generate comic pages with panels and all? What type of images should I use to train it?
Thought you were dead. Welcome back.
Still alive and kicking! 😁
Should I remove character tags when pruning/tagging? I'm training on an anime art style and not sure if I should include tags like "Erza Scarlet" for example.
Hey, great video as always!
As a video suggestion, could you make a guide to creating abstract concepts with Stable Diffusion?
I'm trying to create a Solarpunk model that could work on different styles: photorealism, illustration, anime, oil painting etc.
And if anyone has any suggestion on how to do it I'm interested in your guy's thoughts!
Yeah I plan on making more training videos ;)
@@Aitrepreneur Also make a video about how to fix hands.
What if I collect real photos of only 1 person, then train it to create real styles for animated photos?
Is this the opposite of your way of doing things? Can it work?
Does clothing fall under the category of style?
This is great but I'd be interested in training some thing that SDXL doesn't already know about. Would I have to create a model for this?
Can you make one more on character training? The old one is kinda outdated, I followed your guide and my results were pretty good.
I did one a few weeks ago: th-cam.com/video/N_zhQSx2Q3c/w-d-xo.html
"No effort required whatsoever..."
That pretty much sums it all up.
the runpod template doesnt work agian pls help im a patreaon supporter
Hi. thank you for the tutorial. I get this message at the beggining of the training. The following values were not passed to `accelerate launch` and had defaults used instead:
`--num_processes` was set to a value of `1`
`--num_machines` was set to a value of `1`
`--mixed_precision` was set to a value of `'no'`
`--dynamo_backend` was set to a value of `'no'`
To avoid this warning pass in values for each of the problematic parameters or run `accelerate config`.
What should the regularization images contain for style training?
Also been meaning to ask don't you find when training a character using another celebrity's name that the model is trained on that the model's image generation still picks up a lot of mannerisms and locations that the original celebrity was trained with despite the trained character Lora being used? Like hypothetically if you were a doppelganger for Ben Stiller and lived in the same universe as Zoolander and you decided to train your own face but instead of using Bstiller as the name of the Lora you use Derrik Zoolander because it's a great match and your AI model would certainly know who he was even after training it would seem to be that it would continually show you at modeling runways and dressed fashionably, and doing his poses even though the face would look less dramatic than Zoolander and more like typical Ben Stiller (Edited to add the Zoolander thing)
Is it available for Radeon yet?
Whwre do you see that all the styls
This is good but What I would really like to see, esp for patreon subscribers. is a tutorial on how to in-paint, and make hyper realistic stuff. or maybe even better is how to use comfy UI and all of its little addons and stuff.
Could you send the link to the Text Replacer app?
Thank you for all of your hard work. Its mind simply amazing. The time to do this AND make this video must have been crazy tough!
Ahoj, dík za návod ale mohl bys prosím zapnout automatické titulky jaksi nejsem zdělán v tvém jazyce. Zdravý odběratel z České Republiky.
Stable Diffusion content = interesting
I used your settings right (SDXL-style.json) with 40 images and 20 reps, I think it overfitted very early on, would you lower the learning rates or change something else?
Yes you can change the learning to rate to something like 0.0009 instead
On the character training tutorial we used 10 epochs and you said it would take about 2 hours. I have a 4090 and it took about 50 minutes. Is this a difference in hardware speed or could it have been incorrect settings?
I have 1660ti laptop, should I quit before I even try installing AI ?
10:40 - What's that purple URL website? is it a local instalation of SD?
right? what is that?
Anyone know how can i do it in google colab please ?!
in Runpod, when i trying to connect to ports, there is message "The port is not up yet", or a "Bad gateway"
in containers log there is - Syncing Kohya_ss to workspace, please wait... but it lasts forever
I cant get SDXL to work at all. In a1111 it just gives errors and can't load the checkpoint. Also didn't work at all with Easy Diff which I much prefer to a1111, where it just gives a weird collage of colors in the generated images. So far all the times I've posted on different videos asking for help, no one has ever replied. I'm close to giving up on all this.
10:42 what is this website??
Upload into GPT and ask which generated image is best
Is there a minimum VRam requiring for this?
18GB
Question though - if SD already knew what walking dead style was, why bother making another Style LORA for it?
can you talk about AnimateDiff extension
Will you make a tutorial on how to make a Lora for a particular pose / situation ?
Btw, do you know how they train "photorealism loras" ? Or "details lora" ? I'm wondering if for each image in the training dataset they put a value for details or photorealism in the captions.
Yes more training videos coming ;)
I want to learn and teach people how to train literally ANYTHING!
@@Aitrepreneur Thanks !
All of your trainings seem to just be enhancing things that already exist in the model, what if what you want to train, say a concept (like a skateboard trick performed by a range of people), is not in the base model? Would we be better off training a model refinement instead of a LoRA? How does that differ from LoRA training?
Everything that is supposed to be a variable has to be named. I.e. you want to change eye colours? Describe the eye colours!
But you can also train a bias on purpose. For example you don't like beards but your base model has a lot of beards and you need to use negative prompt to keep the guys shaved? Train the model with shaved guys and don't mention that they are shaved, so the model will start to default to beardless guys more.
That way you just need to know what you want to be optional later and base your concept around that. If you are not persistent with these things the model gets confused and produces rubbish.
This is great info...but I realized that repeats is nothing more than a way to describe how often you would be theoretically save a checkpoint/LORA because the minimum is one epoch. Also I never experienced that upping your batch size reduces training time. It reduces steps, but iteration time increases as well...so its just a trade off and the training total time stays the same!
For Gradient Accumulation to not get weird, just make sure that Batch Size times Gradient Accumulation is not higher that your total images count! 🙂
Yes, the author does not seem to understand the point of repeats. It's there so you can balance your dataset. You have images in folders with the repeat number: woman20, dog10, sword15. You can use it to add more concepts to same LORA and/or balance the importance of the dataset. Want more dog than sword? Then set dog repeats higher.
@@digidope thanks for the info, I did not know that you can create multiple subfolders with different repeats and different subjects... I always thought you'd need another training pass to do so! I will do some testing!! 😁
@@swannschilling474 Read the documentation. DOCS folder has series of README documents. There is also examples like: image_folder_structure
For some reason I can only train with my RTX 4080 if I use the lowvram preset... Is that right?
How much Vram is that 16Gb? You should be able to use a higher resolution at least
@@Aitrepreneur yes, 16gb... It fails because of so little amount of vram...
Idk if some of the settings can help without changing it from 1024px
I mean the first setting I would change is either resolution or network rank, that's the main 2 parameters that affect a lot the vram usage (with batch size)
TBH compressing pngs sounds like a terrible idea. In my graphic design workflow if there's any gradient on the picture, compressed pngs with these online tools won't pass unless it's an unimportant background asset.
Can you please make tutorials on anime characters training and anime style? I think such a video would be very helpful for the community 💟
It's the same thing as a normal style, I trained the jojo's bizarre adventure anime style using the same technique
@@Aitrepreneur Thank you very much for the answer. I will also try to create similar styles
Any work around for running SDXL on RTX 2070 Super 8gb?😢
Use --medvram arguments and --xformers
got it thanks!@@Aitrepreneur
Thank you for File Rename Basic. Windows rename is HORRIBLE. I used to switch over to my mac just to do proper renaming. No I can do it in windows, no problem... =]
The rambling is more than I can take.
hi overlord
Hey 👋
so textual inversion embeddings lost the game?
I mean kinda yeah, no real reason to use them anymore when you have LORA, they work the same way already
@@Aitrepreneur It's funny as they have always worked the same. For some reason nobody, including you didn't train LORA's first. But isn't the princible still different, so result should also be different? Negative embedings are still TI's not LORAs.
What happened for the reupload?
youtube demonitized and buried my last video so reup here again, let's hope it works this time...
I followed the tutorial and started training with 10 images and it takes whopping 12hours for 15 epochs with RTX3090 😮 I have all the same settings and 20 repetitions.
I do 1 epoch, usually works well enough for what I need.
@@joelchamp1949 I did the whole 15 epochs for the 12 hours and when testing the results, the epochs 14 and 15 are the best. The style starts appear around epoch 8 but not as that strong. It depends on dataset and size on dataset. I had only 10 images, but if i have had 200 then less would have been enough.
@@digidope yeah cool. If I'm not mistaken doing 1 epoch would be the equivalent of your 15th epoch, some might say it's overtrained but I've been using it for art style so I want the maximum effect to carry over.
reupload?
Yes... youtube being youtube again demonitized and buried my last video...
allah hakbar ragazzuoauoli nuovo stile da rappresentare come intro in hdio
How to change face on video ..
Thanks SO much for the content.
Why do you shake the video? like at :53
It's kinda annoying of people trying to read. If you don't want people to read, then I would suggest not adding an image that people would want to read.
Second comment 😅
A bit late I'm afraid xD
Has any youtuber made a specific video about training objects? I need some advice, tips, and tricks for training objects. I am looking to train a specific style of cake. Can anyone point me in the right direction? I have made about 15 Loras about this specific cake, and none of them are perfect. Thank you for any help you can provide.
I love your videos, but wtf is up with the constant renaming of your videos?
Wtf
Probably trying different things to get better CTR
$$$
Rah-bee
So basically... I went to the video in the first place to learn how to properly caption training images after being intimidated by the thought of precisely describing everything for a style... only to be told to suck it up. UGH.
0:57 from left Volodymyr Zelenskyy - world leader😎❤
not a fan of the shaky avatar
Why are you training a style that SDXL already knows?
Nice tutorial as always but in 40 minutes there's way too long about character training, just for the sake of comparing. Maybe focus more on the essential that's only related to what people come in for : style. ;) Cheers.
Can you teach us how to train a Ai to only make basic pose drawings?
What the hell is up with these giant month long periods between videos now?