Training (Fine-Tuning) Your Own Stable Diffusion Model Using Colab
ฝัง
- เผยแพร่เมื่อ 6 มิ.ย. 2024
- IMPORTANT NOTE: YOU CANNOT RUN THIS CODE WITHOUT UPGRADING TO COLAB PRO
Training, or Fine-Tuning, your Stable Diffusion model cannot be easier with DreamBooth! Using Google Colab, I will walk you through generating images with specific subjects, objects, or styles!
📣📣📣I have just opened a Discord page to discuss SD and AI Art - common issues and news - join using the link: / discord
👀 Struggling to run Stable Diffusion locally? You can try DiffusionHub!
🤑Get 20% MORE DiffusionHub CREDIT using the link below or using the promo code LAURA20!
Link: diffusionhub.io?fpr=laura17
🤙🏻 Follow me on Medium to get my Newsletter:
- Get UNLIMITED access to all articles: / membership
- Laura: / lauracarnevali
- Intelligent Art: / intelligent
📰 Article:
/ training-fine-tuning-y...
📌 Links:
Google Colab Notebook: colab.research.google.com/git...
Step by Step guide by Excalibro1: github.com/Excalibro1/fast-st...
Ruiz et al., DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation: dreambooth.github.io
Patil et al., Training Stable Diffusion with Dreambooth: wandb.ai/psuraj/dreambooth/re...
Hugging Face: huggingface.co
Crop with Birme: www.birme.net
00:43 What is Fine-Tuning?
00:53 What is DreamBooth?
03:09 Open the Colab Notebook
03:33 Connect Colab to Google Drive
03:51 Install Dependencies
04:05 Model Download/Load
05:32 Create a New Session
06:29 Load an Existing Session
07:10 Upload Instance Images
11:32 Captions
12:53 Concept Images (Heavy Regularization)
14:05 Training the Model
19:36 Test the Trained Model
20:05 Create Images w/Stable Diffusion
25:30 Upload Model to Hugging Face
#aiart #stablediffusion #generativeart #googlecolab #colab #training #dreambooth #modeltrains #stablediffusionai #stablediffusiontutorial
Note that the name of the instance images needs to represent the unique identifier. Need to call that name when writing the prompt!
Very cool and clear. Thanks again Laura!
Super helpful, thanks!
Great video! Great job!
this is what i was looking for! thank u so much!
Great job thanks! Could you explain hot to train Lora, in order to use combined with other models
Thanks for the tutorial!!!
Could you make a tutorial about textual inversion embeddings, please?
Welcome to Thailand :) im one of your fans , hope you really enjoy your holiday.
Thanks! I love Thailand! 🥰
Excellent video and very well explained.
Great video, thank you very much!
Thank you Laura! Very helpful video!! Do you know how we can create a Lora using Colab also? Thanks again!
BTW, did you make a part 2?
Hi, yes, you can, although I had some running issues last week. I'll try to make a video if it works again.
@@LaCarnevali Fantastic! Thank you very much!
Hello Laura, really enjoyed the video! I tried following what you did step by step however when I launched the "Test the trained model" cell, I got a traceback error message which prevents me from testing the model. Do you have any advice or is there an updated version of the notebook I should download? (I got colab pro)
Hi, has someone the link with the second part? I want to know how to continue with the training if Colab doesn´t allow me to continue using the GPU.
Thanks for the video Laura!!
hi - do you have a tutorial on how to create the folder "LaCarnevali/Laura-test"? I have the batch of images, i have stable diffusion running but im trying to train a LoRA based on my own images. Im doing all this on a mac. Also - LOVE your videos - keep up the good work!
Hi, you need to create that inside your Google Drive...just right click> new folder
Forgot how to do it and came back, still an awesome video :D thanks again
hey how did you do it?
@@Minahil-zw6wtsadly it doesn't work anymore because they havent updated it.
@BlakeSevann2 do you have any other suggestions, am looking for a pre trained model for making an app where user can provide his images and he can get different ai generated images , like in remini app if you have used it
@@Minahil-zw6wt unfortunately i don't :(
Thank you for this video, it can be frustrating when you don't really understand what's going on and are just following tutorials
Thank you,, so easy
Ciao Laura, complimenti davvero per il video e soprattutto grazie! Avrei una domanda: volendo effettuare il training su di un volto usango come modello Realistic Vision v2.0, il processo sarebbe lo stesso e basterebbe poi caricare su Drive il ckpt del modello e selezionarlo su SD? Oppure bisognerebbe fare altro? Grazie di nuovo
Ciao! Si esattamente lo stesso :)
Hey Laura, awesome vid. I have a question though. In the training part, when I run the code, the training starts and after a minute, when it gets to the "training the Unet" part it stops, the code at the bottom says something like "something went wrong" although there is a green check mark next to the play button you click to run the code on the colab notebook. Any ideas?
same issue here
The "Instance Images" tab is missing from the notebook in the link to the video. How do I find that or code for it? I've looked on other websites but can't find it. Some people train their images in another website like Astria and not the google colab, but I want to keep everything in the google colab notebook. Thank you !!! I've only been able to get this far bc of your videos, this seems to be the last missing piece.
I just clicked on the link - the instance tab is still there. Make sure you are opening the correct notebook
colab.research.google.com/github/TheLastBen/fast-stable-diffusion/blob/main/fast-DreamBooth.ipynb
Hi Laura, Great tutorial probably the best I've seen thus far for training LoRA. I did notice at marker 12:39 Instance images is not loading on my page or has been removed. I'm assuming the latter. Do you know why that was removed and what replaced it? Thanks again!!!
Hi, you can use the 'deprecated' folder for that :)
Hello! Thanks for great tutorial! I have a problem on the step Upload Model to Hugging Face. Google colab deletes the runtime and displays a message that it does not support this action (apparently, loading the model on hugging face). Could anyone help me fix this error, please? 🙏
Thank you for the brilliant tutorial, very clear and accessible. I am new to the process and was just wondering if I make my own set of concept images to inform my instance images - will the end result be entirely created from my own image data inputted? Or will Stable Diffusion's vast image dataset also inform the final image?
Thanks, it will use the dataset used to trained the model (it depends which model you used, i.e., civitai, v1.5). I understand you are new to it, but just a hint - you might use Lora models in addition to your model.
@@LaCarnevali great! Thanks for the advice
@@LaCarnevali Oh... 😮how do we do that?
Thanks for the tutorial, but now the localtunnel asks for IP. Where I can find the IP address required by local tunnel?
Hi Laura ❤🙏thank you for this tutorial, I have been looking for something like this for a while now!!!
I have a few questions if you could please clear up for me.
1- When u crop and RENAME your INSTANCE images do they have to be the same name as UR SESSION?
2- If I do not want to train BOTH 1- face /a character AND an AESTHETIC (my images have a pinkish coloring) do I need to upload both INSTANCE and CONCEPT images?
I am not sure I understand the difference between the two and whether I can do both in ONE TRAINING SESSION.
3- Any tutorials you have in mind that show how to do CONCEPT training?
🙏❤
1) The name of the instance images needs to represent the unique identifier. Need to call that name when writing the prompt!
2) The INSTANCE images are those used to train the model. They need to be almost perfect, I would say. They need to vary though, show the face/character from different prospective, different dresses, etc. If you have only one image with a pinkish color, that's fine. However, if you have all of them with that pinkish color, which you don't want, I would suggest to change the pictures! XD
CONCEPT images are used to regularize the model. If you are training your face, you don't need to use them. If you need to train the model on a specific car (BMW), for example, it is preferable to upload lots of different cars in different environments, shapes, etc, so that the model can generate more variety of your car in different places, positions, ... otherwise, the model will focus too much on the INSTANCE images and will reproduce them as they are. Also, the CONCEPT image name doesn't matter.
I'll add this to my to-do list :)
Hope it s clear. XD
I've tried using external captions and when training it always seems to truncate my captions to the first 50 - 55 characters instead of using the whole caption. Have you come across anything similar?
Hi Laura, thanks for all the explanation. I have a doubt about if I want to create a Model from scratch, how do you do that?. I was looking for it in your videos but I didn´t find it. and after that, make the fine tuning step. Grazie
Hi, training from scratch requires many images (thousands) and time. I haven't made a video, but might think about making one
Hey Laura, thanks for this really cool content and tutorials, really helping me in my learning Journey.
Im trying this one out but sadly when i try to import drive im getting the error message "
ValueError: Mountpoint must not already contain files" I already installed SD onto google drive, do I have to create a different folder to link to? Any help would be amazing. Grazie Mille x
When you run the mounting code, make sure it is mounting to the right directory.
from google.colab import drive
drive.mount('/content/drive/')
You might want to change the directory /content/drive/ to the actual directory (not sure where you have your SD folder)
Very nice and clear, Anything regarding data Privacy ?
Hi Laura, on the "Training" part of the Google Collab I get the error "No model found, use the "Model Download" cell to download a model.". When looking at the python code it's getting stuck in the while not Resume_Training and MODEL_NAME=="": loop. Any suggestions? Not sure what I did wrong.
try cancel the folder from your drive and re-run the notebook
@@LaCarnevali I tried that many times and it still wouldn’t work. Are there hidden files that are stored in the drive?
There’re some, but should be that. Do you have Colab pro?
@@LaCarnevali No, I only have the free version. Is pro required?
@@mrkeyzzz Yes, the free version was banned by Google last month, unfortunately. So you can now run it only on the pro version
Hi Laura! Couldn't find the video for reloading the trained model without going through all the steps again. Where can I find more information about it?
Hi,
You can just skip the training by putting 0 to all steps or running only the section where you need to upload the model.
Otherwise, if you are using colab, you can use this notebook (still need to run all of the steps but it s less complicated) colab.research.google.com/github/TheLastBen/fast-stable-diffusion/blob/main/fast_stable_diffusion_AUTOMATIC1111.ipynb
thanks!!
Hi, at the end of the notebook when we try to open webui, it ask for a tunnel password?
Thanks for the tutorial, but now the localtunnel asks for IP, and i cant seem to find it, is there an alternative for this? Please help
Hey Laura, thank you so much for this amazing Video! I have a question though: Whenever I reach the section "test the train model" I won't get a URL for my Model, but receive this message: " ModuleNotFoundError: No module named 'cchardet' " Why is that and how can I fix it? Thank you so much in advance!
Same here, did you manage to solve this?
probably need to upgrade to Colab Pro
Did you solve?@@AntanasStasiunas
@@AntanasStasiunas I think this would require the paid version of collab i,e, Collab PRo
no you just need to run !pip install cchardet
Please can we get one on textual inversion for Google collab
where can I find the fast dreamBooth google drive folder?
if i have many images so how should i add caption to every image
Hi Laura! Thanks for sharing all of these videos. Do you think is possible to re train (fine tune) a model to get more "diverse" results? For Example, if I generate "pictures of CEOs" I'm gonna get lots of white man pictures, if I re train with 100 pictures of CEO of different genders and races, do you think is possible to get more diverse results? And also, do you think I can see a difference when using DreamBooth or LoRA? Thanks!
Hi! Yes, you can do that. In my opinion, DreamBooth is going to be more precise as you are training the full neural network model, rather than only one part (LoRA).
Hello Laura,
First of all, thank you so much for these incredible guides.
I tried running your Colab code from this tutorial and successfully completed the training 2 times. However, when it comes to testing, I am not able to launch the SD. Also, every time I run this Colab, I am not able to launch my previous Colab SD, which I did from your other tutorial. Am I supposed to install them in seperate drives???
Hi, you can try this notebook to test it:
colab.research.google.com/github/TheLastBen/fast-stable-diffusion/blob/main/fast_stable_diffusion_AUTOMATIC1111.ipynb
@@LaCarnevali I am already running this very AUTOMATIC notebook after watching your other video. But after running the training session from the Dreambooth notebook in this current training tutorial, I am getting error, and unable to launch the AUTOMATIC notebook. However, if wipe the installation, and perform a new clean installation, I can run the AUTOMATIC notebook just fine. Both the notebooks are from your tutorial.
Steps I went trhough:
1). Ran AUTOMATIC notebook and Installed SDv1.5 from the link you have provided. SD works fine for various sessions(Shutdown -Restart many times.)
2). Ran Dreambooth training notebook and completed style training 1 time. Testing unable to launch SD. Getting errors.->So shut down session.
3). Ran AUTOMATIC notebook, but unable to launch SD. Getting errors.-> So, shut down the session.
4). Wiped the sd drive in GDrive.
5). Repeated the above steps from step 1 to 4 for a second time., but for for face training. Still not able to launch AUTOMATIC webui. Getting errors about missing files, and errors in webui.py and more.
So, I had to wipe the sd drive and reintall the whole SD clean using the AUTOMATIC notebook. I hope I can run both Dreambooth training and the AUTOMATIC webui from the saem Colab pro account. Otherwise, I'll have to pay for another Colab pro account. Help please.
Also, if there is some way, or platform, where I can get personalised assistance/guidance from you, for Generative AI training, it will be helpful. I really connect with your teaching style.
-Thanks in advance.
please part two, a way so that the image does not display hand or body errors🙏🙏
Hi Laura. I really like your videos. It could be possible to train a model to complete objects. Imagine I have objects hide by others and I want to complete those incomplete. I'm thinking in an actual application. Imagine you have fruits (i.e., apples on a tree) hidden by the leaves. I'd like to train model able to complete those fruits which are completed.
What's the best course of action when the images in my dataset are all the same dimensions but it's not a square (690*760)?
For this type of training you need to resize your images. Use Birme or Photoshop.
Good video, but could you uuse the screen recording of the full screen? It's kind of confusing when it jumped around when the video is cut. And super zoomed in i'm losing overview of where you are on the page.
hello Laura, I have trained my model and I have used it. but now i want to use that trained model again. so I enter session name and it says session loaded, but now I don't know what to do next. please help me. thanks in advance.
You can use this notebook: colab.research.google.com/github/TheLastBen/fast-stable-diffusion/blob/main/fast_stable_diffusion_AUTOMATIC1111.ipynb
Hi laura. How can i train stable diffusion for a specific style with dreambooth? For example i want to train it so that i can convert myself into a marvel character etc.
You can probably do that without training - anyway, it will require another video 🫠
That's awesome! But I am encountering some tensor size issues can anyone here help me how to resolve this issue
Hello Laura I try to follow your lecture, but there is some issue.
I finished training model, and want to go in to the local tunnel url, but The site want me some tunnel creator's Ip address. How can I solve it?
Once you close your notebook, you cannot go back to it. Every time you open a new notebook you need to start again. Also, why do you need to go back to the local tunnel? You should have your output model in your drive :)
Yeah I'm also facing the same issue. How you fixed it? I also passed my public ip there, but still it is not accepting that. please tell me?
@@LaCarnevali But at least first time, we need to test it on local tunnel. But it is asking for IP and we don't know what to put there. I also put the public ip there but it is not accepting that. Please tell how to resolve this.
Hi Laura! Well I guess Google is banning the use of webUI’s from the colab servers. What options do we have now? RunPod?? What’s your thoughts? Thanks! Jason
Not sure if this is actually going to happen, but yes, Runpod can be the option B. Would like to find something free though ... :/
@@LaCarnevali I'm not sure if there's anything out there that's available freely to use. Even with Colab I ended up subscribing, because my "free" ran out.. 😔
thx for sharing, may I ask if that's impossible for Mac user to train a lora using tools like kohya_ss?
You can do that with Mac, it can be slow though (it depends on what mac you are using). Otherwise, you can use kohya in Colab
Hi, just wondering I can no longer see the option to add concept images(regularization) any ideas?
I cannot see them as well - it seems they have updated the notebook
If I have thousand images which will be for model..
Which settings you suggest? :)
At this moment I'm trying 1024,1024, and total is 120950 steps.. (maybe too much?)
The choice of the settings depends also on type of training, image type, image quality, etc.
nice video there, I tried with the safetensor model of dreamshapper, but it did not work. I had the model in my gdrive and provided the link to the model path but could not get pass the model download stage. Any ideas?
Have you upgraded to Colab Pro?
@@LaCarnevali Yes
@@LaCarnevali Hello, yes
Can you show how to save it as a checkpoint directly?
we have 25gb outfit data.can we train on It.same steps.
I don't have the window above "Dependencies".
it can be that the notebook has been updated since the video - do you have requirements?
I still have troubles with how long each and every cell takes to load... I´ve already tried it with a smaller dataset to make it faster...does the training section take half an hour for everyone else? Does it maybe have something to do with that colab keeps informing me that I got disconnected from the GPU? Sorry, i am new to this...
Hi, it can be. When you open the notebook, go into runtime, and change runtime to GPU to make sure you're using it
One thing I’m a bit confused about - what if the model already has existing images that are labeled as “Laura” from the original weights? How does it know that “Laura” should only refer to the images you fine tuned?
It would be better to call the model with something the model doesnt know, like L4vr4. Including numbers within the model name is ideal.
The "Use_localtunnel" page is asking for "Endpoint IP", do you have any idea how to obtain it?
Not sure - I would avoid the local tunnel
regularization part is not in the code anymore
Can you do a tutorial on how to train a Lora or Textual Inversion with Google Colab?
🙌🙌🙌🙌
Hey Laura, thanks for the tutorial I've run into an issue I can't figure out. When testing the trained model I dont get a local link even when I've selected local tunnel. I get this error.. any ideas? TypeError: check_deprecated_parameters() missing 1 required keyword-only argument: 'kwargs'
Have you upgraded to Colab Pro?
@@LaCarnevali yep I have! Do I need to have pre installed stable diffusion on my Mac first?
@@josephivill6533 Nope, you don't need any local installation - try this link to test your model:
colab.research.google.com/github/TheLastBen/fast-stable-diffusion/blob/main/fast_stable_diffusion_AUTOMATIC1111.ipynb
Make sure you mount to Google Drive, and your model is in the right folder inside your drive.
Hi Laura. Is this colab notebook still working? I had it working a few days ago, but now it gives an error when I get to the training cell.
it works, but it's under payment (~10$ / month)
is there a reason why the part for inserting concept images doesnt show up anymore? has something in this notebook changed since one year ago?
yes, it has been updated
Hello, thanks for the very helpful Tutorial. When i want to test the trained Model in Google Colab there is an error: ModuleNotFoundError Traceback (most recent call last)
in ()
6 from subprocess import getoutput
7 from IPython.utils import capture
----> 8 from pyngrok import ngrok, conf
9 import base64
10
ModuleNotFoundError: No module named 'pyngrok'. Can you help me with that?
Click +code on top left and run pip install pyngrok
Hi Laura, while Testing the trained model i cant seem to access the webpage,
Error: endpoint IP is not correct. Please try again or contact whoever gave you this link for the correct public IP.
Are you using Colab pro?
@@LaCarnevali No the free version
@@uroojakmal1726 unfortunately, google has banned the free version of A1111 with Colab, so you can run it only with the pro - you might try ComfyUI (I think it's still free )
@LaCarnevali Thanks a lot dear for your help.
Hi everyone, do you think it's a good idea to fine tunning Stable Diffusion for movie poster generator? I have a dataset with movie poster images and their txt overview.
Hi, I'm trying to do the same. I think that is a good idea btw. What's the prompt that are you using? Are you getting any good results? I would like to talk about it
After training to use tunnel it is asking for tunnel password from where can I get tunnel password?
Is there a way to do this without getting colab pro? Is there a free way?
unless locally, not that I m aware
if it okay, i would like to request about 1 click colab, there so many of people don't know about that and is it hard to us 1 click colab stable diffusion.
Hi! Sure, thank you for the suggestion :) will do that
Hi Laura! Tunnel URL dont work. I clicked "Click to Submit" but it needs Endpoint IP. How can I solve this?
You can also run it without tunnel if that gives you issue. Make sure you have the Colab Pro version, as Google banned the free version last month or so.
@@LaCarnevali How come!!! Very sad. Thanks a lot for your help.
@@LaCarnevali How can i run it without tunnel?
Can we train with sdxl?
yup, there should be an updated kohya file to train SDXL
How can I do this with Dreambooth in Automatic1111?
This requires another video lol
Ciao Laura, non è possibile installare Stable Diffusione per Mac con processori Intel?
Ho provato, ma appena apre stabile diffusion su browser appena chiedi una generazione crasha per via di Python.
Risolvibile secondo te?
Ciao Valerio. Si puoi installarlo. Ti consiglio di seguire questo video:
th-cam.com/video/dIB4GW3iACQ/w-d-xo.html
Unica differenze per Intel riguarda Homebrew. Homebrew viene installato in una directory differente: /usr/local
Quindi, dovrai installare le libraries usando quanto segue:
/usr/local/bin/brew install cmake protobuf rust python@3.10 git wget
Dovrai poi seguire NEXT STEPS sul terminal (dovresti avere qualcosa del genere):
==> Next steps:
- Add Homebrew to your PATH in /Users/$USER/.zprofile:
echo 'eval $(/opt/homebrew/bin/brew shellenv)' >> /Users/$USER/.zprofile
eval $(/opt/homebrew/bin/brew shellenv)
Copy and paste the first line on the same terminal window, then hit enter
echo 'eval $(/opt/homebrew/bin/brew shellenv)' >> /Users/$USER/.zprofile
Then Copy and paste the last line on the same terminal window, then hit enter
eval $(/opt/homebrew/bin/brew shellenv)
@@LaCarnevali proverò oggi. Sei troppo buona. Complimenti per la competenza.
@@LaCarnevali ciao Laura, non ci sono riuscito. :( le mie lacune da programmatore sono evidenti.
@@valeriocalabrese5000 Ciao Valerio, prova a far partire il programma con il seguente:
./webui.sh --no-half
Se questo non funziona, potresti dirmi che errore ti appare sul terminal quando fai click su genera?
@@LaCarnevali ciao Laura, se ti va quando hai 20 minuti liberi. Lo faccio passo passo con te. E ti dico gli errori. Me ne dava diversi.
[HELP] during the Upload to Huggingface cell collab disconnected but I have my session saved in drive, how do I manually upload to Huggingface and use the darn thing. Where is the continuation of this vid?
[PS. I can't find any yt tutorial to help me, if anyone can link a tut,that b awesome]
Hi, sorry I still need to upload the continuation. Regarding your question, you can use the same notebook - use the Create/Load a Session section to link to your saved session, them use the Upload The Trained Model to Hugging Face section.
Otherwise, you can upload anything on HuggingFace just clicking on your avatar on top-right, and choose +New Model/+New Dataset/+New Space
hello laura i am having an issue whenever i start stable diffusion and generating image it disconnects or loading any model
what error do you get?
@@LaCarnevali disconnected and i cant reload it again so i have to restart the whole thing again
@@GAMINGGEEKzzz oh yeah - this is a common issue unfortunately. Nothing we can do about that
What are the words you used to generate your cat in these images?
Sorry, I don't remember!
I'm having an issue with my access to Google Colab. I'm unable to log in due to an access restriction.
Can you please help me resolve this issue?
I appreciate your help in advance......
Do you have Colab Pro? What is the issue exactly?
I think I am falling in love
Hi Laura thank you for the video! I was able to train the model but I have some problems with Gradio in particular:
ImportError: cannot import name 'Recordable' from 'gradio.events' (/usr/local/lib/python3.10/dist-packages/gradio/events.py)
Did they change some packages? Thank you a lot!
Try running it without Gradio?
😍🥰👍🙏
Have you made the video to continue this one ? The one to show how to use our model.
yes, but I haven't uploaded it, need to do that sorry >.
@@LaCarnevali Thanks a lot! Your videos are really helpful. Also, I've been trying to train a LoRA with Kohya, but I am getting an error. I searched a solution for hours, but couldn't find one. This is the error type: TypeError: memory_efficient_attention() got an unexpected keyword argument 'scale'. Maybe you could make a video about the possible error or answer here ? Anyay thanks for you videos they are awesome.
When i am going to local tunnel tab so its asking for password how to get it
hey, did you find the answer?
Hello, do you know if I can buy google colab pro with a google play gift card?
Hi, sorry, but I m not sure about that
BRO LAURA MEANS dICK IN MY COUNTRY MY WHOLE OFFICE IS LAUGHING SORRY BRO BUT FOR ME ITS LOT OF FUN
10:28 -- I know that English is not your first language, but in this case, when you mention that these files must have the same name. That's not really the case, otherwise, it won't work at all. What you meant was that these files must have the same prefix, but we need to change the suffix _(prefix from Old French "prefixe", from Latin "praefixus" and "suffix" from modern Latin "suffixum")_ e.g. prefix "Laura" and suffix "2" or "3", etc. ... "Laura 1", "Laura 2", etc.
4:09
12:03 isnt she looking left?
LOL, yeah
Are you italian or argentinian? XD
Italian