Unlock LoRA Mastery: Easy LoRA Model Creation with ComfyUI - Step-by-Step Tutorial!
ฝัง
- เผยแพร่เมื่อ 16 มี.ค. 2024
- Have you ever wanted to create your own customized LoRA model that perfectly fits your needs without having to compromise with predefined ones? In this easy-to-follow tutorial, I'll guide you through the process of creating your LoRA model using ComfyUI. No more limitations imposed by standard models, just the freedom to create exactly what you desire. Join us and discover how to bring your vision to life with LoRA!
** Links from the Video Tutorial **
LoRA Paper: arxiv.org/abs/2106.09685
Min-SNR Expanation: github.com/kohya-ss/sd-script...
Kohya Reposotory: github.com/bmaltais/kohya_ss
Lora-Training-in-Comfy (My Fork): github.com/Nuked88/Lora-Train...
Lora-Training-in-Comfy (Official): github.com/LarryJane491/Lora-...
Image-Captioning-in-ComfyUI (My Fork): github.com/Nuked88/Image-Capt...
Image-Captioning-in-ComfyUI (Official): github.com/LarryJane491/Image...
Algorithm List: github.com/KohakuBlueleaf/LyC...
VIDEO - LoRA Usage Tutorial: • Mastering ComfyUI: How...
VIDEO - Custom Node Installation Tutorial: • Mastering ComfyUI: Cus...
Workflow** : / 100535747
** Let me be EXTREMELY clear: I don't want you to feel obligated to join my Patreon just to access this workflow. My Patreon is there for those who genuinely want to support my work. If you're interested in the workflow, feel free to watch the video - it's not that long, I promise! 🙏
❤️❤️❤️Support Links❤️❤️❤️
Patreon: / dreamingaichannel
Buy Me a Coffee ☕: ko-fi.com/C0C0AJECJ - แนวปฏิบัติและการใช้ชีวิต
![[Tutorial] Create Your Own LoRA | Easy and Free! - Google Colab](http://i.ytimg.com/vi/Un9SHPVAAbE/mqdefault.jpg)
Thank you, my friend! ComfyUI is so much easier with your videos!
This was amazing. Struggled a lot with Konya the past few weeks. Going to try comfy now
Ive always wanted to do training ..thanks for this!
Thanks! Have been struggling with the original version (did not get it to work). Your nodes really worked great and simple in the first attempt. The long explanation of the Lora-traning node also appreciated.
(Note: building the lora with 45 images did take some time > 3.5 hrs, using a 4070ti)
Thank you....
awesome
Unfortunately these custom scripts bricked my Comfy installation "Assertion Error - Torch not compiled with CUDA enabled"
I had to remove your scripts and run a python and Comfy update to get it back.
Hello and thanks, a question, if I interrupt the process there is a backup or I have to begin from start? Thank you
I receive error messages when creating image captions.
Error occurred when executing GPT Sampler [n-suite]:
list index out of range
File "E:\AI\AITools\ComfyUI\execution.py", line 141, in recursive_execute
input_data_all = get_input_data(inputs, class_def, unique_id, outputs, prompt, extra_data)
File "E:\AI\AITools\ComfyUI\execution.py", line 26, in get_input_data
obj = outputs[input_unique_id][output_index]
Please help me fix this, thanks.
Good Video. It would be much more interesting if SDXL was supported. Also training LoRA for LLM through ComfyUi in a no-code way would be interesting.
I agree, I hope sdxl is supported soon, also, if I find a method to make LoRA for LLM via comfyui I will see if I can make a video!
What advantage does SDXL give you? Thanks!
@@DreamingAIChannel What advantage does SDXL give you? Thanks!
@@brianmonarchcomedy Well, compared to SD 1.5, it offers a higher native resolution of 1024x1024, higher quality images and, in general, better "prompt understanding"
@@DreamingAIChannel thanks! I’ve been using the “Lora training in ComfyUI” node. do you happen to know if it is SDXL compatible? Or would I be better off using Kohya? I’m trying to make realistic people (could be me or celebrities) in cool stable diffusion graphic environments. Thanks!
what should be resolution of these images?
I am considering doing some 360 capturing like you would for photogrammetry image to 3d workflows... and then making character loras out of them. Im thinking I can use an open pose sequence to train a face sequence, a body sequencec, and several outfit sequences from at least 12 angles, or perhaps 7 with 5 mirrored. If I'm feeling extra dedicated, might do some lighting variations. Please, give me your thoughts on this approach. It might be useful for character animation, for instance.
I have never heard of such an approach but I think it can really work. Certainly a 3D model gives you more control over what you supply to the model, so that the training can be more precise. Let me know how it goes!
I'm doing the same thing, did you try this yet?
@@collectiveunconscious3d I'm trying to get photogrammetry to work right now
pls help me, i can't run lora caption load
error
Error occurred when executing LoRA Caption Load:
cannot access local variable 'image1' where it is not associated with a value
uhm, i need the full log!
@@DreamingAIChannel its because his pics are jpg and not png. I had the same issue, mine now is train finished, prompt executed in 3.00 seconds. Nothing is trained and no lora in model folder
@@amorgan5844hi! usually in the execution log I can understand what's wrong, so if you want you can send me that and I can help you!
@@Nuked please help me, i get an error like this:
C:\My Program Files\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\Lora-Training-in-Comfy/sd-scripts/train_network.py
C:\My Program Files\AI\ComfyUI_windows_portable\python_embeded\python.exe: Error while finding module specification for 'accelerate.commands.launch' (ModuleNotFoundError: No module named 'accelerate')
how to fix it?
Thank you for the video, but... The "training" took 17 seconds, produced no file in the specified directory, and I got no error messages. it just said "Training completed in 17.56 seconds." in the terminal. :(
Hi! You need to give me the full log (the one that's in the console when you start the training) so we can see what's wrong!
@@DreamingAIChannelI actually got this working by using someone else's guide. You mention the folder structure being important, and in your example you show the proper structure, but you neglected to say that the folder containing the dataset MUST be at the ROOT of whatever directory you're using. It must be like this: "C:Folder/Dataset/". They also had slightly different instructions for naming the dataset folder. I followed this advice, and it finally worked for me. Also, they had much more realistic training settings for someone like me... it already took 14 hours to train 50 epochs... 400 like you advised would have rendered my computer useless for a week.
You say that you don't want people to begin to feel as though you're holding out your workflow in order to elicit patreon subs, and I don't want to try and counter that, but have you considered uploading it to a place like civitai, or comfy workflows, so that people can access it?
@@DreamingAIChannel i got what he got : in simple_launcher
raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)
subprocess.CalledProcessError: Command '['C:\\Users\\ariva\\AppData\\Local\\Programs\\Python\\Python310\\python.exe', 'custom_nodes/Lora-Training-in-Comfy/sd-scripts/train_network.py', '--enable_bucket', '--pretrained_model_name_or_path=D:\\Ai\\stable-diffusion-webui\\models\\Stable-diffusion\\juggernaut_reborn.safetensors', '--train_data_dir=C:/database/hasby', '--output_dir=D:\
ew_ComfyUI_windows_portable_nvidia_cu121_or_cpu\\ComfyUI_windows_portable\\ComfyUI\\models\\loras', '--logging_dir=./logs', '--log_prefix=hasby', '--resolution=672,672', '--network_module=networks.lora', '--max_train_epochs=400', '--learning_rate=1e-4', '--unet_lr=1.e-4', '--text_encoder_lr=1.e-4', '--lr_scheduler=cosine_with_restarts', '--lr_warmup_steps=0', '--lr_scheduler_num_cycles=1', '--network_dim=32', '--network_alpha=32', '--output_name=hasby', '--train_batch_size=1', '--save_every_n_epochs=50', '--mixed_precision=fp16', '--save_precision=fp16', '--seed=14', '--cache_latents', '--prior_loss_weight=1', '--max_token_length=225', '--caption_extension=.txt', '--save_model_as=safetensors', '--min_bucket_reso=256', '--max_bucket_reso=1584', '--keep_tokens=0', '--xformers', '--shuffle_caption', '--clip_skip=2', '--optimizer_type=AdamW8bit', '--persistent_data_loader_workers', '--log_with=tensorboard', '--clip_skip=2', '--optimizer_type=AdamW8bit', '--persistent_data_loader_workers', '--log_with=tensorboard', '--clip_skip=2', '--optimizer_type=AdamW8bit', '--persistent_data_loader_workers', '--log_with=tensorboard', '--clip_skip=2', '--optimizer_type=AdamW8bit', '--persistent_data_loader_workers', '--log_with=tensorboard', '--clip_skip=2', '--optimizer_type=AdamW8bit', '--persistent_data_loader_workers', '--log_with=tensorboard']' returned non-zero exit status 2.
Train finished
Prompt executed in 7.50 seconds
i have the resolution in that log on 672 because that was the biggest dimension of the largest image as i had 512 originally and got the same message
The arrows are drawn straight for you. With which mode can we do this?You are connecting two models together. Mine is not like yours. Yours is straight, mine is like a
rope
Hi! Here i've explained how to put the straight lines: th-cam.com/video/AjwfswzLmxU/w-d-xo.html
your fork rules :)
how to make comfyui into telegram bot? please tutor
All my captions were generated, and at the moment of execution, after having correctly set up the advanced LoRA training node in Comfy, I started the training and received the following error message: Error occurred when executing LoRA Caption Save:
list index out of range
File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\Image-Captioning-in-ComfyUI\LoRAcaption.py", line 53, in save_text_file
filename = self.generate_filename(path, namelistsplit, file_extension)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\Image-Captioning-in-ComfyUI\LoRAcaption.py", line 65, in generate_filename
filename = f"{namelistsplit[counter-1]}{extension}"
~~~~~~~~~~~~~^^^^^^^^^^^
Hi,If you already have the captions just disable those nodes (right click > bypass) and keep only the training one, so you won't have to deal with that error!
Please help me. How to fix this:
The following values were not passed to `accelerate launch` and had defaults used instead:
`--num_processes` was set to a value of `1`
`--num_machines` was set to a value of `1`
`--mixed_precision` was set to a value of `'no'`
`--dynamo_backend` was set to a value of `'no'`
To avoid this warning pass in values for each of the problematic parameters or run `accelerate config`.
Train finished
Prompt executed in 5.62 seconds
wait, you don't have any error? Can you tell me what you have on top of this?
I want to hear your real voice cause this one is definitely not yours, but AI cloned
the training is finished in 4 seconds and nothing is added to the folder i created for the model.
the log :
D:\AI\Comfyui\ComfyUI\custom_nodes\Lora-Training-in-Comfy/sd-scripts/train_network.py
The following values were not passed to `accelerate launch` and had defaults used instead:
`--num_processes` was set to a value of `1`
`--num_machines` was set to a value of `1`
`--mixed_precision` was set to a value of `'no'`
`--dynamo_backend` was set to a value of `'no'`
To avoid this warning pass in values for each of the problematic parameters or run `accelerate config`.
C:\Users\HelpTech\AppData\Local\Programs\Python\Python310\python.exe: can't open file 'D:\\AI\\Comfyui\\custom_nodes\\Lora-Training-in-Comfy\\sd-scripts\\train_network.py': [Errno 2] No such file or directory
Traceback (most recent call last):
File "C:\Users\HelpTech\AppData\Local\Programs\Python\Python310\lib
unpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users\HelpTech\AppData\Local\Programs\Python\Python310\lib
unpy.py", line 86, in _run_code
exec(code, run_globals)
File "C:\Users\HelpTech\AppData\Local\Programs\Python\Python310\lib\site-packages\accelerate\commands\launch.py", line 1027, in
main()
File "C:\Users\HelpTech\AppData\Local\Programs\Python\Python310\lib\site-packages\accelerate\commands\launch.py", line 1023, in main
launch_command(args)
File "C:\Users\HelpTech\AppData\Local\Programs\Python\Python310\lib\site-packages\accelerate\commands\launch.py", line 1017, in launch_command
simple_launcher(args)
File "C:\Users\HelpTech\AppData\Local\Programs\Python\Python310\lib\site-packages\accelerate\commands\launch.py", line 637, in simple_launcher
raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)
subprocess.CalledProcessError: Command '['C:\\Users\\HelpTech\\AppData\\Local\\Programs\\Python\\Python310\\python.exe', 'custom_nodes/Lora-Training-in-Comfy/sd-scripts/train_network.py', '--enable_bucket', '--pretrained_model_name_or_path=D:\\AI\\Comfyui\\ComfyUI\\models\\checkpoints\\bismuthmix_v30.safetensors', '--train_data_dir=D:/AI/Art/milffy', '--output_dir=D:\\AI\\Art\\milffy model', '--logging_dir=./logs', '--log_prefix=Milffy', '--resolution=512,512', '--network_module=networks.lora', '--max_train_epochs=5000', '--learning_rate=1e-4', '--unet_lr=1.e-4', '--text_encoder_lr=1.e-4', '--lr_scheduler=cosine_with_restarts', '--lr_warmup_steps=0', '--lr_scheduler_num_cycles=1', '--network_dim=32', '--network_alpha=32', '--output_name=Milffy', '--train_batch_size=1', '--save_every_n_epochs=100', '--mixed_precision=fp16', '--save_precision=fp16', '--seed=26', '--cache_latents', '--prior_loss_weight=1', '--max_token_length=225', '--caption_extension=.txt', '--save_model_as=safetensors', '--min_bucket_reso=256', '--max_bucket_reso=1584', '--keep_tokens=0', '--xformers', '--shuffle_caption', '--clip_skip=2', '--optimizer_type=AdamW8bit', '--persistent_data_loader_workers', '--log_with=tensorboard']' returned non-zero exit status 2.
Train finished