Friend, I would like to know if there is a command to pause the training. For now I do a forced closure and I don't know if it will harm the samples. I would really appreciate it! 😢
Don’t worry, killing the command process won’t harm the samples! To resume from the last checkpoint, use the --resume_from_checkpoint flag in the command line command
Unfortunately that’s typically a problem with ai. Unless you are training a hand lora in particular, the hands could warp and move somewhat unrealistically since ai tends to struggle with the concept of hands. Some seeds will be better than others
Friend, do you know if it is better to train without a background, whether it is a simple or white background to help learn the physics of the character. Or it's worse and then you don't understand any more instructions. With my setup I can't do many tests because it takes me a long time to train. Any advice helps me!
@ I would suggest using diverse backgrounds, not just simple or white! If you just use a simple white background, the Hunyuan model will tend to generate white backgrounds no matter what your prompt is.
There is already one in the diffusion-pipe repo! Take a look in this video where I am setting up the configs, you’ll see where I pull the example config settings file
Great video, thanks a lot! I just want to know ,do we need to stop the training first to test the LoRAs, or is it possible to test them simultaneously?
It’s a question of how much VRAM you have available. If you use a GPU like an H100 or maybe an A6000 Ada, you can test while training is running. Training takes almost exactly 24GB of VRAM when using 33 frame videos, and Hunyuan takes roughly 8-12GB minimum to run, so you need at least 32GB to attempt that. If you’re training with only images, you may be able to do low resolution testing on 24GB of VRAM.
When ever I try to run the Hunyuan video wrapper workflow to use loras, I get this error: "Error(s) in loading state_dict for AutoencoderKLCausal3D: Missing key(s) in state_dict: "encoder.down_blocks.0.resnets.0.norm1.weight etc.." - where as when I use the native Hunyuan workflow I get no such error. Im wondering if I need a different vae or something..
@@TheArt-OfficialTrainer Im using all the Kaji specific nodes, though I think Ive figured out the issue - I dont have sageattention and triton installed and they look like a pain to
You don’t need to use sageattention, you can just use flash attention in that dropdown instead. I’m not sure whether Triton is required if you don’t use sageattention.
@@TheArt-OfficialTrainer I didnt have flash attn installed either, so i used comfy attn which seems to be preinstalled. is there a big difference between sage, flash and comfy attn?
@@TheArt-OfficialTrainer Cheers for your help so far man. Im currently trying to get kijai's img2vid working (IP2V) but get this error AttributeError: 'LlavaForConditionalGeneration' object has no attribute 'final_layer_norm' .. Chat gpt seems to think this is related to not having a transformers lib installed, but there was no mention of needing to install a transformers lib on kijai's git page
They help save VRAM! With Kijai’s custom nodes, If you don’t use torch compile and block edit, you won’t be able to fit the full 720x1280x129f on 24GB VRAM with those nodes, I think you can fit the full resolution into
@@TheArt-OfficialTrainer How, by renting GPU online? What would happen if I tried to do it with a RTX 3060, just take a long time or not even possible?
Yeah, this tutorial uses RunPod. On a 3060, there’s a chance you could train using 512x512 images. And if that doesn’t work, you could even try 256x256. Just make sure to adjust the bucket size correctly.
@@TheArt-OfficialTrainer Thanks. I found there actually are a bunch of Hunyuan LoRAs available on CivitAI anyway, though if I ever do want to make one your video will come in handy.
thanks for showing this end to end, very helpful.
Glad it was helpful!
Nice work dude. Very helpful, clear, and detailed tutorial. I really appreciate the effort that went into this.
Glad I could help!
You are the best, THANK YOU SO MUCH!!!!!!!!! Hands down one of the best AI channels.
Thank you for your support! Glad I could help.
Good job, thanks for the valuable tutorial!
Absolutely stunning ! Thank you !
Really cool tutorial, detailed and clear, thank you
Thank you for watching, I’n glad I could help!
Amazing vid, thx!
Glad it helped!
great video, thanks!!!
Friend, I would like to know if there is a command to pause the training. For now I do a forced closure and I don't know if it will harm the samples. I would really appreciate it! 😢
Don’t worry, killing the command process won’t harm the samples! To resume from the last checkpoint, use the --resume_from_checkpoint flag in the command line command
@@TheArt-OfficialTrainerI understand. Friend I have been training, but loras have deformed hands. Do you know what parameter may be affecting that?
Unfortunately that’s typically a problem with ai. Unless you are training a hand lora in particular, the hands could warp and move somewhat unrealistically since ai tends to struggle with the concept of hands. Some seeds will be better than others
Friend, do you know if it is better to train without a background, whether it is a simple or white background to help learn the physics of the character. Or it's worse and then you don't understand any more instructions. With my setup I can't do many tests because it takes me a long time to train. Any advice helps me!
@ I would suggest using diverse backgrounds, not just simple or white! If you just use a simple white background, the Hunyuan model will tend to generate white backgrounds no matter what your prompt is.
Fantastic video!! Can you please share a good config settings file to get started?
There is already one in the diffusion-pipe repo! Take a look in this video where I am setting up the configs, you’ll see where I pull the example config settings file
Great video, thanks a lot! I just want to know ,do we need to stop the training first to test the LoRAs, or is it possible to test them simultaneously?
It’s a question of how much VRAM you have available. If you use a GPU like an H100 or maybe an A6000 Ada, you can test while training is running. Training takes almost exactly 24GB of VRAM when using 33 frame videos, and Hunyuan takes roughly 8-12GB minimum to run, so you need at least 32GB to attempt that. If you’re training with only images, you may be able to do low resolution testing on 24GB of VRAM.
When ever I try to run the Hunyuan video wrapper workflow to use loras, I get this error: "Error(s) in loading state_dict for AutoencoderKLCausal3D:
Missing key(s) in state_dict: "encoder.down_blocks.0.resnets.0.norm1.weight etc.." - where as when I use the native Hunyuan workflow I get no such error. Im wondering if I need a different vae or something..
Are you using the “Hunyuan Decode” node? There is a specific one for Kijai’s nodes
@@TheArt-OfficialTrainer Im using all the Kaji specific nodes, though I think Ive figured out the issue - I dont have sageattention and triton installed and they look like a pain to
You don’t need to use sageattention, you can just use flash attention in that dropdown instead. I’m not sure whether Triton is required if you don’t use sageattention.
@@TheArt-OfficialTrainer I didnt have flash attn installed either, so i used comfy attn which seems to be preinstalled. is there a big difference between sage, flash and comfy attn?
@@TheArt-OfficialTrainer Cheers for your help so far man. Im currently trying to get kijai's img2vid working (IP2V) but get this error AttributeError: 'LlavaForConditionalGeneration' object has no attribute 'final_layer_norm' .. Chat gpt seems to think this is related to not having a transformers lib installed, but there was no mention of needing to install a transformers lib on kijai's git page
Whats the purpose of the block edit node and torch compile settings node (noob)? why not just use the lora select?
They help save VRAM! With Kijai’s custom nodes, If you don’t use torch compile and block edit, you won’t be able to fit the full 720x1280x129f on 24GB VRAM with those nodes, I think you can fit the full resolution into
Following your video, I encountered the following error: No module named 'hyvideo' Is there a solution for this error?
During Lora Training? Or during ComfyUI workflow?
@@TheArt-OfficialTrainer Lora
@임수경-e5v can you share the actual error? I would guess either you’re missing a package or the path to your hunyuan video model is incorrect
I have a better idea, you make the loras and then make them available for download, because who has $1600 for a 24 GB rig?
Training a lora this way should only cost $3 max! There are too many Lora possibilities for me to create all the ones that people want.
@@TheArt-OfficialTrainer How, by renting GPU online? What would happen if I tried to do it with a RTX 3060, just take a long time or not even possible?
Yeah, this tutorial uses RunPod. On a 3060, there’s a chance you could train using 512x512 images. And if that doesn’t work, you could even try 256x256. Just make sure to adjust the bucket size correctly.
@@TheArt-OfficialTrainer Thanks. I found there actually are a bunch of Hunyuan LoRAs available on CivitAI anyway, though if I ever do want to make one your video will come in handy.
Do u have a discord or any type of contact/ socials so I can ask u questions if I struggle w things / need help?
Message me on my Patreon!
I’m thinking about creating a Discord in the future, but haven’t done it yet