A little explanation on base shift and max shift for anyone looking to play with that: Base shift is a small, consistent adjustment that stabilizes the image generation process, while max shift is the maximum allowable change to the latent vectors, preventing extreme deviations in the output. Together, these parameters balance stability and flexibility in image generation. Using a bird as an example: Increasing Base Shift: Raising the base shift results in a more consistent and stable depiction of the bird. For instance, the image might consistently show a bird with clear, well-defined features such as a distinct beak, feathers, and wings. However, this increased stability could lead to less variation, making the bird’s appearance feel repetitive or overly uniform. Decreasing Base Shift: Lowering the base shift allows for more subtle variations and intricate details, like nuanced patterns in the bird’s feathers or unique postures. However, this added variability might make the bird’s image less stable, with occasional irregularities or minor distortions. Increasing Max Shift: A higher max shift enables the model to explore the latent space more freely, leading to creative or exaggerated interpretations of the bird. For example, the bird might develop surreal colors, elongated wings, or fantastical plumage, but it risks straying far from a realistic bird representation. Decreasing Max Shift: Reducing the max shift tightly constrains the model, resulting in a more controlled and realistic depiction of the bird. The image is likely to stay close to a conventional bird appearance, but it might lack creative or distinctive elements that make the bird unique or captivating.
Thanks so much ! It's a great landmark for local gen video and sets the bar on speed, even if the trade off was model depth and image quality, i remember when SD1.4 was about the same in both regards! here's the next 12 months for Video gen !!
Could you create a video that explains all the basic parameters and what they affect? Examples would be great because written articles often don't have any and trying them with slow GPU is just very slow way to learn. I have learned by trial and error what for example CFG does but still don't fully understand its interactions with other node parameters.
i provided some "sampler/CFG/steps" garden type workflows so people can see and experiment for themselves, also we did a lot of testing on Livestream, so if you can't run them - that's the best bet. I'll be renaming/adding more info to the live stream descriptions soon, so watch this space.
I had to update from the batch file in the updates folder (comfyui) and then the custom nodes finally installed correctly. Simply using the built in update everything button in manager did not work.
I also included some .bat files for the comfyui in my DJZ-Nodes for example to force the installation of a node using it's requirements (can copy into any custom node using requirements and run) and also the pip resolver update for ComfyUI portable.
don't forget that this is the research/eval model so basically a demo for the full model that is coming next - i think it did a great job setting the bar for speed, despite having some weaknesses :)
in the V2 pack release (github has no versions only civit) you will see "loopedmotion3" and "Extendo" these show ways to make the video as long as you want by chaining the last frame as the first frame with img2video, it has limitations sure, but with some skill & planning you can get unlimited length tbh
@@FiveBelowFiveUK That's awesome man.. Yeah currently running a 4090 in my office, and the dev setup about 20 workflows. I haven't figured out the PULID yet, but, I'd think there would be some path of creating a movie with the PULID being connected to the movie gen, starting with some storyboard. I haven't tested that, but I assum that's it.
I have not tested this personally, but i have people in my groups that claimed to do video generations on 8GB vram systems, if a little slower than what i'm showing.
These companies need to roll out this stuff more gradually-these constant dopamine spikes are wrecking my sleep! Oh, and don't think we forgot-you still owe us that deep dive into flux tools. 😉
Do Not Fear ! The Flux Dive is coming - we are adding the Detailer DAEMON to all the workflows, this is an considerable effort if you saw the size of my Flux pack - don't miss it :) it's coming very soon !
Another 8-12 months and these obscure interfaces will start to go away in favor of far more intuitive controls and production friendly ways to create video.
actually, this is an extendible platform with API support. I do a lot of contract work and what you see here is often in the backend of many popular web based paid services. In other words, if you are able to and have the right skillset - you can build web services on top of this. Additionally it is possible to create standalone software with the same pipeline that comfyUI is running on, so actually we are already there - maybe you will build the next app with those intuitive controls you desire. All it takes is an idea. The code is already here.
THIS IS NOT HUGE....I tested half the day and all night....and no way this even comes close to what commercial platforms are able to offer....the only thing they do better is faster renders but the quality and prompt adherence is pure shite! Not to mention it is far too early to be claiming something that has unfriendly UX is going to be huge or is the next best thing.... Disingenuous, at best; dishonest, at worst.
@@fotszyrzk79 I don't know what you're expecting out of a fast model that doesn't require a GPU that breaks bank that isn't fully trained yet and will be fully trained and released open source, but it really seems like you're missing the point.
A little explanation on base shift and max shift for anyone looking to play with that:
Base shift is a small, consistent adjustment that stabilizes the image generation process, while max shift is the maximum allowable change to the latent vectors, preventing extreme deviations in the output. Together, these parameters balance stability and flexibility in image generation.
Using a bird as an example:
Increasing Base Shift: Raising the base shift results in a more consistent and stable depiction of the bird. For instance, the image might consistently show a bird with clear, well-defined features such as a distinct beak, feathers, and wings. However, this increased stability could lead to less variation, making the bird’s appearance feel repetitive or overly uniform.
Decreasing Base Shift: Lowering the base shift allows for more subtle variations and intricate details, like nuanced patterns in the bird’s feathers or unique postures. However, this added variability might make the bird’s image less stable, with occasional irregularities or minor distortions.
Increasing Max Shift: A higher max shift enables the model to explore the latent space more freely, leading to creative or exaggerated interpretations of the bird. For example, the bird might develop surreal colors, elongated wings, or fantastical plumage, but it risks straying far from a realistic bird representation.
Decreasing Max Shift: Reducing the max shift tightly constrains the model, resulting in a more controlled and realistic depiction of the bird. The image is likely to stay close to a conventional bird appearance, but it might lack creative or distinctive elements that make the bird unique or captivating.
@@Andro-Meta good explanation!
thanks for adding this detailed reply - Pinned !
This is HUGE! Thanks for being a hero in the community and showing us how powerful local video gen could be!
Thanks so much ! It's a great landmark for local gen video and sets the bar on speed, even if the trade off was model depth and image quality, i remember when SD1.4 was about the same in both regards! here's the next 12 months for Video gen !!
Thank you for presenting this stunning LTX model. Try right now out
Hi thanks for all your work. I will test it today. Will leave some review videos on civitai when I get it to work.
always appreciate to see what people make ! thx
Could you create a video that explains all the basic parameters and what they affect? Examples would be great because written articles often don't have any and trying them with slow GPU is just very slow way to learn. I have learned by trial and error what for example CFG does but still don't fully understand its interactions with other node parameters.
i provided some "sampler/CFG/steps" garden type workflows so people can see and experiment for themselves, also we did a lot of testing on Livestream, so if you can't run them - that's the best bet. I'll be renaming/adding more info to the live stream descriptions soon, so watch this space.
i was never interested in local models until this came out im going to find the best settings and squeeze every last thing of this goldmine
yes if you have an idea this is an amazing leap forward !
I think the model is trained at 25 also I've been getting 5 seconds no problem however have to do 3 seconds whenever I change the prompt
yes i changed my workflows in V2 to reflect this, i think we have to do (24fps*Seconds)+1
did some testing with this and noticed that higher number of steps like 50-70 help fast moving scenes.
interesting
❤thank You sir
I had to update from the batch file in the updates folder (comfyui) and then the custom nodes finally installed correctly. Simply using the built in update everything button in manager did not work.
I had the same problem. I try the same thing you did and it worked. Thanks
I also included some .bat files for the comfyui in my DJZ-Nodes for example to force the installation of a node using it's requirements (can copy into any custom node using requirements and run) and also the pip resolver update for ComfyUI portable.
What software do you use to create the white moving avatar bro . Thanks
I built it out with ComfyUI of course :) I give a tip how it's used in plain sight on my first channel video, you should see there
This is fast, but we need to be able to control the strength of the latents and images.
don't forget that this is the research/eval model so basically a demo for the full model that is coming next - i think it did a great job setting the bar for speed, despite having some weaknesses :)
so what is the longest you got it to do? can you do 30 seconds?
in the V2 pack release (github has no versions only civit) you will see "loopedmotion3" and "Extendo" these show ways to make the video as long as you want by chaining the last frame as the first frame with img2video, it has limitations sure, but with some skill & planning you can get unlimited length tbh
@@FiveBelowFiveUK That's awesome man.. Yeah currently running a 4090 in my office, and the dev setup about 20 workflows. I haven't figured out the PULID yet, but, I'd think there would be some path of creating a movie with the PULID being connected to the movie gen, starting with some storyboard. I haven't tested that, but I assum that's it.
It would be cool if you opened a discord server where people could discuss their projects and problems
and i have one ! the join link is in the description of this video ;) Jump In !
What Vram is required to run this ? Or is it one of those that needs RTX 4090 to run locally ?
works fine on RTX 3060 12gb
I'm running fine 4070 rtx 8vram 16 gb this ltx
i've had people in my group using my workflows report 8gb to 12gb and all working fine, if a speed impact at the 8GB end of the scale.
thanks for sharing - everyone needs this type of info
thanks for sharing your results and specs, this is needed by a lot of people !
its great and all but is it 8 vram crash proof
I have not tested this personally, but i have people in my groups that claimed to do video generations on 8GB vram systems, if a little slower than what i'm showing.
Can be any made for free .so not with paid website ....? ❤
these are all running on local GPU, that means you just need Nvidia Graphics card and there is nothing to pay for local generations.
@FiveBelowFiveUK amazing what about cloud can run prompts free way?
These companies need to roll out this stuff more gradually-these constant dopamine spikes are wrecking my sleep! Oh, and don't think we forgot-you still owe us that deep dive into flux tools. 😉
Do Not Fear ! The Flux Dive is coming - we are adding the Detailer DAEMON to all the workflows, this is an considerable effort if you saw the size of my Flux pack - don't miss it :) it's coming very soon !
Another 8-12 months and these obscure interfaces will start to go away in favor of far more intuitive controls and production friendly ways to create video.
actually, this is an extendible platform with API support. I do a lot of contract work and what you see here is often in the backend of many popular web based paid services. In other words, if you are able to and have the right skillset - you can build web services on top of this. Additionally it is possible to create standalone software with the same pipeline that comfyUI is running on, so actually we are already there - maybe you will build the next app with those intuitive controls you desire. All it takes is an idea. The code is already here.
THIS IS NOT HUGE....I tested half the day and all night....and no way this even comes close to what commercial platforms are able to offer....the only thing they do better is faster renders but the quality and prompt adherence is pure shite! Not to mention it is far too early to be claiming something that has unfriendly UX is going to be huge or is the next best thing....
Disingenuous, at best; dishonest, at worst.
He made it pretty clear that for local video ai creation, this is huge. And this is.
Agree... it is fast as hell, as the OP mentioned, but quality isn't that good... I feel like CogVideoX is better, albeit slower.
The only huge thing is fast rendering time, the outcome is shit!
@@fotszyrzk79 I don't know what you're expecting out of a fast model that doesn't require a GPU that breaks bank that isn't fully trained yet and will be fully trained and released open source, but it really seems like you're missing the point.
@@Andro-Meta give me usecase for this model.