If you watch any tv show, you'll notice a scene is split up into 6 to 9 second clips between camera angles. Establishing shots can be up to 15 seconds. Check it out. Cue up your favorite show and start counting until it switches to another camera. With this in mind, it should be possible to replicate a scene one segment at a time. Then you just have to splice them together using premiere or its open source equivalent.
long ago, around 2007, I remember watching something on Discovery channel, a special. The person who was being filmed said these words. In the future, actors will no longer be used because movies will be created without them. We're just about there. Good or bad, it has its applications.
I make music in a band, and I create music videos for our songs on the platforms. I have used AI as a collaborator on a song, and I use scenes created by AI as best I can, with what I can find. Your channel keeps me on the cutting edge of Music and Video Creation. I am very excited at the idea of me putting myself and my other band members in some AI videos. Thank you for this channel.
@@_GOWI_ If you have any songs on TH-cam, please send me a link. After that, I work in the studio with TH-camrs: they put down a acapella video, or My lead singer playing synth and singing, then I add the rhythm section. Once the song is done, we start to put together the music video with AI. Send me a TH-cam link to one of your songs please?
- [00:00](th-cam.com/video/jZWRENqCl6I/w-d-xo.html) 🎥 Overview of Story Diffusion Model - Introduces "Story Diffusion," an open-source AI video model. - Highlights the model's ability to maintain character consistency and adhere to reality and physics in videos. - Discusses the model's advancement in character consistency, including facial, clothing, and body type consistency. - [01:03](th-cam.com/video/jZWRENqCl6I/w-d-xo.html) 🖼️ Creation of AI Comics using Story Diffusion - Demonstrates the generation of AI Comics with consistent characters and movement. - Explains the process of creating sequential images and animating them for comic generation. - Discusses the potential of AI Comics production enabled by Story Diffusion. - [02:56](th-cam.com/video/jZWRENqCl6I/w-d-xo.html) 🤔 Comparison with Existing AI Video Generators - Contrasts the capabilities of Story Diffusion with other AI video generators. - Highlights the realism, consistency, and fluidity achieved by Story Diffusion. - Discusses limitations and minor imperfections observed in Story Diffusion's output. - [07:11](th-cam.com/video/jZWRENqCl6I/w-d-xo.html) 💻 Accessibility and Usage of Story Diffusion - Addresses the accessibility of Story Diffusion, available as open-source but lacking a usable interface. - Describes options for accessing Story Diffusion, including downloading/installing or running online on a Cloud Server. - Discusses the significance of Story Diffusion's multiple character inclusion feature for scene consistency. - [09:45](th-cam.com/video/jZWRENqCl6I/w-d-xo.html) 🧠 Mechanisms Behind Story Diffusion Model - Explains the technical mechanisms employed by Story Diffusion, including consistent self-attention and story splitting. - Describes how consistent self-attention ensures visual coherence across generated images. - Discusses the process of story splitting and motion prediction in creating narrative sequences. - [14:48](th-cam.com/video/jZWRENqCl6I/w-d-xo.html) 📈 Evolution of AI Video Generation - Reflects on the advancements in AI video generation enabled by Story Diffusion. - Emphasizes the potential for creating realistic and cohesive scenes with AI-generated videos. - Encourages exploration of Story Diffusion's capabilities and contributions to the field of AI video production.
7:28 You got me at multiple consistent characters in scenes. It’s really important for storytelling to be able to have multiple consistent characters interacting with each other.
An open-source neural network similar to TOR? What if we make an open-source neural network that would work as a swarm, in the spirit of the Tor network or torrents, crypto? So that all computers would be part of its "consciousness", and not some servers?
When I see glitches such as the ball going through the metal rim I can't help but to think what this technology can do someday for special effects that are closer to realism than CGI while also being simpler to accomplish.
love the channel but going to be a little critical... the problem with this is that its not open source, you can't make videos, - Source code of Video Generation Model not even available - Pretrained weight of Video Generation Model not even available You need to be skeptical what you cant test yourself Machine learning professionals from Alibaba and Bytedance companies and tiktok etc all have made huge claims but have never made it open source. Also the comic book strips frankly suck and where considered bad 3 years ago. Since you could just use controlnet, fine-tuned models and SD1.5
It has always worked for me. Click on one of the examples at the bottom and it will pre-fill the fields, then click generate. It will work. Then, you can edit the fields to fit your needs. I have no idea how to make a movie with enough frames for fluid movement, however.
@@redleader7988 yeah i've been chipping away at it and exploring it locally and the functionality to generate video doesn't seem to be present, unless i'm missing something
Or animating existing comics… I can see it already, no need for Hollywood adaptations when you can simply plug in a comic and have it create a authentic 1-1 movie based on any book or comic / manga…
For what i read in their paper, their attention method work through batch generation, video length is limited by memory and that is probably why resolution is so "low".
I've tried it today and I would just say it's ok but far from what the demo has shown. It still suffers from the typical Stable Diffusion problems which is prompt understanding and complex scenes. In its current state, I don't think it is useable. It is an improvement from what we had. Hopefully future updates could fix some of the obvious issues.
can you plz explain how to continue to further shots with same scene, consistent character, and camera moves when we are creating a movie? because all the view gets changed for next shot, and sometimes character as well. Every expert on youtube is showing just 1 shot. Please let us learn to shoot a scene of 6 shots, 3 minutes long, showing 7 friends. Refer to any sample scene from series FRIENDS. Many thanks
I haven't tried this tool yet, but I'm thinking about using keywords like "Wide shot", "Long shot", "Panoramic", maybe even "Fisheye" - then cropping it, then AI upscaling.
@@JohnSmith-gt3be "Created" by the machine, having stolen centuries long blood, sweat and tears hard work of creative people, making billions for AI companies, rendering all artists dead or living irrelevant. That is some life and content we can suck on!
@@Zareh_Abrahamian I think it’s an amazing idea. When the horsing industry was destroyed by the automobile industry the world became a much better place. I guess you’re just on the side of the horses today.
I'm trying to test a video in the github page, but I keep getting an "ERROR" in the render section a few seconds after... Although this has happened without an image reference, I'm not sure if I fully understand what they mean by "𝘪𝘧 𝘺𝘰𝘶 𝘢𝘥𝘥 𝘵𝘩𝘦 𝘙𝘦𝘧-𝘐𝘮𝘢𝘨𝘦, 𝘮𝘢𝘬𝘪𝘯𝘨 𝘴𝘶𝘳𝘦 𝘵𝘰 𝘧𝘰𝘭𝘭𝘰𝘸 𝘵𝘩𝘦 𝘤𝘭𝘢𝘴𝘴 𝘸𝘰𝘳𝘥 𝘺𝘰𝘶 𝘸𝘢𝘯𝘵 𝘵𝘰 𝘤𝘶𝘴𝘵𝘰𝘮𝘪𝘻𝘦 𝘸𝘪𝘵𝘩 𝘵𝘩𝘦 𝘵𝘳𝘪𝘨𝘨𝘦𝘳 𝘸𝘰𝘳𝘥: 𝘪𝘮𝘨, 𝘴𝘶𝘤𝘩 𝘢𝘴: 𝘮𝘢𝘯 𝘪𝘮𝘨 𝘰𝘳 𝘸𝘰𝘮𝘢𝘯 𝘪𝘮𝘨 𝘰𝘳 𝘨𝘪𝘳𝘭 𝘪𝘮𝘨" I suppose you have your image subject and just put the letters img right after that word in the prompt? Any idea how to make the thing work? Tried in 2 browsers as well, and still getting that Error in red....
tbh my hopes are still on LTX. It is an awesome platform already but certainly has to learn alot. Time will tell how well it´ll turn out, but I´m rather positive at moment.
I can't testify to the video stuff. However, I tried the "comic book" (think graphic novel) stuff on 2024-MAY-09. Here's the problem. The system ignores your "story beat" prompts and just generates five different versions of your "character" prompt. Now, that's not a bad thing because you can then take that into Photoshop and move everything around into place. However, this is not a one-stop shop for graphic novel generation. Second, if you say "speaking", the character is not speaking. If you say "knock on door", the photo looks like a selfie. If you choose "photographic" you'll still get comic book style. Also, it does not apply the images to a "page". So if you ask for a 1024x512 page, you'll get five 1024x512 panels and you'll have to build your own page in Photoshop. Unlike a Stable Diffusion site that will give you four images per press of the button, this site will give you the same quantity of images but make sure the uniforms are consistent. It seems as if the system uses text-to-image to make a baseline and then runs that through an image-to-image to get a final result. So far, there are a lot of promises but very few victories. We'll see.
I have some stuff but it's not stable diffusion or story diffusion as my graphics card popped. 😢 But I do have easy instructions on installing local models with any hardware ect.
Is there a valid reason why the links of the sites featured in the video are not included in all the hefty amount of links provided? I don't understand why AI content creators neglect to include the mentioned links and/or timestamps in their videos. Not only does it make the video more useful, but I believe it is essential for creating a valuable content.
13:01 that's not a scene in England...it's an astoundingly accurate rendition of a view probably across Princes Street up towards the mound in Edinburgh, Scotland. I've seen that view often. Details like the unusual lamp and recognisable buildings are astoundingly accurate, actually. I nearly fell off my seat when I saw it! Good grief the bus is even the 900 city link Edinburgh to Glasgow bus and the text on it is legible!
@robertruffo2134 nonsense. Try learning how this stuff works. There is not a compression algorithm known to man that could compress the volume of information in the training data down to the size of the download. The model learns by training on data not dissimilarly to how a human does. It doesn't have a database with the information in it. Only weights and biases representing what it learnt from studying the training data. When people say things like you say: 1. I feel for them. Sucks that this affects livelihoods. 2. But it's clear you don't actually understand how this works.
@@iaincampbell4422 Dude, I know how this works. I even worked on earlier versions of this kind of system. If you think they learn "the way a human does" you have drunk many liters of tech hype Kool-Aid. Also, yes they do "keep" training data. Have a look at some recent lawsuits.
This definitely seems to have only the comic generation part open source right now - the video generation scripts are not included. Same with the demo; comics only right now.
Hmmm, thus far it draws the same character ok, but does not draw even basic story panel ideas for my explorer on a strange planet comic, so like 'leaving ship', 'discovering ancient ruins', just give the same character in same pose with slightly different backgrounds.
honestly asking, but has anyone ever made one the huggingface demos make run? I mean of any ai animation tool made by any bytedance group. Up until now no colab or demo page ever worked same in this case.
I get nothing but errors on the demo and when I did get a generation, it was garbage and couldn't even download result from the hugging face site. Until I see this working locally, I can't take Story diffusion seriously. Something seems off🤔
hey Samson i love your videos... I was wanting to ask you a couple of questions if you have time to answer...1 What is the AI video generator out there that anyone can just use that has the longest videos? and 2 When do you think Sora will be ready for people to use? like how long will it be before just anybody to use it ? i know you may not know the answer to #2 but maybe you can give me your best educated guess at it ... thankyou so much for making these videos for people like myself
I'd say that the answer will depend on when you return to this video to look at the replies :) At the incredibly fast pace of current development, the answer will vary from _week_ to _week_ ...
@@EnderViBrittania I love technology. I've been using computers for almost 35 years, I've done computer/internet programming for several years. This is different: typing "write me a symphony in the style of Beethoven" and the machine pooping out symphonies, where the person typing cannot play a note, is the very end of human creativity which will turn humans into cucumbers. Think what you wish.
@@9thebear Typing a few words and "creating" "your own" full length movie _just as_ you imagine it? Where is the creative process? Where is the artist in all that? This is why I am saying AI will destroy humanity by depriving it of creativity, turning humans into cucumbers.
why is so hard for AI to keep the person same through out the whole video ? It's like it's not working on same method as other video creation softwares
Impressive stuff! But it also frustrates me that some new A.I. papers present all their examples in a square video. No one wants square video's, they should give examples in 16:9. I'm hoping for a platform that combines different features, like adding the audio (speech and singing) from a file to a video. Everything is so scattered right now, technologies need to merge into a basic set of video and audio features. Hard to say what we'll all be able to generate in one year's time.
I'm waiting for the day when we can scan in the pages of Japanese manga and also load in a episode or two of the related anime series have ai figure out the characters the voices and the music and create all new episodes based on the manga.
Embarking on a journey of storytelling and video exploration, and VideoGPT seamlessly integrated into my process, amplifying the professionalism of my content.
... all of the physics and context consistency issues gets solved by an inevitable linkup between something like unreal engine and AI ... to a large degree these crude iterations aren't worth getting into too deeply because a year from now they'll be seen as something like mid-journey 1 ...
It saddens me to see us entering into an era of low quality ai generated content spam that may drown out the genuinely creative works of real people. It's like the fast food of the creative industry, cheap and low quality. I hope monetized content creation platforms such as TH-cam and others add an option to filter out all ai generated content from our feeds.
Samson, this is what I made with 2 ai generators after learning about them on your channel th-cam.com/video/SCq2VkSXAeY/w-d-xo.html a full ai video on my original song, Beware Fallen Angels. I had props to make my original song the usual way, but after seeing your talking about haiper and a few others, I got inspired to try to do my story line in my lyrics with these tools. And I had never used any ai prior to that, even chats, never tried. So had a quick crash course, and found there were lots of limits and lots of will exerting I had to do with ai. I found not one alone could deliver, and not all together could deliver, as I had to fill in the 'gaps' with derivative work myself, having to 'trick' the ai to give me one of what I wanted in a scene when they would simply fail, then do the old fashioned 3d props and digital art to then create the needed scene myself. As well as had to use creative cropping and such and add digital pieces to cover problems and transform to my vision. Overall, I created hundreds of 4 seconds clips, and like making a 'real' film, many at the end, ended up on the 'cutting room floor' not used, but a necessary part of the process. I appreciate your efforts exposing these different ai tools, I find your opinions interesting, and appreciate your efforts to draw attention to various new ones, which I believe, some will just be fake fronts that appear to be real ai, but just go to the length to make imitation sora videos, in order to draw in the public for whatever other reasons, those perhaps revealing themselves by never fleshing out their tools. Any rate, I was able to make a complete story line all the same. I just want to add, in my case it isn't like an artist lost income from me because my budget as an indie musician does not exist to hire outside and so I always do my own work. And to me this is the value of ai videos, that it gives individuals unable to make use of professional talents, to have their own visions realized.
My heart is breaking for visual artists, genuinely. I used to wish for the advancement of technology, now I regret for what I wished. I'm super impressed with all of this, but genuinely, I say sincerely, this scared me and fills me with existential dread. It seems like we're gaining something amazing, but deep down I know we're losing something we won't get back. God help us all.
If you watch any tv show, you'll notice a scene is split up into 6 to 9 second clips between camera angles. Establishing shots can be up to 15 seconds. Check it out. Cue up your favorite show and start counting until it switches to another camera. With this in mind, it should be possible to replicate a scene one segment at a time. Then you just have to splice them together using premiere or its open source equivalent.
If it's consistent. Comyfui can with a ton of work. But quality is not great with SDV
long ago, around 2007, I remember watching something on Discovery channel, a special. The person who was being filmed said these words.
In the future, actors will no longer be used because movies will be created without them.
We're just about there.
Good or bad, it has its applications.
in 1992 when I was in gr 7 I made a speech about this in school :D
@@beofonemind it seems we're all making the connection.
the tech would be great to pre-vis shots before an actual shoot or better yet render the whole movie to perfect the flow before shooting begins
This is the first time I seen multiple character interact with each other
Yeah.. we gonna need that for smut vids
I was thinking of using it to make realistic fight scenes for my superhero comics , but you do you 😄😄
@@Mightyflynn77 I was thinking of using it to make realistic fight scenes for my superhero comics , but you do you 😀😀
@dwainmorris7854 will this cartoon feature tentacled creatures? I would buy that
I still need to see it generate something. Hoping the gentleman in the vid gives it a shot. I don't know where to start.
I make music in a band, and I create music videos for our songs on the platforms. I have used AI as a collaborator on a song, and I use scenes created by AI as best I can, with what I can find. Your channel keeps me on the cutting edge of Music and Video Creation.
I am very excited at the idea of me putting myself and my other band members in some AI videos. Thank you for this channel.
Hello can we collaborate ? Me too i have the same idea of u
@@_GOWI_ What kind of music do you do?
@@ricktheexplorer kppp
@@ricktheexplorer kpop
@@_GOWI_ If you have any songs on TH-cam, please send me a link. After that, I work in the studio with TH-camrs: they put down a acapella video, or My lead singer playing synth and singing, then I add the rhythm section. Once the song is done, we start to put together the music video with AI.
Send me a TH-cam link to one of your songs please?
This is extremely important. I don't want companies making up rules about what can and can't be produced with their AI models.
- [00:00](th-cam.com/video/jZWRENqCl6I/w-d-xo.html) 🎥 Overview of Story Diffusion Model
- Introduces "Story Diffusion," an open-source AI video model.
- Highlights the model's ability to maintain character consistency and adhere to reality and physics in videos.
- Discusses the model's advancement in character consistency, including facial, clothing, and body type consistency.
- [01:03](th-cam.com/video/jZWRENqCl6I/w-d-xo.html) 🖼️ Creation of AI Comics using Story Diffusion
- Demonstrates the generation of AI Comics with consistent characters and movement.
- Explains the process of creating sequential images and animating them for comic generation.
- Discusses the potential of AI Comics production enabled by Story Diffusion.
- [02:56](th-cam.com/video/jZWRENqCl6I/w-d-xo.html) 🤔 Comparison with Existing AI Video Generators
- Contrasts the capabilities of Story Diffusion with other AI video generators.
- Highlights the realism, consistency, and fluidity achieved by Story Diffusion.
- Discusses limitations and minor imperfections observed in Story Diffusion's output.
- [07:11](th-cam.com/video/jZWRENqCl6I/w-d-xo.html) 💻 Accessibility and Usage of Story Diffusion
- Addresses the accessibility of Story Diffusion, available as open-source but lacking a usable interface.
- Describes options for accessing Story Diffusion, including downloading/installing or running online on a Cloud Server.
- Discusses the significance of Story Diffusion's multiple character inclusion feature for scene consistency.
- [09:45](th-cam.com/video/jZWRENqCl6I/w-d-xo.html) 🧠 Mechanisms Behind Story Diffusion Model
- Explains the technical mechanisms employed by Story Diffusion, including consistent self-attention and story splitting.
- Describes how consistent self-attention ensures visual coherence across generated images.
- Discusses the process of story splitting and motion prediction in creating narrative sequences.
- [14:48](th-cam.com/video/jZWRENqCl6I/w-d-xo.html) 📈 Evolution of AI Video Generation
- Reflects on the advancements in AI video generation enabled by Story Diffusion.
- Emphasizes the potential for creating realistic and cohesive scenes with AI-generated videos.
- Encourages exploration of Story Diffusion's capabilities and contributions to the field of AI video production.
Thank you sir! Very helpful!!
@@OpenEscapeGaming you are very welcome.
7:28 You got me at multiple consistent characters in scenes. It’s really important for storytelling to be able to have multiple consistent characters interacting with each other.
Oh yes. Definitely
An open-source neural network similar to TOR?
What if we make an open-source neural network that would work as a swarm, in the spirit of the Tor network or torrents, crypto? So that all computers would be part of its "consciousness", and not some servers?
When I see glitches such as the ball going through the metal rim I can't help but to think what this technology can do someday for special effects that are closer to realism than CGI while also being simpler to accomplish.
love the channel but going to be a little critical... the problem with this is that its not open source, you can't make videos,
- Source code of Video Generation Model not even available
- Pretrained weight of Video Generation Model not even available
You need to be skeptical what you cant test yourself
Machine learning professionals from Alibaba and Bytedance companies and tiktok etc all have made huge claims but have never made it open source. Also the comic book strips frankly suck and where considered bad 3 years ago. Since you could just use controlnet, fine-tuned models and SD1.5
After a "testrun" I had just dozzen of errors, cant even get the huggingface app to run just producing errors since more than 2h. So, forget it.
I have tried to make this work using the demo, It never does though. I am not sure about what I am doing wrong. I always get an error.
Same
Yea it’s broken for everyone on the demo face I think
same here
It has always worked for me. Click on one of the examples at the bottom and it will pre-fill the fields, then click generate. It will work. Then, you can edit the fields to fit your needs. I have no idea how to make a movie with enough frames for fluid movement, however.
@@redleader7988 yeah i've been chipping away at it and exploring it locally and the functionality to generate video doesn't seem to be present, unless i'm missing something
8:19 The ability to use this AI tool for creating comics (and graphic novels) would be amazing.
Or animating existing comics… I can see it already, no need for Hollywood adaptations when you can simply plug in a comic and have it create a authentic 1-1 movie based on any book or comic / manga…
@@sssurrealI would love this
there is already 1 comic based AI out there, though I don't remember it being too good.
@@SoullessAIMusic yes I remember, it's on huggingface, not that good though
@@SoullessAIMusic good for suggestive storyboards though?
For what i read in their paper, their attention method work through batch generation, video length is limited by memory and that is probably why resolution is so "low".
I tried signing up, Story Diffusion only creates comic style still images, it does not create videos. This is spam.
I've tried it today and I would just say it's ok but far from what the demo has shown. It still suffers from the typical Stable Diffusion problems which is prompt understanding and complex scenes. In its current state, I don't think it is useable.
It is an improvement from what we had. Hopefully future updates could fix some of the obvious issues.
Thanks so much for this quick feedback. You saved us a lot!
how did you try it? users on reddit say there is no repo no weights, maybe there will no be release
@@artemedutech There is a huggingface demo you can try.
can you plz explain how to continue to further shots with same scene, consistent character, and camera moves when we are creating a movie? because all the view gets changed for next shot, and sometimes character as well. Every expert on youtube is showing just 1 shot. Please let us learn to shoot a scene of 6 shots, 3 minutes long, showing 7 friends. Refer to any sample scene from series FRIENDS. Many thanks
I hope they add a landscape 16:9 aspect ratio, so we can create clips for AI movies.
I haven't tried this tool yet, but I'm thinking about using keywords like "Wide shot", "Long shot", "Panoramic", maybe even "Fisheye" - then cropping it, then AI upscaling.
*have AI generate, not _create_
@@Zareh_Abrahamian It still fits within the definition of ‘create’. It’s bringing to life content that does not exist anywhere else on the internet.
@@JohnSmith-gt3be "Created" by the machine, having stolen centuries long blood, sweat and tears hard work of creative people, making billions for AI companies, rendering all artists dead or living irrelevant. That is some life and content we can suck on!
@@Zareh_Abrahamian I think it’s an amazing idea. When the horsing industry was destroyed by the automobile industry the world became a much better place. I guess you’re just on the side of the horses today.
Absolutely extraordinary. AI video is developing faster than we can keep up with it.
man i love your channel and your videos, thanks so much for all this updates and news, you rock!
thanks!
I can’t get past it asking me to: When setting the reference image, please add the trigger word ’img’ in the Description Field.
WTP.
Thanks you. This is really cool. Now all we need is a tutorial on how to run this locally.
no idea how to run it. any idea?
And how to actually create videos with it.
Hmm... I can't get it to work. It just keeps saying error.
I'm trying to test a video in the github page, but I keep getting an "ERROR" in the render section a few seconds after... Although this has happened without an image reference, I'm not sure if I fully understand what they mean by "𝘪𝘧 𝘺𝘰𝘶 𝘢𝘥𝘥 𝘵𝘩𝘦 𝘙𝘦𝘧-𝘐𝘮𝘢𝘨𝘦, 𝘮𝘢𝘬𝘪𝘯𝘨 𝘴𝘶𝘳𝘦 𝘵𝘰 𝘧𝘰𝘭𝘭𝘰𝘸 𝘵𝘩𝘦 𝘤𝘭𝘢𝘴𝘴 𝘸𝘰𝘳𝘥 𝘺𝘰𝘶 𝘸𝘢𝘯𝘵 𝘵𝘰 𝘤𝘶𝘴𝘵𝘰𝘮𝘪𝘻𝘦 𝘸𝘪𝘵𝘩 𝘵𝘩𝘦 𝘵𝘳𝘪𝘨𝘨𝘦𝘳 𝘸𝘰𝘳𝘥: 𝘪𝘮𝘨, 𝘴𝘶𝘤𝘩 𝘢𝘴: 𝘮𝘢𝘯 𝘪𝘮𝘨 𝘰𝘳 𝘸𝘰𝘮𝘢𝘯 𝘪𝘮𝘨 𝘰𝘳 𝘨𝘪𝘳𝘭 𝘪𝘮𝘨" I suppose you have your image subject and just put the letters img right after that word in the prompt? Any idea how to make the thing work? Tried in 2 browsers as well, and still getting that Error in red....
Open Source is still very underrated but things like these demonstrate the potential in AI.
Is there any video generation feature released yet? I couldn't find an option to generate videos in the github reference
tbh my hopes are still on LTX. It is an awesome platform already but certainly has to learn alot.
Time will tell how well it´ll turn out, but I´m rather positive at moment.
Can I create consistent characters from a picture?, in order to make a character of myself for example
what if I want to use my own photos and animate them with a prompt?
I’m here for an open source Ai 👏🏾 but any indication what it’s trained on?
I can't testify to the video stuff. However, I tried the "comic book" (think graphic novel) stuff on 2024-MAY-09. Here's the problem. The system ignores your "story beat" prompts and just generates five different versions of your "character" prompt. Now, that's not a bad thing because you can then take that into Photoshop and move everything around into place. However, this is not a one-stop shop for graphic novel generation. Second, if you say "speaking", the character is not speaking. If you say "knock on door", the photo looks like a selfie. If you choose "photographic" you'll still get comic book style. Also, it does not apply the images to a "page". So if you ask for a 1024x512 page, you'll get five 1024x512 panels and you'll have to build your own page in Photoshop.
Unlike a Stable Diffusion site that will give you four images per press of the button, this site will give you the same quantity of images but make sure the uniforms are consistent. It seems as if the system uses text-to-image to make a baseline and then runs that through an image-to-image to get a final result.
So far, there are a lot of promises but very few victories. We'll see.
Dude how do I install locally? Would you consider doing a setup tutorial please?
I have some stuff but it's not stable diffusion or story diffusion as my graphics card popped. 😢 But I do have easy instructions on installing local models with any hardware ect.
Real question: Do you have access to Sora or Story Diffusion? If not, how are you drawing any of these conclusions?
I think they are just speculative conclusions.
How well does it perform with dynamic action like people fighting or playing sports etc?
That sounds like violence...
@@snickle1980 could be! Sports can be violent.
What are the vram requirements though?
GPU >20 GB
@@soku3889 aint no normal consumer have that rip
Is there a valid reason why the links of the sites featured in the video are not included in all the hefty amount of links provided?
I don't understand why AI content creators neglect to include the mentioned links and/or timestamps in their videos. Not only does it make the video more useful, but I believe it is essential for creating a valuable content.
looks great and will only improve over time cant wait to see what the future brings!
This looks great; AI creating video content is a trend for the future.
Video related code is not yet released. Only makes images so far.
I try it on Hugging face, on white tiger with armor, and there is no consistency of the armor, what do I do wrong ? this is far from this video
Leon Ding leaked a lot of Google AI technology to the P R C
no way to download and take advantage of this model on iPad's DrawThings app?
I tested the demo page, and i have only images when generating.... not a video, have you got it to work?
13:01 that's not a scene in England...it's an astoundingly accurate rendition of a view probably across Princes Street up towards the mound in Edinburgh, Scotland. I've seen that view often. Details like the unusual lamp and recognisable buildings are astoundingly accurate, actually. I nearly fell off my seat when I saw it!
Good grief the bus is even the 900 city link Edinburgh to Glasgow bus and the text on it is legible!
That's because it's just an illegal scape of real, likely copyrighted video
@robertruffo2134 nonsense. Try learning how this stuff works. There is not a compression algorithm known to man that could compress the volume of information in the training data down to the size of the download. The model learns by training on data not dissimilarly to how a human does. It doesn't have a database with the information in it. Only weights and biases representing what it learnt from studying the training data.
When people say things like you say:
1. I feel for them. Sucks that this affects livelihoods.
2. But it's clear you don't actually understand how this works.
@@iaincampbell4422 Dude, I know how this works. I even worked on earlier versions of this kind of system. If you think they learn "the way a human does" you have drunk many liters of tech hype Kool-Aid. Also, yes they do "keep" training data. Have a look at some recent lawsuits.
It seems like this isn't actually open source yet (the video generating part)... Am I correct in this understanding?
This definitely seems to have only the comic generation part open source right now - the video generation scripts are not included. Same with the demo; comics only right now.
Hmmm, thus far it draws the same character ok, but does not draw even basic story panel ideas for my explorer on a strange planet comic, so like 'leaving ship', 'discovering ancient ruins', just give the same character in same pose with slightly different backgrounds.
That's because, like all AI, it's likely mostly a scam
honestly asking, but has anyone ever made one the huggingface demos make run? I mean of any ai animation tool made by any bytedance group. Up until now no colab or demo page ever worked same in this case.
I get nothing but errors on the demo and when I did get a generation, it was garbage and couldn't even download result from the hugging face site. Until I see this working locally, I can't take Story diffusion seriously. Something seems off🤔
Update: Hugging face demo seems to be working better now. Servers were probably getting hammered.
what kind of computer do you need to run this at home?
GPU >20 GB
hey Samson i love your videos... I was wanting to ask you a couple of questions if you have time to answer...1 What is the AI video generator out there that anyone can just use that has the longest videos? and 2 When do you think Sora will be ready for people to use? like how long will it be before just anybody to use it ? i know you may not know the answer to #2 but maybe you can give me your best educated guess at it ... thankyou so much for making these videos for people like myself
I'd say that the answer will depend on when you return to this video to look at the replies :) At the incredibly fast pace of current development, the answer will vary from _week_ to _week_ ...
@@GwynethLlewelyn thank you I appreciate that
Is there just a paper? How can we run the code?
Opensource, sounds more than good. Will give it a testrun now
How did it go?
@@ArtCype They just released a comic strip generator, but no videos in the script.
Woah. Exciting times. Things are improving so fast now. I can handle some of the glitch if i get more continuity and length 👍
So exciting to witness humanity having itself robbed of creativity after 40000 years of making art...
@@Zareh_Abrahamian Ok, luddite.
@@EnderViBrittania I love technology. I've been using computers for almost 35 years, I've done computer/internet programming for several years. This is different: typing "write me a symphony in the style of Beethoven" and the machine pooping out symphonies, where the person typing cannot play a note, is the very end of human creativity which will turn humans into cucumbers. Think what you wish.
@@Zareh_AbrahamianAnd now you can create your own full length movie just as you imagine it, isn’t that cool?
@@9thebear Typing a few words and "creating" "your own" full length movie _just as_ you imagine it? Where is the creative process? Where is the artist in all that? This is why I am saying AI will destroy humanity by depriving it of creativity, turning humans into cucumbers.
2:24 a glitch spotted on her eyes!
it dosnt work
why is so hard for AI to keep the person same through out the whole video ? It's like it's not working on same method as other video creation softwares
Anyone know how to install this locally?
Maybe I'm missing something. But all the stuff I've seen from Sora recently and in this video looks way better to me.
Tried several Times with severel Options but Says ERROR!
What is the software?
Anyone knows how to get a consistent background using Gradio or ComfyUI ?
I don’t think Sora AI is out to the public yet but we will see it’s great to have good competition though
When can I use this?
I’ll have to try this one. Thank you for the videos ❤️
it dosnt work
for the critics: last year we were a Will Smith eating spaghetti
Impressive stuff! But it also frustrates me that some new A.I. papers present all their examples in a square video. No one wants square video's, they should give examples in 16:9. I'm hoping for a platform that combines different features, like adding the audio (speech and singing) from a file to a video. Everything is so scattered right now, technologies need to merge into a basic set of video and audio features. Hard to say what we'll all be able to generate in one year's time.
Couldn't agree more!
I only wish that the term *create* when using AI would be replaced by *having AI generate* whatever is labelled AI "art".
Getting better and better. New storyboard less, actor less movies soon on the way.
I can't figure out how to install it, lol. Help Please....
I'm waiting for the day when we can scan in the pages of Japanese manga and also load in a episode or two of the related anime series have ai figure out the characters the voices and the music and create all new episodes based on the manga.
Embarking on a journey of storytelling and video exploration, and VideoGPT seamlessly integrated into my process, amplifying the professionalism of my content.
Apakah wajah yang dibuat Sora mengambil dari real person?
Wow this amazing! I have more faith independent developers more than Sora.
In the demo clip with the black girl and headphones her necklace disappears as she moves.
There's still hope (however tiny) that this monster will never wake.
An Advertising company can literally spend pennies with this breakthrough
I don´t know how to install it in order to use it 😢. Someone who can help me?
Maybe nobody is using it, I really wanna install too (I have 4090, and I read they used a A10 24 gb for running videos)
... all of the physics and context consistency issues gets solved by an inevitable linkup between something like unreal engine and AI ... to a large degree these crude iterations aren't worth getting into too deeply because a year from now they'll be seen as something like mid-journey 1 ...
I KNEW that wolves spawned at different spots in the woods!
There had to be a reason they put it in so many games!
Until they add mouths that move with voice (either within or addon or after effect), there won't be movies as we understand them.
6:09 Sora still looks at least one generation farther .... but beeing open Source makes Story Diffusion very very interesting!
It doesn't work
This seems to be the best open-source model thus far.
Open Sora isn't much better and requires double the VRAM.
Thank you for this video!! Very useful to me :) You explain very well.
Does not work even test demo program does not work needs more work, even test examples do not work.
The Demo seems to be cooked at this point, no matter what I do it results in an error.
Ok so it's not just me then.
Guys plz inform about tool which r coming soon, not like Sora and LTX, which God only knows when they gona released
can you make locall install turorial?
Is there censorship of fight scenes and figures holding weapons
DOES THE FREE CREDITS TO GENERATE VIDEOS LIKE LEONARDO AI - REFRESH EVERYDAY?
make a installation guide please
HIIIIIII very good video, but CAN you do one video for install it and use it? Please!
It saddens me to see us entering into an era of low quality ai generated content spam that may drown out the genuinely creative works of real people. It's like the fast food of the creative industry, cheap and low quality. I hope monetized content creation platforms such as TH-cam and others add an option to filter out all ai generated content from our feeds.
Just like an AIBlocker addon?
Doesnt work… 😢
I just need something that will simply take a an existing video file or image and add existing music to it for me
You use video editing software for that. For a task as simple as this, there are probably free options on the web that can run in your browser.
Samson, this is what I made with 2 ai generators after learning about them on your channel th-cam.com/video/SCq2VkSXAeY/w-d-xo.html a full ai video on my original song, Beware Fallen Angels. I had props to make my original song the usual way, but after seeing your talking about haiper and a few others, I got inspired to try to do my story line in my lyrics with these tools. And I had never used any ai prior to that, even chats, never tried. So had a quick crash course, and found there were lots of limits and lots of will exerting I had to do with ai. I found not one alone could deliver, and not all together could deliver, as I had to fill in the 'gaps' with derivative work myself, having to 'trick' the ai to give me one of what I wanted in a scene when they would simply fail, then do the old fashioned 3d props and digital art to then create the needed scene myself. As well as had to use creative cropping and such and add digital pieces to cover problems and transform to my vision. Overall, I created hundreds of 4 seconds clips, and like making a 'real' film, many at the end, ended up on the 'cutting room floor' not used, but a necessary part of the process. I appreciate your efforts exposing these different ai tools, I find your opinions interesting, and appreciate your efforts to draw attention to various new ones, which I believe, some will just be fake fronts that appear to be real ai, but just go to the length to make imitation sora videos, in order to draw in the public for whatever other reasons, those perhaps revealing themselves by never fleshing out their tools. Any rate, I was able to make a complete story line all the same. I just want to add, in my case it isn't like an artist lost income from me because my budget as an indie musician does not exist to hire outside and so I always do my own work. And to me this is the value of ai videos, that it gives individuals unable to make use of professional talents, to have their own visions realized.
So is it available for use now or??
this would be much cooler if it actually worked...
I desperately hope it will fail...
My heart is breaking for visual artists, genuinely. I used to wish for the advancement of technology, now I regret for what I wished. I'm super impressed with all of this, but genuinely, I say sincerely, this scared me and fills me with existential dread. It seems like we're gaining something amazing, but deep down I know we're losing something we won't get back. God help us all.
what about music?
Google VEO Is the king
2:23 that isn't very consistent LOL eye popping out.
This is like Stable Diffusion 2 trying to do videos. It's literally no more impressive than Sora...