crazy times, every day there is a new advancement that shakes up any current workflow I have as a creator, but Gen3 IMG>Video is hitting all the right notes, for most stuff, very exciting time.
And to think this is just in alpha. I bet within a year it will really start to scare people, especially if/when you can run it locally without limitations on the prompts.
Appreciate it!! Caught your video last night as well! Fantastic job on it! LOVE the idea of using a Gen3 output as a backplate! Can't help but think of all the wild things that will open up once camera control and motion brush drop as well!
Stoked to hear that it is helpful for you! I mean, as warned in the video, ChatGPT can get stupid from time to time-- and sometimes you'll need to modify the output prompt- but, always easier to rewrite than stare at that blinking cursor!
Wow, Gen-3 Image to Video is a total game-changer! I love how easily it generates videos from images, but I'm also a bit frustrated with the limitations - like the 16:9 aspect ratio and weird character movements. Still, it's amazing to see how far AI video generation has come. I'm excited to experiment with it and see what kind of creative projects I can make. Thanks for the review, Theoretically Media - you always keep me up-to-date on the latest tech
Agreed! like you I was frustrated by the 16:9 limits at first, but it’s not so bad with their easy crop. I think we’re going to have problems with movement for a while…
Thanks so much!! Love your videos as well! And hey, the guitars in my studio aren't just for display, so if Kev wants to come over with his drum set.... =)
I always think: can someone put me into cryosleep and wake me up in two or three years time. Also: Imagine video inpainting to adjust tiny glitches. Thanks for keeping us up-to-date Tim!
Love to see you compaire how good video generators do with different scenes like Action, Dancing and eating scenes. Plus maybe give us tips on how to achieve the best results as well 🙏🏼
I like it, but at $15/mo for a dozen images doesn't work and I don't have a reason to create more. At this point, AI video will remain in the hobby category.
Totally understand. You might want to check out Kling, as I mentioned toward the end of the video, they do have a sale right now for $5/month. Easily the cheapest video generator around right now.
Second the OP: go with Kling, if you want Runway’s $15/ month plan, otherwise, you need the unlimited RunwayML at $95/ month I created a video with Runway and Udio after 20 hours of work, which is a 5 minute short film about 1984. Like the OP says, everything you’re seeing in most videos is cherry picked. It still takes an excruciatingly long time to get what you want
I'm testing Runway against Kling. So far, very impressed with Runway! Amazed at how fast it rendered and the lip sync feature has been pretty spot on..
I'm going to try your GPT today. I only have the standard plan on Runway and feel like it eats up credits for underwhelming generations. I've had much better luck with Luma so far when giving them the same prompts and images. I'm running out of money to sub to every tool so Kling is gonna have to wait. 🤣
Please do! Keep an eye on that GPT. There are times when it works really well, others when it gets REALLY stupid. I think we’re all at the whim of OpenAI’s electric bill that month. High suggestion to try Kling out (6 gens free a day) and run a comparison with your Luma videos. Both are really good. Just see which you like better.
@@TheoreticallyMedia Hmm, I did not know of the 6 free gens a day... I will definitely give it a whirl. I will also blame you if it turns into yet another subscription, hah.
They're saying they're going to release 3D before video. Which-- I'm actually kind of more excited about? MJ's stance is not to release video until it can do something better than anything that's currently out. I think 3D is their sauce to get there.
@@mfmageiwatch LOL 6.1 is supposed to fix hands in the distance. I just rerolled a 6.0 gen with hands in the air. 2 of the 5 hands have 6 fingers and 1 has no thumb LOL. The 6.0 version had all 5 finger hands. Oh Midjourney, how we love to give you our money.
@@TheoreticallyMediais it fake 3D or actual 3D model? Cause a pseudo 2.5d 3D or just a picture that spins around is quite boring and useless vs video and pics.
@@HORRORinWV uhhh, idk. On Luma you can add a beginning frame and an end frame in image form. I’ve used this to make psychedelic hallucination replications on my second channel. This technique was used for all those ai meme transition videos that have been going around lately
Yup, keyframes. I'm not sure Runway will go that direction, but they do have Camera Control and Motion Brush in their back pocket. And I'm really excited to see how those will be implemented in!
Gen-3 has lot of moving elements and dynamics, while luma and Kling have fluid camera motion and character consitency. Creativity goes 100% to Gen-3. using all 3 of them accordingly for different shots will have Bonkers result. Job well done.
I'm constantly getting a slow zoom or some effect that appears as if it's on top of the image (like a filter effect) and not too the image itself. Any strategies to prevent this? Btw, I really appreciate the free guide on gen3 prompts, I'm referencing it a lot but still get the slow zoom effect with very little motion elsewhere.
It's all still incredibly rough looking. We have a looong road ahead of us. The timeline isn't as fast as I once thought it would be. Definitely feels the hype is dying down now
It is-- Have you seen that Hype Cycle Chart? (Gartner hype cycle)-- currently we're past the "Peak of inflated expectations" and solidly in the "Trough of disillusionment" -- well, some of us are. Next up (and where I am) is the "Slope of enlightenment" where you understand what these things can do, and start optimizing and building with them. It might not be as "wow!" now, but that's fine-- it'll keep getting better. In the meantime, just keep experimenting and creating. The masses will slowly circle back.
That'll for sure be addressed in an update at some point. For now, you could always generate and re-crop in post. But, yeah, that's a lot of extra work. And if you're super picky like I am (and I presume you are, and that's a good thing!) it'll be A LOT of extra work!
Thanks for this! I got some good results after repeating back to it my subject. For example, I had it place my cat on a regal throne. At first, it would completely morph my cat, but after having the cat described again, it didn’t morph it Game changing feature will definitely change my workflow, and get me more focused on starting off with a good storyboard for future films using images from MidJourney or Google’s Imagen3
Here’s my original prompt: The camera zooms out from the close-up of the cat to reveal her sitting regally on a plush, throne-like couch in a whimsical living room. The room is filled with playful toys, colorful pillows, and soft morning sunlight streaming through large windows, creating a cozy, enchanting atmosphere. Here was my final one, will link to the before/ after video too: The camera zooms out from the close-up of the grey British Shorthair cat with striking amber eyes and a daisy charm collar, revealing her sitting regally on a grand, ornate throne upholstered in plush red velvet with gold accents, in a lavish throne room filled with gold, diamonds, and treasure. The room is softly lit by the morning sun, emphasizing her queenly presence.
Haha, that's a good pitch for Netflix! "So, what's the Man in the Blue Business Suit do in the show? "Well, he really just kind of walks around the city and usually answers his phone..." "We'll TAKE IT!"
@@TheoreticallyMedia Dude... sometimes he answers two phones.... sometimes he just take it out and looks at it... it speaks to people on so many different levels... Tubi have already put in an offer.
Runway has this to say currently: Due to heavy demand, generations on Free Plans are temporarily unavailable. Paid Plans are unaffected. (See Plan options). We thank you for your patience.
I'm new to all this. Obviously the still images are the raw material that's fed into these video generators, but what material are they referencing to get the movements? If someone were to ask why the guy in the video (3:34) has a beard and shades, we'd obviously just say that's what the source image had. But if we were asked why he's walking in that particular _way,_ or at that particular _speed,_ or tilting his head at such-and-such _moment,_ at that particular _angle,_ for that particular _duration,_ where is all that coming from? A particular video it's mimicking? Or maybe an amalgamation of many videos? If one video in the generator's brain/dataset is of a person walking normally, and another video in there is of someone walking weirdly, how does it choose which to mimic when it generates? And what keeps it from jumping from one walking style to the other randomly?
Could you please do a video about RunwayML’s custom model? I’d love to have consistent characters throughout. Right now, it’s hit or miss. I use the same seed, repeat the same prompt for the person, but it doesn’t always work
Yeah, I've got some stuff coming up on that soon. I'm not sure the custom model is currently connected to Gen-3 on Runway-- need to test that, but even then-- that will only apply to Text to Video. Currently, I think the best way is to train up an Image Generator and then use that for consistency in Image to Video. Will have a video up on that soon.
Always informative, always makes me chuckle - double-jointed wig wearing synth synth in a rip off over-priced Williams -Sonoma showroom on fire. Tim, you are a rock star. ( But we knew that ). Thanks man!
@@TheoreticallyMediaPRICE COMPARISON: Runway Gen3 - max 10 sec video - paying yearly is 20% cheaper than below pricing - 10 credits gets you 1 second of video $15/m = 62.5 sec/month $35/m = 225 sec/month $95/m - unlimited Kling - max 5 sec video - paying quarterly, half-year or yearly is NOT cheaper than monthly pricing - 10 credits gets you 5 seconds of video - ALL THESE PRICES WILL DOUBLE ON JULY 5, 2024: FREE = 330 sec/day $5/m = 330 sec/day + additional 330 sec/month $18.50/m = 330 sec/day + 1500 sec/month $46/m = 330 sec/day + 4000 sec/month NO UNLIMITED PLAN yes, Runway is EXPENSIVE, unless you go unlimited.
Gen3 looks good, the hands are funny. You would think by now it would be worked out but this shows AI really doesn't see the image and is guessing . Orbit and characters turning is a tough one for ai, funny though sometimes it just does it on it's own, hopefully someone works that out.
I think they're wisely (for now) keeping orbits to a minimum with characters, as that was something that was a real issue with Gen-2. They minute you start changing the angle, the character's face would totally change. That said, Kling and Luma seem to do it well, so maybe Runway is just keeping it tight until they model is 100% there.
HI where can I email you a video I made using MJ - then GEN3? I also tried Cling, but I have to say that Gen3 was the best. Dream Machine was the weakest. I should have waited a little before I spent the money! LOL
I need to make a kickstarter video for a boardgame, which one would you use. Im fine with $15 a month but dont know how many videos that gets me or which program is better over all.
The two phones thing is so me. I live in London and do most business in the US so I'm constantly toggling between a us and a UK phone. Wish I had his build though.....
Haha, time to hit the gym! Maybe start by deadlifting phones? Oh, and totally- the two phone thing is kind of common here too...although, I'm an idiot and just keep my one phone, so my contacts are a total mess and I have no work/life balance!
Great stuff Tim, as always I hope for those of us odd balls there's ways to keep making it weird! Also left you a message on Discord, you have a spammer on the channel fyi. :)
I KNOW exactly what you mean when you say how STUPID CHATGPT is, got a real hot tip for that one, so u know how u can scroll in the window, now middle mouse click and point down, that will make the window think ur interacting with it constantly thus creating better output, sounds stupid I know, I build custom GPTS all the time and experience the stupid everyday because i am to lazy to wait it out, but every time i do stay there and scroll lock it works 86% of the time, adhering to custom script requirements thus fixing that issue as well as how good the model its self can give you an answer on any given day(aka 86% of the time) Glad I Illuminated that for u Hope it helps🚀🚀💪💪
Does it make consitent models over and over? I find that to be the most difficult thing to get Ai to do is use the same female model throughout the scenes I need to generate.
Most of them not fully yet. There are some tricks and workarounds to make it happen...but, yeah-- still kind of not fully there. Vidu was the only one that I've seen that has it built in as a feature. It's close? You can check out the look that I did here: th-cam.com/video/UB7mRFewxIA/w-d-xo.html
Wait, so 3:08, 4:16, 4:48 and 5:11 were all originally still images? It'd be nice to first see the original images before going straight to the video. As it is, I can't really judge how faithful the video generator is being to the starting image. 2:41 and 3:24 are nice demonstrations though.
So, Tim; or anyone who knows really; I see that Kling is finally available for all, which Tim mentioned too and that's good; but what happened to the whole "long video output up to 2 minutes feature" that we heard about before it became available outside of China? Kling seems to be just as limited as the others from what I can tell. How is it that we are still limited on these models to like a measly 10 second outputs or so at this point? Just seems crazy to me! If I'm missing something here, let me know as I'm really just waiting on any of these newer models that can output a decent 1 minute or more before I even want to bother with them, especially if they are charging money for these already. Let me know, thanks guys.
So with Kling you can EXTEND up to 2 (or maybe 3?) minutes. So, it generates the first 10 seconds, and then you can re-prompt for a second generation off that first one-- That said, I've found that details start to get really fuzzy after the first two extends.
@@TheoreticallyMedia Yeah, I saw that extending option, but exactly, it gets warped a lot when extending in any ai video model still. I'm more after like you could input something from the get go like, "create a one minute long video of people hiking in the mountains and change the scene every 5 seconds", or something like that. The only thing I know of so far that we can kinda do that with is Noisee in Discord, but the quality of the output of it is pretty rough, I'd say it's a "gen and a half" behind most of the others out now. Oh well, guess I'll just keep using Noisee for now until the newer models have the kind of input options I'm wanting, a waiting game as always, lol. Thanks for the response Tim, much appreciated 🙂👍
Been messing around with it. What I learned. This is not for very stylised images. GEN2 strangely enough is better for those. I was using graffiti style characters and they simply transformed into something totally different or just didn't do anything at all. Real world images seem to fair better.
I was going to mention that in the video, but it got cut for time. Same thing happened. Had a stylized character and it turned it into a real one. Kinda cool, but not what I was looking for
Bro, did you see the newest videos I put out on X? Mixing art flow trained images with Gen 3 animation and lip sync, instant cinematic video. Thanks for another video my friend
Is there any open source alternatives that you can run on your own machine though that's comparable in quality? Like T2I, I2I you have stable-diffusion, Fooocus, InvokeAI, ComfyUI, A1111 etc. for for front-ends. But the I2V, if it exist at all in those, isn't on the same level as Runway, Kling or Dream factory.
I'm still new to AI prompting but assimilate pretty well. didn't take long to learn that 'gluteus maximus and buxom' don't easily flag AI restrictions. Invisible/see through vs. naked. so have you tried a more logically elementary method for these situations? such as, Instead of a "guy walking off of a plank" something like 'guy walks to the edge of the wood platform then hops into the ocean to catch a cool dip'. or 'man fails to notice oily spot near foot slips and falls into the ocean'.. just like the more provocative ex: that causes the AI to retain the logic of what should be underneath clothing and so it depicts it, and in obedience to the prompt given it violates mature content settings. It's kind of like a child, get as close as possible to the look you want using familiar terminology and then trick it to design the imagery with alternate definitions and professional lingo. instead of burn up the nice looking living room, you could say. someone left a cigarette lit which slowly caused a trickling fire to engulf each item in the room one by one. the AI interprets an "accident" is ok to depict burning up a house vs. you simply telling it to burn up a house without a positive intent. find simple positive sneaky synonym words to depict complex graphic things to bypass the filter
Hedra is really great! Planning on making a video on them when they have their next big update! I kinda missed them the first time around-- I talked to them and got an early sneak peek, but then life got in the way, and by the time I had a moment to make a video, basically everyone else had covered it! Still, want to show them some love soon!
Gen-3 is still mostly wasting my time (and money) - way too many useless renders (bad anatomy, not following the prompt closely). Kling doesn't produce quite as many useless renders, but I'm still burning through credits without much to show for it - I definitely prefer the image quality and "acting" on Kling, though. Luma is the best bang for the buck for me - the image quality isn't as good as Kling, but I can get something usable without too many garbage renders. Maybe my video prompting skills needs to improve, I don't know...I'm using pretty good Midjourney images as the source.
Try out the GPT I made for video prompting. Although it's obviously optimized for Gen3, I was running the same prompts in Kling and Luma and getting similar results. Might help on the control front a bit. Although, often these Video Generators will make up their own mind about what they want to do, regardless of what we ask!
All it does is zoom in images with very little motion or nothing at all You need to try 30+ times to get a good result. You must have the unlimited plan otherwise you'll burn your credits fast with no results.
@@armondtanz 60 seconds or so per month for the standard plan i think. Gen-3 is really only worth it if you can afford the 100 bucks a month pro plan with unlimited generations.
Wait, he can't be Dr Doom. I'm not watching that. Why is it always him? He wasn't right for sherlock either. Mads MIkelsen or Nicola Coster-Walder would have been awesome.
Regardless how good all these pop up fly by night AI companies are, if you want to be the leader in your field, you're gonna have to make it more affordable for the average creator, if not, then you're not gonna last long in the game
I think Runway Gen-3 image to video is okay. Facial distortion is still a problem, but sometimes Runway get´s it right...sometimes 🙂Good I am on a unlimited plan, so I can practice and get no sleep 🙂
I gotta play with the prompting more. I've seen some pretty good emotional outputs since the video went up. It's always kind of the thing when you're doing "day 1" type videos. Sigh, life of TH-cam...
@@TheoreticallyMedia so true. Its a trial and error that gets better over time ss always. :) Great times ahead. Keep up you fantastic videos. Love them!
Yeeeeeaaaah. I was just talking about that today. That’s a big wish for video. I think we’ll see it soon enough. Maybe that’s what a revised motion brush might look like? Fingers crossed?
crazy times, every day there is a new advancement that shakes up any current workflow I have as a creator, but Gen3 IMG>Video is hitting all the right notes, for most stuff, very exciting time.
And to think this is just in alpha. I bet within a year it will really start to scare people, especially if/when you can run it locally without limitations on the prompts.
A solid year and we will have something that can actually feel like justice for the independent creative
Great video as always Tim!
Appreciate it!! Caught your video last night as well! Fantastic job on it! LOVE the idea of using a Gen3 output as a backplate! Can't help but think of all the wild things that will open up once camera control and motion brush drop as well!
Thanks for taking the time to make a Runway Gen-3 GPT! Getting the correct prompt counts a lot when it costs so many credits to generate a video!
Stoked to hear that it is helpful for you! I mean, as warned in the video, ChatGPT can get stupid from time to time-- and sometimes you'll need to modify the output prompt- but, always easier to rewrite than stare at that blinking cursor!
Wow, Gen-3 Image to Video is a total game-changer! I love how easily it generates videos from images, but I'm also a bit frustrated with the limitations - like the 16:9 aspect ratio and weird character movements. Still, it's amazing to see how far AI video generation has come. I'm excited to experiment with it and see what kind of creative projects I can make. Thanks for the review, Theoretically Media - you always keep me up-to-date on the latest tech
Agreed! like you I was frustrated by the 16:9 limits at first, but it’s not so bad with their easy crop. I think we’re going to have problems with movement for a while…
Always love these videos -- thanks for doing them!
Thanks so much!! Love your videos as well! And hey, the guitars in my studio aren't just for display, so if Kev wants to come over with his drum set.... =)
@@TheoreticallyMedia haha we should try a cross over at some point! we'll message you on that other platform
Yes: image to video! Every time I see one of your videos pop up, I get excited! Thanks for the comparisons, Tim.
I always think: can someone put me into cryosleep and wake me up in two or three years time. Also: Imagine video inpainting to adjust tiny glitches. Thanks for keeping us up-to-date Tim!
Man, two of three years is going to basically feel like 50. I know it's a cliche at this point: But things are moving fast!
Love to see you compaire how good video generators do with different scenes like Action, Dancing and eating scenes. Plus maybe give us tips on how to achieve the best results as well 🙏🏼
Try out the GPT I have linked in the comments. It seems to really help! (depending on how stupid ChatGPT is being at random times of the day!)
I like it, but at $15/mo for a dozen images doesn't work and I don't have a reason to create more. At this point, AI video will remain in the hobby category.
Totally understand. You might want to check out Kling, as I mentioned toward the end of the video, they do have a sale right now for $5/month. Easily the cheapest video generator around right now.
@@TheoreticallyMedia i will give it a try. I wasn't even interested in AI video until Gen3. It is fun watching the evolution.
Second the OP: go with Kling, if you want Runway’s $15/ month plan, otherwise, you need the unlimited RunwayML at $95/ month
I created a video with Runway and Udio after 20 hours of work, which is a 5 minute short film about 1984. Like the OP says, everything you’re seeing in most videos is cherry picked. It still takes an excruciatingly long time to get what you want
@@TheoreticallyMedia Can I ask you please whats happening with Kling? It doesn't work at all I can't even try it... its stuck at 100% of the load.
getting better each time
My mind fries out every time I start thinking about Gen-5!
I'm testing Runway against Kling. So far, very impressed with Runway! Amazed at how fast it rendered and the lip sync feature has been pretty spot on..
Happy 100K Yay!
Thank you! It's been such a wild ride! I think my award is finally showing up this week!
Thank you Tim. That was a good summary.
0:43 I did not notice the reflection, in fact, I did not notice even the RV, or the sea, or the mountains
LOL
Could we extend the video 4 hours? :)
Extended prompt: Remove ...... :)
Cant stop laughing
RunwayHub
Great video as always!
Appreciate it! Thank you so much!
I'm going to try your GPT today. I only have the standard plan on Runway and feel like it eats up credits for underwhelming generations. I've had much better luck with Luma so far when giving them the same prompts and images. I'm running out of money to sub to every tool so Kling is gonna have to wait. 🤣
Please do! Keep an eye on that GPT. There are times when it works really well, others when it gets REALLY stupid. I think we’re all at the whim of OpenAI’s electric bill that month.
High suggestion to try Kling out (6 gens free a day) and run a comparison with your Luma videos. Both are really good. Just see which you like better.
@@TheoreticallyMedia Hmm, I did not know of the 6 free gens a day... I will definitely give it a whirl. I will also blame you if it turns into yet another subscription, hah.
@@TheoreticallyMediaI tried to use kling but asked for chinese number.
Is there an international link?
Mid journey has been pretty quiet for awhile? I remember them talking about doing video a year ago?
They're saying they're going to release 3D before video. Which-- I'm actually kind of more excited about? MJ's stance is not to release video until it can do something better than anything that's currently out. I think 3D is their sauce to get there.
Mj has been quiet lately? Their newest version literally dropped today.
@@mfmageiwatch LOL 6.1 is supposed to fix hands in the distance. I just rerolled a 6.0 gen with hands in the air. 2 of the 5 hands have 6 fingers and 1 has no thumb LOL. The 6.0 version had all 5 finger hands. Oh Midjourney, how we love to give you our money.
@@TheoreticallyMediais it fake 3D or actual 3D model? Cause a pseudo 2.5d 3D or just a picture that spins around is quite boring and useless vs video and pics.
@@TheoreticallyMediaI hope by 3D they mean like lenticular flip posters
Always fascinating updates and demonstrations, Tim. And it all looks so surreal to me.
Most of the people in Hollywood will soon be out of a job.
Thank you for the content and the prompt builder unfortunately the price is overwhelming 😢
Check out Kling pricing toward the end of the video. If you catch that sale, it is as low as $5 a month-- so, that's not too bad!
@@TheoreticallyMedia yeah kling is good for sure however gen 3 looks better
I really hope they add beginning and end image frame prompts like luma. It allows for so much more control, especially for my personal use cases
Do you mean like key frames?
@@HORRORinWV uhhh, idk. On Luma you can add a beginning frame and an end frame in image form. I’ve used this to make psychedelic hallucination replications on my second channel. This technique was used for all those ai meme transition videos that have been going around lately
Yup, keyframes. I'm not sure Runway will go that direction, but they do have Camera Control and Motion Brush in their back pocket. And I'm really excited to see how those will be implemented in!
Tim is brilliant
Ty man, the link gpt is dropped :P
7:31
😆 🤣 😂
Everybody was kung-fu hallucinating
Haha! Yesssss!!!
@@TheoreticallyMedia any change of getting the instructions you used for the GPT so we can add it to Anthropic workbench?
Vaping up a storm 😂. Awesome video
Bro is chasing clouds over in the corner!
Gen-3 has lot of moving elements and dynamics, while luma and Kling have fluid camera motion and character consitency. Creativity goes 100% to Gen-3. using all 3 of them accordingly for different shots will have Bonkers result. Job well done.
Again funiii and informative 🥰
Haha, thank you! I was a little spicy in this one, I think!
@@TheoreticallyMedia hahaha bring it on! nothing is better than a cleva sense of huma :))))))
I'm constantly getting a slow zoom or some effect that appears as if it's on top of the image (like a filter effect) and not too the image itself.
Any strategies to prevent this? Btw, I really appreciate the free guide on gen3 prompts, I'm referencing it a lot but still get the slow zoom effect with very little motion elsewhere.
It's all still incredibly rough looking. We have a looong road ahead of us. The timeline isn't as fast as I once thought it would be. Definitely feels the hype is dying down now
It is-- Have you seen that Hype Cycle Chart? (Gartner hype cycle)-- currently we're past the "Peak of inflated expectations" and solidly in the "Trough of disillusionment" -- well, some of us are.
Next up (and where I am) is the "Slope of enlightenment" where you understand what these things can do, and start optimizing and building with them.
It might not be as "wow!" now, but that's fine-- it'll keep getting better. In the meantime, just keep experimenting and creating. The masses will slowly circle back.
16:9 limitation is a deal-breaker for me... hope they work on this as I produce mainly for mobile.
That'll for sure be addressed in an update at some point. For now, you could always generate and re-crop in post. But, yeah, that's a lot of extra work.
And if you're super picky like I am (and I presume you are, and that's a good thing!) it'll be A LOT of extra work!
Thanks for this! I got some good results after repeating back to it my subject. For example, I had it place my cat on a regal throne. At first, it would completely morph my cat, but after having the cat described again, it didn’t morph it
Game changing feature will definitely change my workflow, and get me more focused on starting off with a good storyboard for future films using images from MidJourney or Google’s Imagen3
Here’s my original prompt:
The camera zooms out from the close-up of the cat to reveal her sitting regally on a plush, throne-like couch in a whimsical living room. The room is filled with playful toys, colorful pillows, and soft morning sunlight streaming through large windows, creating a cozy, enchanting atmosphere.
Here was my final one, will link to the before/ after video too:
The camera zooms out from the close-up of the grey British Shorthair cat with striking amber eyes and a daisy charm collar, revealing her sitting regally on a grand, ornate throne upholstered in plush red velvet with gold accents, in a lavish throne room filled with gold, diamonds, and treasure. The room is softly lit by the morning sun, emphasizing her queenly presence.
th-cam.com/users/shortsTwi-EqGQktI?si=z7m68q4PHWAj-8Eo
Man in Blue Suit needs his own TV show...
Haha, that's a good pitch for Netflix!
"So, what's the Man in the Blue Business Suit do in the show?
"Well, he really just kind of walks around the city and usually answers his phone..."
"We'll TAKE IT!"
@@TheoreticallyMedia Dude... sometimes he answers two phones.... sometimes he just take it out and looks at it... it speaks to people on so many different levels...
Tubi have already put in an offer.
Runway has this to say currently: Due to heavy demand, generations on Free Plans are temporarily unavailable. Paid Plans are unaffected. (See Plan options). We thank you for your patience.
I'm new to all this. Obviously the still images are the raw material that's fed into these video generators, but what material are they referencing to get the movements? If someone were to ask why the guy in the video (3:34) has a beard and shades, we'd obviously just say that's what the source image had. But if we were asked why he's walking in that particular _way,_ or at that particular _speed,_ or tilting his head at such-and-such _moment,_ at that particular _angle,_ for that particular _duration,_ where is all that coming from? A particular video it's mimicking? Or maybe an amalgamation of many videos? If one video in the generator's brain/dataset is of a person walking normally, and another video in there is of someone walking weirdly, how does it choose which to mimic when it generates? And what keeps it from jumping from one walking style to the other randomly?
Could you please do a video about RunwayML’s custom model? I’d love to have consistent characters throughout. Right now, it’s hit or miss. I use the same seed, repeat the same prompt for the person, but it doesn’t always work
Yeah, I've got some stuff coming up on that soon. I'm not sure the custom model is currently connected to Gen-3 on Runway-- need to test that, but even then-- that will only apply to Text to Video.
Currently, I think the best way is to train up an Image Generator and then use that for consistency in Image to Video.
Will have a video up on that soon.
@@TheoreticallyMedia fantastic ☺️ glad it’s on your radar, and thanks for the reply!
William Sanoma comment hilarious!
Always informative, always makes me chuckle - double-jointed wig wearing synth synth in a rip off over-priced Williams -Sonoma showroom on fire.
Tim, you are a rock star. ( But we knew that ). Thanks man!
I still play around with a free workflow with these type of tools, it’s just fun but I won’t pay for the pro stuff yet
Great video man thanks , how would you rank the video generator ( luma- kling- runway ) from better to worse?
What's holding back open source image-to-video AI?
Massive amounts of training data mostly.
So can you give a summary of the strengths and weaknesses of all the available AI video generators you prefer? 🙏
That's a good idea-- I'll put that on the (ever growing) to-do list!
Gen-3 Image To Video can be used to create cooking recipes without the need for a presenter?
But sadly, just like a cooking show, you can't eat the food after. Where's the AI for that?!
@@TheoreticallyMedia If you add all the ingredients of a recipe in order and instruct Gen-3 to create a video, won't a presentation result?
"She did it and he's next' LOL!
Totally calling that and it’s in the first act of that movie that doesn’t exist! She totally did it!! And he’s not making it to the end of the movie!
What about a price comparison between Kling and Runway for your next video ?
Great idea! I’ll put it on the list! (It is a long to-do list, but that is a worthy video!)
@@TheoreticallyMediaPRICE COMPARISON:
Runway Gen3
- max 10 sec video
- paying yearly is 20% cheaper than below pricing
- 10 credits gets you 1 second of video
$15/m = 62.5 sec/month
$35/m = 225 sec/month
$95/m - unlimited
Kling
- max 5 sec video
- paying quarterly, half-year or yearly is NOT cheaper than monthly pricing
- 10 credits gets you 5 seconds of video
- ALL THESE PRICES WILL DOUBLE ON JULY 5, 2024:
FREE = 330 sec/day
$5/m = 330 sec/day + additional 330 sec/month
$18.50/m = 330 sec/day + 1500 sec/month
$46/m = 330 sec/day + 4000 sec/month
NO UNLIMITED PLAN
yes, Runway is EXPENSIVE, unless you go unlimited.
Gen3 looks good, the hands are funny. You would think by now it would be worked out but this shows AI really doesn't see the image and is guessing . Orbit and characters turning is a tough one for ai, funny though sometimes it just does it on it's own, hopefully someone works that out.
I think they're wisely (for now) keeping orbits to a minimum with characters, as that was something that was a real issue with Gen-2. They minute you start changing the angle, the character's face would totally change.
That said, Kling and Luma seem to do it well, so maybe Runway is just keeping it tight until they model is 100% there.
@@TheoreticallyMediaAbsolutely, morphing will be an issue for a long time i think.
HI where can I email you a video I made using MJ - then GEN3? I also tried Cling, but I have to say that Gen3 was the best. Dream Machine was the weakest. I should have waited a little before I spent the money! LOL
I need to make a kickstarter video for a boardgame, which one would you use. Im fine with $15 a month but dont know how many videos that gets me or which program is better over all.
I love the synth playing a synth imagery. Perhaps you could create an all girl robot band for another video?
The two phones thing is so me. I live in London and do most business in the US so I'm constantly toggling between a us and a UK phone. Wish I had his build though.....
Haha, time to hit the gym! Maybe start by deadlifting phones?
Oh, and totally- the two phone thing is kind of common here too...although, I'm an idiot and just keep my one phone, so my contacts are a total mess and I have no work/life balance!
Cool!
I love your content. Hey that chatgtp link is down
Hi Tim, the Gen3 Prompt Builder unfortunately isn't working. Could you please bring it up again?
Great stuff Tim, as always I hope for those of us odd balls there's ways to keep making it weird! Also left you a message on Discord, you have a spammer on the channel fyi. :)
I KNOW exactly what you mean when you say how STUPID CHATGPT is, got a real hot tip for that one, so u know how u can scroll in the window, now middle mouse click and point down,
that will make the window think ur interacting with it constantly thus creating better output, sounds stupid I know,
I build custom GPTS all the time and experience the stupid everyday because i am to lazy to wait it out, but every time i do stay there and scroll lock it works 86% of the time,
adhering to custom script requirements thus fixing that issue as well as how good the model its self can give you an answer on any given day(aka 86% of the time)
Glad I Illuminated that for u
Hope it helps🚀🚀💪💪
That's REALLY interesting! Off to give it a shot!! (And I'm glad I'm not the only one noticing that ChatGPT catches the stupid from time to time!)
Does it make consitent models over and over? I find that to be the most difficult thing to get Ai to do is use the same female model throughout the scenes I need to generate.
Most of them not fully yet. There are some tricks and workarounds to make it happen...but, yeah-- still kind of not fully there.
Vidu was the only one that I've seen that has it built in as a feature. It's close? You can check out the look that I did here: th-cam.com/video/UB7mRFewxIA/w-d-xo.html
@@TheoreticallyMedia awesome thanks so much!
I think in the future, promt expert will be an actual job
" Prompt Engineer." Good job title.
I would love to see how it handles an image going from sunset to night back to sunrise. I found no that Ai software handles lighting changes well
Wait, so 3:08, 4:16, 4:48 and 5:11 were all originally still images? It'd be nice to first see the original images before going straight to the video. As it is, I can't really judge how faithful the video generator is being to the starting image. 2:41 and 3:24 are nice demonstrations though.
So, Tim; or anyone who knows really; I see that Kling is finally available for all, which Tim mentioned too and that's good; but what happened to the whole "long video output up to 2 minutes feature" that we heard about before it became available outside of China? Kling seems to be just as limited as the others from what I can tell. How is it that we are still limited on these models to like a measly 10 second outputs or so at this point? Just seems crazy to me! If I'm missing something here, let me know as I'm really just waiting on any of these newer models that can output a decent 1 minute or more before I even want to bother with them, especially if they are charging money for these already.
Let me know, thanks guys.
So with Kling you can EXTEND up to 2 (or maybe 3?) minutes. So, it generates the first 10 seconds, and then you can re-prompt for a second generation off that first one--
That said, I've found that details start to get really fuzzy after the first two extends.
@@TheoreticallyMedia Yeah, I saw that extending option, but exactly, it gets warped a lot when extending in any ai video model still. I'm more after like you could input something from the get go like, "create a one minute long video of people hiking in the mountains and change the scene every 5 seconds", or something like that. The only thing I know of so far that we can kinda do that with is Noisee in Discord, but the quality of the output of it is pretty rough, I'd say it's a "gen and a half" behind most of the others out now. Oh well, guess I'll just keep using Noisee for now until the newer models have the kind of input options I'm wanting, a waiting game as always, lol. Thanks for the response Tim, much appreciated 🙂👍
Been messing around with it. What I learned. This is not for very stylised images. GEN2 strangely enough is better for those. I was using graffiti style characters and they simply transformed into something totally different or just didn't do anything at all. Real world images seem to fair better.
I was going to mention that in the video, but it got cut for time. Same thing happened. Had a stylized character and it turned it into a real one. Kinda cool, but not what I was looking for
WOW its geting kinda scary with AI but i love it
Bro, did you see the newest videos I put out on X? Mixing art flow trained images with Gen 3 animation and lip sync, instant cinematic video. Thanks for another video my friend
The link for the GPT is down. And I can't find it on the GPT gallery
Yeah, they yanked it from me! I've got an appeal in to OpenAI. I don't know what the deal is there!
Is there any open source alternatives that you can run on your own machine though that's comparable in quality? Like T2I, I2I you have stable-diffusion, Fooocus, InvokeAI, ComfyUI, A1111 etc. for for front-ends. But the I2V, if it exist at all in those, isn't on the same level as Runway, Kling or Dream factory.
Can you extend the clip in runway?
Not currently. But I’m sure that’ll come in an update. I’ll say; as is, 10 seconds is pretty good though!
I am having a hell of a time getting characters faces to not morph. Any tips?
Hi Tim! I am not able to access the GPT. Please help
hi, the link on your chat gpt prompt is not working
Yeah, apparently it's "under review"-- OpenAI shut it down for some reason. I appealed it. Kinda weird. Hoping it comes back soon!
Kling did not work for me. It got up to 99% and then never finished the video. I tried it 3 times, and the same thing. NOT GOOD.
I'm still new to AI prompting but assimilate pretty well. didn't take long to learn that 'gluteus maximus and buxom' don't easily flag AI restrictions. Invisible/see through vs. naked.
so have you tried a more logically elementary method for these situations? such as, Instead of a "guy walking off of a plank" something like 'guy walks to the edge of the wood platform then hops into the ocean to catch a cool dip'. or 'man fails to notice oily spot near foot slips and falls into the ocean'.. just like the more provocative ex: that causes the AI to retain the logic of what should be underneath clothing and so it depicts it, and in obedience to the prompt given it violates mature content settings. It's kind of like a child, get as close as possible to the look you want using familiar terminology and then trick it to design the imagery with alternate definitions and professional lingo. instead of burn up the nice looking living room, you could say. someone left a cigarette lit which slowly caused a trickling fire to engulf each item in the room one by one. the AI interprets an "accident" is ok to depict burning up a house vs. you simply telling it to burn up a house without a positive intent.
find simple positive sneaky synonym words to depict complex graphic things to bypass the filter
Go Tim
the chat GPT link doesn't work! thank u very much!
Hedra is great too. and free for 5 gens a day. if you cant get your character to talk
You can even get your characters to sing with Hedra! So much fun.
Hedra is really great! Planning on making a video on them when they have their next big update! I kinda missed them the first time around-- I talked to them and got an early sneak peek, but then life got in the way, and by the time I had a moment to make a video, basically everyone else had covered it!
Still, want to show them some love soon!
Gen-3 is still mostly wasting my time (and money) - way too many useless renders (bad anatomy, not following the prompt closely). Kling doesn't produce quite as many useless renders, but I'm still burning through credits without much to show for it - I definitely prefer the image quality and "acting" on Kling, though. Luma is the best bang for the buck for me - the image quality isn't as good as Kling, but I can get something usable without too many garbage renders. Maybe my video prompting skills needs to improve, I don't know...I'm using pretty good Midjourney images as the source.
Try out the GPT I made for video prompting. Although it's obviously optimized for Gen3, I was running the same prompts in Kling and Luma and getting similar results. Might help on the control front a bit. Although, often these Video Generators will make up their own mind about what they want to do, regardless of what we ask!
@@TheoreticallyMedia Will definitely try it - thanks for posting!
@@TheoreticallyMedia My first test with your prompt engine worked well - thanks again!
All it does is zoom in images with very little motion or nothing at all You need to try 30+ times to get a good result. You must have the unlimited plan otherwise you'll burn your credits fast with no results.
News just broke that they've got a Turbo model coming out soon, and...cheaper prices! That's actually super exciting.
8:04 looks more like a game. Still impressive though
It works very well but is the most expensive of all generators. I rather prefer Kling or Vidu.
Im not a fan of tokens. Hard to work things out :(
Whats the link tk the international site for kling? Thx
Tolkens? Like, Hobbits? Haha- Try the GPT out-- might make things a little easier on that front.
@@TheoreticallyMediadamn. One of them brain freeze moments...
@@TheoreticallyMedia😂I also meant tokens as in currency.
Luma = 100+ prompts for $30
Gen 3???
@@armondtanz 60 seconds or so per month for the standard plan i think. Gen-3 is really only worth it if you can afford the 100 bucks a month pro plan with unlimited generations.
Wait, he can't be Dr Doom. I'm not watching that. Why is it always him? He wasn't right for sherlock either. Mads MIkelsen or Nicola Coster-Walder would have been awesome.
Mads was BORN for that role! Yeah...I mean, the answer of course is: Money. Lots of Money. Sigh.
@@TheoreticallyMedia indeed
Regardless how good all these pop up fly by night AI companies are, if you want to be the leader in your field, you're gonna have to make it more affordable for the average creator, if not, then you're not gonna last long in the game
I think Runway Gen-3 image to video is okay. Facial distortion is still a problem, but sometimes Runway get´s it right...sometimes 🙂Good I am on a unlimited plan, so I can practice and get no sleep 🙂
How is Kling ? Gen3 better in image-to-video?
7:08 LOL very true
i cannot agreee with the acting one, i think gen 3 is actually pretty awesome when it comes to mimics and emotions while telling a story, just try ;)
I gotta play with the prompting more. I've seen some pretty good emotional outputs since the video went up. It's always kind of the thing when you're doing "day 1" type videos. Sigh, life of TH-cam...
@@TheoreticallyMedia so true. Its a trial and error that gets better over time ss always. :) Great times ahead. Keep up you fantastic videos. Love them!
Can it make anime scenes?
Prompt for pink hair girl?
nice vid thanks, gen 3 gpt not working.
@00:18 Dream Factory? 😂
This a BETA we don't know about? 🥲😱
Haha, I KEEP doing that for some reason!! that's really what I WANT to it be named for some reason! HA!
@@TheoreticallyMedia And now... So do I! 😂😭
If only this had controlnet...
Yeeeeeaaaah. I was just talking about that today. That’s a big wish for video. I think we’ll see it soon enough.
Maybe that’s what a revised motion brush might look like?
Fingers crossed?
Kling doesn't even work.
the cost the same - ) 6 videos 10s per month - still it is hit or miss as with other models.
I’d say a little less miss. Not perfect, and more restrained, but nowhere near as bad as Gen 2
10:55 If that would be true, then we would see plenty of good movies generated only by ai, but there is none.
By the end of the year, if not sooner. I’d say: animation level, and probably around 8 to 10 minutes
There’s a lot protecting that from happening in Hollywood rn
The prices need to go down. If its too expensive for ai utubers, its too expensive lol
I don’t know, check out those (admittedly on sale) Kling prices. That’s worth looking into, if you ask me.
Gen 3's Image-to-Video is surprisingly garbage and very generic technology lol sorry
Kling's Image-to-Video is freaking impressive on the other hand.