The example with paint is what sold me completely. This could be a renaissance of indie amateurish web animation ala "flash animations" and Newgrounds, where it's not about quality of drawings and animation but just the content, the ideas, people doing something creative and a lot of entertainment many people can feel encouraged to try their hands at
There is a very intense video of him shaming some Ghibli artists, more of an attack on AI, for the wunky souless results they showed him. He's very disappointed.
@@empire4627I mean it could be useful. I dont think it can ever replace humans. But maybe for small things like simple headturns. The problem is A.I always lacks character in animation as they usually go from A to B and ignore a lot of principals
@@CantHandleThis6 "Can't ever replace humans"? You don't know about Moore's Law and the rate at which AI research is progressing? Can't now doesn't mean it can't in the future.
@@isekaidev7803 Even if A.I can get that good it can never produce intent. It would need an actual personality and consciousness to do that. At that point its just human. You ever listen to a song or watch a show and feel the passion for a project through the screen. A.I can never truly replicate a Demon Slayer Tanjiro vs Rui moment, or a Deku vs Muscular moment animation wise (granted the latter was more a performance thing) the inherent passion will be missing. It might be able to make cool shit but it will always be better to give scenes like that to humans and maybe just let A.I reduce some workload.
@@CantHandleThis6 That is exactly what is happening now. Many companies have fired a huge number of employees and making the rest use AI. The articles don't say what AI, but I assume it's things like ChatGPT. AI isn't going to replace humans as it's going to be doing all the work without any humans. It's going to be replacing 1/10, then 1/6, then 1/5, then 1/4 th etc people and the rest will use AI to do more work per person using AI.
at first it'll shave 90% of the time off of creating an episode. then in a few years it'll just spit out an infinite amount of episodes of anything you want. hard to imagine. but its happening.
@@imanigordon6803 You can't copyright an idea, only the execution of that idea. But I think copyright will eventually be kicked into the dustbin of history by AI anyway, and good riddance. Its primary outcome these days is maintaining corporate strangleholds over arrays of IPs they bought out, none of which have any relation to any of the people involved, which is a total perversion of its original purpose (to protect creators).
This means that with a good enough workflow, an entire 20-minutes anime could be made in 11 hours and be fully automated, using GPT-4, Controlnet, and this tool
As an animator myself, no this wouldn't actually speed up the workflow too much, research animation principles such as follow through, lines of action, squash and stretch. An AI can anticipate certain actions which is an animation principle in itself, but it's very rudimentary. So what this would be good for is speeding up the in-between process which will help key animators. Even SORA AI can't come close to actual animation at the moment despite creating fluid movement.
Not a chance. Unless you want to give away creative control in “make a cool 20 min animation “ People don’t get how much time gets into planning scripting etc. For more like family guy and such yes maybe it could be cut down a lot tho
Just putting it here, doraemon had a gadget that turned comic book to anime episodes, so this is yet another 22nd century gadget coming to life, can't wait to meet real life doraemon
this current version is absolutely amazing for animation clean up and pacing.. seen many high budget animations where they could have done certain scenes way better if they interpolated the frames better instead if just cutting away.
@@neomicryo Sure, that’s very true. But what do you think happens when we have passed the point where humans are needed any longer for anything? Do you think the people in charge, who say the world is over populated will need this many?
@@ghost-user559dude why are thinking all this now? When it happen then you have to show concerns it's gonna take centuries to make all things automated, specially passionless job slavery
5:53 That scene is btw from the One Piece Film "Baron Omatsuri and the secret island" (2005). I‘d recommend it even to non One Piece fans, it‘s the best out of all One Piece films in my opinion.
Definitely a tool that will help save a lot of time and money for in-betweening work in anime studios. It could definitely help cut cost, and perhaps unfortunately, a few in between animators might lose their jobs.
If you have your old flipbooks, you might try putting sequences from them into it. Anyway, pretty interesting. Many years ago, maybe 1985 or 86, one of the guys on the Amiga Compuserve forum was working hard to build a tweener. Of course, it wasn't neural net based, just morphing by connecting points in one image to points in another and linearly pulling them together over time, but it stayed with me how very much that guy really wanted something like this. Don't know what happened to him, but he would certainly be blown away.
maybe it'll save an anime studio some time animating between keyframes, but I'm not sure how helpful this would be, you kinda have to first be able to draw 2 high quality anime frames
really cool. could see this theoretically making animators' lives easier. (e.g. only animating 1/3rd as many frames.) but not sure it can work with their workflow (e.g. what if they need fine edits). But we're getting real close to useful stuff.
Lol what most people dont realize it how good your input has to be , means you better be an artist of some form. You can see even on the glowing piece flick how inbetween frames would be necessary. For all true artists and new artists this tool is dope. For the rest no its not sora
@@jaredf6205 ok create me 2 coherent pics that would result in a clean animation like a character where the arm bends around an elbow in 90 degree , just wit ai gen tools
Also, how will this work if you are using to speed things up on the workload but the end product cannot be licenced and distributed internationally due to indivdiual copyright laws - AI generated content cannot be copyright for example?
Bro, this is awesome. You can simply create key frames and have this do the heavy lifting in-between. As an artist I'm not butthurt at all. I'm loving how these AI tools speed up pipelines and allow smaller companies to compete soon with the big studios.
You would think that animation would be the most successful as it can absorb the uncanny valley of A.I. video generations far easier and the bonus of being able to digitally correct the animations generated. Getting closer.
@@Wonders_of_Reality it seems it's only good at interpolation. That is just a part of animation... not the whole. Animators not getting fired yet 😂 . But on the bright side, just take 10 friends animators + ai and make your own company. It depends how you look at it... On one side you get fired and depressed, on the other side you become business owner and rich.
I feel bad for Japanese animators that earn a living interpolating anime. They were already underpaid and exploited, soon they're gonna be out of a job entirely
On one hand you people complain about those animators being exploited with low wage, now you complain about them being out of jobs. You keep blaming AI, instead of blaming the real cause here, the greedy corporations ? :D So blind and ignorant.
@@mine.moment no one thinks the concept and technology of ai is bad but the entire gen ai system is based off of an illegal data scraping of artists work, and it was all done by greedy corporations, and people are feeding money to these literal criminals by paying for AI. AI should assist artists not replace them, and so far, the industry is just laying off animators and artists. Its the fact that companies will not use AI to make life easier for artists, but instead just pay them even less, or lay off most of them, while keeping one or two to continue to work at an exploitative wage just so they can squeeze more money out in a shorter time.
@@mine.moment well of course i would complain about the low wage and i would complain about the AI cuz their both two different problems and both bad. And the greedy corporations are bad, and so is AI.
i have the feeling the training data they used is the exact animation that they used as examples. still impressive that it’s so much better than the others but i want to see how it does with frames it hasn’t been trained on.
@@Samuel-wl4fwoh that’s sick. i didn’t finish the whole video to see it was open source and he even did his own example which turned out pretty decent 😂
I have been predicting an Indie Animation Boom eversince SD 1.5 began to be Fine Tuned to hell and back. Now with SD3 Medium releasing to the public on the 12th and this. Let's just say the mad lads working with Confy UI are going to have one fun summer ahead of them. Especially once ControlNet gets updated for the new architecture.
it's not there yet, but inbetweening might get processed by AI at some point.. it's already a terribly paid job anyway with terrible conditions.. Kinda feels like 2D animation will somehow be like 3D one if this keeps up, just drawing key poses and let AI do the rest.. interesting technology honestly, but don't expect to make a whole anime or something without skill or talent, this tool won't change that
@@paulsheldon8838 nah, so many passion projects are ruined or made not as hood because the person can’t animate and has to come to a middle ground with the people who make it. Part of the reason people say mangas and books are better. We also will have way more of them made so by default there will be more good ones. And when the visuals are all really good for all the animes/shows people will have to really focus on the story and characters to make their show worthwhile
When done with real video (in a year or so) then one can take video scene beginning and end and just regenerate badly recorded moments. Or do every even and have a new movie. Incredible stuff being released.
16:04 I agree with you that this tool would be so much better if we could input more images. Just using 2 images is not enough, because the animated clip turns out too short.
It’s about the fifth video I see showing tooncrafter and what bugs me is that so far only the samples provided on the project website are actually impressive. I tried it, and I’m also underwhelmed by the results. They’re not bad and much better than any other kind of in betweening frame interpolation I’ve seen, but still they have the same flaw that every AI tool I know have : giving you ALMOST what you want, but not quite, which makes you have to do a lot of post processing for being really usable. Sometimes it’s not that much and all in all it’s faster, but sometimes the time you gain is not significant. And the problem is that there’s no way to tell in advance if it’s going to do a good job or totally mess up, it’s all trial and error, which is not a really sustainable workflow.
Most interesting. May I ask if this (or another tool) may be capable of creating and maintaining the very same set of character’s skills/proportions/look and use them in different environments? Meaning: can I use the characters I’ve created (both with drawing and digital art) for creating an entire illustrated/animated story? Thanks for sharing this video
I knew this was coming but damn. P.S. since midjourney has character stuff now. you could concievably use MJ to generate frames, and this to generate animation. If need be you could maybe repose with SD with openpose.
I really dont want these creative ai stuff what I want is a medical renaissance thats what the world needs cures for cancer tinnitus foreskin regeneration all that stuff thats what I want so badly and pray to all the universe we see that soon
Learning to animate isn't that tough, but it's way more rewarding... and more importantly, controllable. In a way, it isn't the animating that is the hard part, it's the construction of visual story and conveying the right thing. With all these AI tools, it's a bit like throwing a dart and hoping it lands in the desired spot. If you have to throw 10, 20, 100 darts to get roughly what you want, it probably would have been quicker to just animate it and get exactly what you were after in the first place. Anyone who tries to make an animated series or even a single episode with this is still facing a big challenge, and it might be even harder using this method. Now. that said... If this was able to be integrated into an animation software as a way of tweening frames, perhaps with guides and a time line, that would be pretty useful... probably, maybe. Whether that's actually possible is a different matter. I'm not totally shitting on this, because it is interesting, and I appreciate that it's actually designed to do the tweening, rather than just generating everything on a prompt (and stealing artists work)... though it still is kind of stealing animators work isn't it... it didn't learn to animate on it's own. Animators are already shit upon, underpaid, abused... especially the tweeners... who could be made jobless by this after spending years earning their dues. (By the way tweening is basically short for inbetween frames). The key(frame) artist is the one who defines the pivotal moments, and then an army of tweeners do all the inbetween parts. But the upside to this, is that perhaps those tweeners could start creating their own works, with less overhead. It's very hard to tell at the moment. Studios can be awful places, and it may be a good thing to break them. But it might also be a very very bad thing. The will to really create an animation masterpiece requires so much commitment, even with this tech. The ADHD generation will possibly fail to see it through... especially with new AI wonders appearing, distracting them. So, I am really unsure on all this. If you have managed to read all the way through this comment, then perhaps you have the metal to do something good. Those who saw too much text and stopped reading after the first paragraph will probably never achieve anything in animation... doing it the slightly harder way, or with this kind of tech. But if you got to this bit, then maybe you have the concentration to make it work. But there is no real quick fix, and if you truly have a vision, you probably don't want AI to do the work for you. And just to qualify what I am saying, I've been an animator for 30+ years, and started when I was about 15 years old in 1991... and I'm still learning things now. If I used this tech, it'd be to make some cheap paid work slightly cheaper. But I still think it'd be faster to do it myself (I work in 3d, not 2d... and 2d anime is actually mainly 3d pretending to be 2d these days). If you wanted to check out my work, look up waroftheworlds1934 on YT. I'd love to see you over there on my solo project. It's had very very positive feedback. Cheers for the vid Matt. If I was to say anything, I slightly wish you were a bit more measured in enthusiasm and broaden your thoughts to both positive and negative impacts of these technologies. You should consider doing interviews with musicians, animators and film makers about AI music, animation and film. You might find those videos would be very insightful and popular, and the two way conversation might be a lot more constructive than you might think. Much love y'all. Whatever you do, be creative... and don't be afraid to work hard to make things happen. A tool is just a tool.
@@saudiphilippines Any positives out of it, I have no complaints. I come from the angle that even with this tech, it doesn't remove the mountain of work to make something of significance. In a way, I'm trying to say that finding a quick fix might be a detriment to someone who has a story to tell. Learning to do it yourself can be generally faster and more controllable (and improves you as a person). However, there are case scenarios that fall outside what is worth trying to do yourself, or is even possible. An example would be with AI music. I've created some amazing pieces that replicate the sound and feel of 1930s music, that would simply be inhibitively expensive to recreate, especially on a small budget. That has value. But I'd never stop making music myself that I know I can achieve. The creation process is the reward, and not the final result. The fact is, if this technology is going to remain, then we need to find a balance that doesn't reduce human creativity into a headline. It's too boring and puts a person into a permanent consumption state, and that's bad.
As an animation film producer, I am thrilled by this. Because of the hefty price of good animation, I keep waiting for do it yourself animation like this to get better. Its moving fast and getting closer to being able to make it cheaper to do a full animated film.
This along with the news that SD3 is to be released 12th of this month made my day! I hope this works well on 3D art too, I'd love trying it on mine, it's a lot of work animating all the pretty details like hair, clothing and vegetation.
The main issue im noticing with this is the timing of things, it seems very robotic. Dont get me wrong, extremely impressive stuff, but once again, the computer isnt accounting for the 12 principles of animation.
I can see this being used for especially simple animations like Family Guy, American Dad and South Park. Dude! What a time saver. Because most of the time is spend on animation movement. This is so cool.
It's something I seriously want to get into but it needs a huge system though. maybe 64gb of ram minimum and maybe 24gb video card minimum. It's seriously system resource-heavy.
Incredible! but- The amount of work it would take to fix that many frames you might as well just do it yourself to begin with. Is it amazing? YES. but we're not 100% there yet. For animatic type workflow and prototyping scenes this is off the charts useful though; Testing emotional feedback to certain timings etc. without actually having to invest countless hours- crazy
I see a lot of weebos worried about this but I don't think it's as bad as you might think, and I'm pro traditional animation. Interpolating = in-between (in animation terms). That's like an apprentice job, the kind studios source to China or some South Eastern Asian country when limited in budget and/or time. I don't think this changes a lot how animation studios have been running for decades now. This can help getting more shows out, if anything.
this is massive news for the advancement of technology but a apocalyptic one for the artists, AI once in its maturity can replace thousands of jobs and i dont think we are ready to take this massive leap as it will affect millions.
Hey Matt, is there a good way to convert footage to sketches to use this tech effectively by starting with a green screen video filmed on anything? would love to see you try to connect these technologies end to end!
@@jumpy2783 It's pretty similar to most interpolation tech we've seen over the years, including what is in modern tvs with motion smoothing. It also seems to have alot of the same issues like artifacting and deforming the image.
I love that small companies and individuals will be gaining tools to flesh out their projects and make awesome things. I don’t think more traditional animation will die, but I will lament its fading to a sub-sect of people and companies that appreciate legacy animation. If I want the little guy to have tools to flesh out awesome things, I have to accept the larger companies using them too when they could make something wholly unique and impressive without it.
I want to use this to go and make my manga have "still animations" like in a harry potter photo... I think that would be cool. While also colorizing the pages.
3d practically killed hand drawn animation and most artists are against AI but I think this may be the tool to bring back "traditional" animation back into the spotlight, so disney can go make shitty hand drawn movies instead of another generic pixar flic and 2d animators will be once again employed to do what they spent all those years learning to do.
I see a lot of potential with this not for finished products, but rather auto-generated in-betweens and rough drafts to speed up the animation process. Weirdness doesn't matter then because you can just erase it or draw over it like they were guidelines.
All that money and time I invested into learning animation seems kinda worthless now. Its not at the point of it being a problem yet, but it looks like its getting close :(
Don't worry. You will still be hired. Also, it will actually help ease your work with the sketch feature so that you don't need to color or shade all frames by hand anymore.
Wow bruh, so amazing bruh, its fire bruh! Now on to a more serious reply, Professional Animators "decide" where frames and inbetweens are placed and how the lines are drawn for the feel/flow of the animation, its neither a set rule nor is it "calculated". AI is good for rendering frames and repetetive tasks based on calculations, not for making traditional 2d animation, it wil always look/feel weird in terms of timing/follow-though and overlap. But yeah, i do agree on this video on one thing, for sh*tty anime its good enough, the bar is set pretty low anyway so AI can keep squirting out generic bad looking crap for the masses... AI as a tool for auto coloring or animating liquids could be very usefull though. AI doesnt create, it will never make a movie like for example Ren and Stimpy, where all the core animation techniques are pushed to the max. AI==input -> output in a warping, copy/paste kind of way... that is tweening, not animating, and based on how it works, this will (luckily) never change.
The example with paint is what sold me completely. This could be a renaissance of indie amateurish web animation ala "flash animations" and Newgrounds, where it's not about quality of drawings and animation but just the content, the ideas, people doing something creative and a lot of entertainment many people can feel encouraged to try their hands at
Ideas are the plates for the spaghetti.
I am not going to watch them.
"I like rusty spoons" IYKYK
@@artyshreds6274 ?
@ecsrepairjust learn to draw clown
I hope this AI could advance enough to be able to assist in finishing unfinished scenes from the thief and the cobbler
Or berserk
That'd be pretty cool actually
When high fidelity enough. Also, Watership down, as intended.
Gotta throw in some cameos from Aladdin.
I tried it ,tweaked the settings, sadly it doesn't understand the complexity (masterpiece) of the images. But it works great with coloring
Miyazaki would die from heart attack hearing about this tool
There is a very intense video of him shaming some Ghibli artists, more of an attack on AI, for the wunky souless results they showed him. He's very disappointed.
@@empire4627I mean it could be useful. I dont think it can ever replace humans. But maybe for small things like simple headturns. The problem is A.I always lacks character in animation as they usually go from A to B and ignore a lot of principals
@@CantHandleThis6 "Can't ever replace humans"? You don't know about Moore's Law and the rate at which AI research is progressing? Can't now doesn't mean it can't in the future.
@@isekaidev7803 Even if A.I can get that good it can never produce intent. It would need an actual personality and consciousness to do that. At that point its just human. You ever listen to a song or watch a show and feel the passion for a project through the screen. A.I can never truly replicate a Demon Slayer Tanjiro vs Rui moment, or a Deku vs Muscular moment animation wise (granted the latter was more a performance thing) the inherent passion will be missing. It might be able to make cool shit but it will always be better to give scenes like that to humans and maybe just let A.I reduce some workload.
@@CantHandleThis6 That is exactly what is happening now. Many companies have fired a huge number of employees and making the rest use AI. The articles don't say what AI, but I assume it's things like ChatGPT. AI isn't going to replace humans as it's going to be doing all the work without any humans. It's going to be replacing 1/10, then 1/6, then 1/5, then 1/4 th etc people and the rest will use AI to do more work per person using AI.
Couple more papers down the line and we'll be generating our own anime. The next few years are gonna be wild.
at first it'll shave 90% of the time off of creating an episode. then in a few years it'll just spit out an infinite amount of episodes of anything you want. hard to imagine. but its happening.
@@Chris_Hruskabetter copyright if you have a good idea 😂
@@Chris_Hruska What would you make?
@@imanigordon6803 You can't copyright an idea, only the execution of that idea. But I think copyright will eventually be kicked into the dustbin of history by AI anyway, and good riddance. Its primary outcome these days is maintaining corporate strangleholds over arrays of IPs they bought out, none of which have any relation to any of the people involved, which is a total perversion of its original purpose (to protect creators).
The hardest part will still be writing good content, AI can help with that too but can't do it alone.
This means that with a good enough workflow, an entire 20-minutes anime could be made in 11 hours and be fully automated, using GPT-4, Controlnet, and this tool
No, GPT is too nerfed for anime
As an animator myself, no this wouldn't actually speed up the workflow too much, research animation principles such as follow through, lines of action, squash and stretch. An AI can anticipate certain actions which is an animation principle in itself, but it's very rudimentary. So what this would be good for is speeding up the in-between process which will help key animators. Even SORA AI can't come close to actual animation at the moment despite creating fluid movement.
@@AbyssalSodagive it a few more months
@@MarcoCholo-iz9js I've been hearing "give it a few months" for over a year now. It's still going to be a while before it can do half decent animation
Not a chance. Unless you want to give away creative control in “make a cool 20 min animation “
People don’t get how much time gets into planning scripting etc.
For more like family guy and such yes maybe it could be cut down a lot tho
The MAPPA employees will finally be able to go home and sleep!
Nah, they just fire the other employees and keep two guys to do all the extra work
they will out of work and no longer afford a home.
as if
they will just make the animation even more incredible with all bells and whistles at all time - when this can be done in 1080p that is
they will just make more anime but still work the same time
Just putting it here, doraemon had a gadget that turned comic book to anime episodes, so this is yet another 22nd century gadget coming to life, can't wait to meet real life doraemon
this current version is absolutely amazing for animation clean up and pacing..
seen many high budget animations where they could have done certain scenes way better if they interpolated the frames better instead if just cutting away.
This is amazing for the indie animation industry
Imagine how many fewer frames they’ll have to draw
Imagine how many artists they will no longer need
@@ghost-user559 technology replaces jobs, this has been a thing for centuries
@@neomicryo Sure, that’s very true. But what do you think happens when we have passed the point where humans are needed any longer for anything? Do you think the people in charge, who say the world is over populated will need this many?
@@ghost-user559dude why are thinking all this now? When it happen then you have to show concerns it's gonna take centuries to make all things automated, specially passionless job slavery
@@ghost-user559 you got it.
The colorising animation is the most useful part of this at this point. I'd like to see it on random not anime start to finish tho
5:53 That scene is btw from the One Piece Film "Baron Omatsuri and the secret island" (2005). I‘d recommend it even to non One Piece fans, it‘s the best out of all One Piece films in my opinion.
6:10 is from One Piece too
This is incredible. I’d love to try this. The “Caustic” effects with the water is amazing
finally some advancements in ai video, moving at a similar pace to image generation at the start
Definitely a tool that will help save a lot of time and money for in-betweening work in anime studios. It could definitely help cut cost, and perhaps unfortunately, a few in between animators might lose their jobs.
This is pretty impressive. I hope AI video evolves quickly. I'm still frustrated that I can't even prompt SVD yet.
If you have your old flipbooks, you might try putting sequences from them into it.
Anyway, pretty interesting. Many years ago, maybe 1985 or 86, one of the guys on the Amiga Compuserve forum was working hard to build a tweener. Of course, it wasn't neural net based, just morphing by connecting points in one image to points in another and linearly pulling them together over time, but it stayed with me how very much that guy really wanted something like this. Don't know what happened to him, but he would certainly be blown away.
Now I really want it to be the program creator. Haha
maybe it'll save an anime studio some time animating between keyframes, but I'm not sure how helpful this would be, you kinda have to first be able to draw 2 high quality anime frames
The quality of animation is better than in some modern anime, so it should be useful.
Cant wait till No Game No Life s2 is animated by AI
Haha, this!
Why do all the example images come from an already existing anime?
Ikr?
because it's trained on existing data....no originality
Because the people who want to use this tool don't draw
Ai needs references so it uses existing data from other works and trained itself through repeatable processes.
Can't wait till I am able to remake one punch man season 2 in season 1 quality
what was wrong with season 2? the animation wasn't as good but still pretty high quality
@@thomblueart8448 Certain scenes were alright, but the majority was subpar. Lack of fluidity, incorrect proportions, still frames.
@@chineseducksauce9085 I might be a bit biased just for Garou being in it I guess!
@@thomblueart8448 happens
@@thomblueart8448Season 2 trash
just a few years and then I will animate one punch crossover with every piece of fiction ever existed and defeat every villain with one punch
Is this really that much better than animatediff in stable diffusion and morphing between a number of frames using ComfyUI?
really cool. could see this theoretically making animators' lives easier. (e.g. only animating 1/3rd as many frames.) but not sure it can work with their workflow (e.g. what if they need fine edits). But we're getting real close to useful stuff.
Welp rest in peace in-between frame animators ✌️
I've tried this, and it works ok. I just don't know how to get the sketch in. What do you upload as the sketch? A video, or a animated gif?
17:45 that is pippo from doraemon, i miss her to this day, she sactificed her race to save earth 😭😭😭
This is legit amazing. Video title is definitely accurate for this one. Thanks for showing us these demos!
I waited my entire life for this kind of technology and it's finally here!
Lol what most people dont realize it how good your input has to be , means you better be an artist of some form. You can see even on the glowing piece flick how inbetween frames would be necessary. For all true artists and new artists this tool is dope. For the rest no its not sora
just use ai to generate a good image
You just have to combine this with existing AIs
@@jaredf6205 ok create me 2 coherent pics that would result in a clean animation like a character where the arm bends around an elbow in 90 degree , just wit ai gen tools
I agree.
So what if you add more frames? Or is that not possible?
You can input how many frames you want in between the keyframes you draw
@@jajankenanims5409 oh hell yea thank you. 🙏🏾. I currently don't have my computer to test it. But man I can't wait to get back to work.
Also, how will this work if you are using to speed things up on the workload but the end product cannot be licenced and distributed internationally due to indivdiual copyright laws - AI generated content cannot be copyright for example?
I think this is only the case if it's only generated using AI.
I actually see this as a good thing for anime companies as it could help artists be more efficient once this technology becomes better and better.
Being more efficient means they need less animators…
@@SamWilkinsonnno, same number of animators but ton lot of project handled by each of them 💀
@@andra9694 wat
@@SamWilkinsonnthey can use this as a tool
@@SamWilkinsonnIt means that the businesses will either replace them or expect more of them and overwork them twice as much 😂
Bro, this is awesome. You can simply create key frames and have this do the heavy lifting in-between. As an artist I'm not butthurt at all. I'm loving how these AI tools speed up pipelines and allow smaller companies to compete soon with the big studios.
You would think that animation would be the most successful as it can absorb the uncanny valley of A.I. video generations far easier and the bonus of being able to digitally correct the animations generated. Getting closer.
10:19 it is caustics, not causatics, Matt :) Keep up the great job, mate.
Finally someone can fix "when they fire all the animators" Twinkle Nora
Finally, but is basic
I imagine if you fed two frames of Twinkle Nora to this thing the entire server would burst into flames.
Well, if this AI can animate Sailor Mercury, then I’ll be worried for my studio. Until then, it’s just another semi-useful tool.
Someone needs to do that xD
@@Wonders_of_Reality it seems it's only good at interpolation. That is just a part of animation... not the whole. Animators not getting fired yet 😂 .
But on the bright side, just take 10 friends animators + ai and make your own company. It depends how you look at it... On one side you get fired and depressed, on the other side you become business owner and rich.
I feel bad for Japanese animators that earn a living interpolating anime. They were already underpaid and exploited, soon they're gonna be out of a job entirely
Well this tool will help them not kill them lmao
@@MilkGlue-xg5vj no it wont it will make most of them not get hired, and human creativity will continue to decline more and more
On one hand you people complain about those animators being exploited with low wage, now you complain about them being out of jobs. You keep blaming AI, instead of blaming the real cause here, the greedy corporations ? :D So blind and ignorant.
@@mine.moment no one thinks the concept and technology of ai is bad but the entire gen ai system is based off of an illegal data scraping of artists work, and it was all done by greedy corporations, and people are feeding money to these literal criminals by paying for AI. AI should assist artists not replace them, and so far, the industry is just laying off animators and artists. Its the fact that companies will not use AI to make life easier for artists, but instead just pay them even less, or lay off most of them, while keeping one or two to continue to work at an exploitative wage just so they can squeeze more money out in a shorter time.
@@mine.moment well of course i would complain about the low wage and i would complain about the AI cuz their both two different problems and both bad. And the greedy corporations are bad, and so is AI.
I’ll be able to make my own Tokyo Ghoul remake if we don’t get one from a big studio. 🥳
Finally One punch man seasons can come out once a year instead of once every 5 years...
Lol
@@NoahtheGameplayerSeason 2 is a non canon event
i have the feeling the training data they used is the exact animation that they used as examples. still impressive that it’s so much better than the others but i want to see how it does with frames it hasn’t been trained on.
Tested it on some kaiyu 8 frames, it was pretty good
@@Samuel-wl4fwoh that’s sick. i didn’t finish the whole video to see it was open source and he even did his own example which turned out pretty decent 😂
I have been predicting an Indie Animation Boom eversince SD 1.5 began to be Fine Tuned to hell and back.
Now with SD3 Medium releasing to the public on the 12th and this.
Let's just say the mad lads working with Confy UI are going to have one fun summer ahead of them.
Especially once ControlNet gets updated for the new architecture.
Music for animation: Udio beta
Sound effects: Audiogen V2
Video for animation: Tooncrafter
😅😊
I don't think this AI will be able to replace animators.Interpolation is already done in the 3d software.
it's not there yet, but inbetweening might get processed by AI at some point.. it's already a terribly paid job anyway with terrible conditions..
Kinda feels like 2D animation will somehow be like 3D one if this keeps up, just drawing key poses and let AI do the rest.. interesting technology honestly, but don't expect to make a whole anime or something without skill or talent, this tool won't change that
4:12 The person actually always keeps their left foot in front. That's an issue that also happens in Sora.
The guys over at Corridor Crew are going to plotz when they see this.
Hah! So I'm not the only one who thought this, huh? 😄
Idk, last time I saw them talking about AI it was very, very negative.
@@mycelia_owIn what sense?
@@mycelia_ow What video was this?
Holy shit. We are about to step into an age of cinema
Age of slop which nobody cared enough about to even make?
@@paulsheldon8838 nah, so many passion projects are ruined or made not as hood because the person can’t animate and has to come to a middle ground with the people who make it. Part of the reason people say mangas and books are better. We also will have way more of them made so by default there will be more good ones. And when the visuals are all really good for all the animes/shows people will have to really focus on the story and characters to make their show worthwhile
7.43
This is geniunely insane though. It colored the entire room based off just these sketches>
When done with real video (in a year or so) then one can take video scene beginning and end and just regenerate badly recorded moments. Or do every even and have a new movie. Incredible stuff being released.
definitely cool stuff, can't wait for it to get better.
16:04 I agree with you that this tool would be so much better if we could input more images. Just using 2 images is not enough, because the animated clip turns out too short.
It’s about the fifth video I see showing tooncrafter and what bugs me is that so far only the samples provided on the project website are actually impressive. I tried it, and I’m also underwhelmed by the results. They’re not bad and much better than any other kind of in betweening frame interpolation I’ve seen, but still they have the same flaw that every AI tool I know have : giving you ALMOST what you want, but not quite, which makes you have to do a lot of post processing for being really usable. Sometimes it’s not that much and all in all it’s faster, but sometimes the time you gain is not significant. And the problem is that there’s no way to tell in advance if it’s going to do a good job or totally mess up, it’s all trial and error, which is not a really sustainable workflow.
Most interesting. May I ask if this (or another tool) may be capable of creating and maintaining the very same set of character’s skills/proportions/look and use them in different environments? Meaning: can I use the characters I’ve created (both with drawing and digital art) for creating an entire illustrated/animated story? Thanks for sharing this video
I knew this was coming but damn. P.S. since midjourney has character stuff now. you could concievably use MJ to generate frames, and this to generate animation. If need be you could maybe repose with SD with openpose.
I really dont want these creative ai stuff what I want is a medical renaissance thats what the world needs cures for cancer tinnitus foreskin regeneration all that stuff thats what I want so badly and pray to all the universe we see that soon
All of my favourite miscellaneous models have always been covered 1st by you.
This is why this is the best channel about ai.
💥 The argument: "It's very weird and bad, but it's AI" 😅😅😅
Learning to animate isn't that tough, but it's way more rewarding... and more importantly, controllable. In a way, it isn't the animating that is the hard part, it's the construction of visual story and conveying the right thing. With all these AI tools, it's a bit like throwing a dart and hoping it lands in the desired spot.
If you have to throw 10, 20, 100 darts to get roughly what you want, it probably would have been quicker to just animate it and get exactly what you were after in the first place. Anyone who tries to make an animated series or even a single episode with this is still facing a big challenge, and it might be even harder using this method. Now. that said...
If this was able to be integrated into an animation software as a way of tweening frames, perhaps with guides and a time line, that would be pretty useful... probably, maybe. Whether that's actually possible is a different matter.
I'm not totally shitting on this, because it is interesting, and I appreciate that it's actually designed to do the tweening, rather than just generating everything on a prompt (and stealing artists work)... though it still is kind of stealing animators work isn't it... it didn't learn to animate on it's own. Animators are already shit upon, underpaid, abused... especially the tweeners... who could be made jobless by this after spending years earning their dues. (By the way tweening is basically short for inbetween frames). The key(frame) artist is the one who defines the pivotal moments, and then an army of tweeners do all the inbetween parts.
But the upside to this, is that perhaps those tweeners could start creating their own works, with less overhead. It's very hard to tell at the moment. Studios can be awful places, and it may be a good thing to break them. But it might also be a very very bad thing. The will to really create an animation masterpiece requires so much commitment, even with this tech. The ADHD generation will possibly fail to see it through... especially with new AI wonders appearing, distracting them.
So, I am really unsure on all this. If you have managed to read all the way through this comment, then perhaps you have the metal to do something good. Those who saw too much text and stopped reading after the first paragraph will probably never achieve anything in animation... doing it the slightly harder way, or with this kind of tech. But if you got to this bit, then maybe you have the concentration to make it work. But there is no real quick fix, and if you truly have a vision, you probably don't want AI to do the work for you.
And just to qualify what I am saying, I've been an animator for 30+ years, and started when I was about 15 years old in 1991... and I'm still learning things now. If I used this tech, it'd be to make some cheap paid work slightly cheaper. But I still think it'd be faster to do it myself (I work in 3d, not 2d... and 2d anime is actually mainly 3d pretending to be 2d these days).
If you wanted to check out my work, look up waroftheworlds1934 on YT. I'd love to see you over there on my solo project. It's had very very positive feedback.
Cheers for the vid Matt. If I was to say anything, I slightly wish you were a bit more measured in enthusiasm and broaden your thoughts to both positive and negative impacts of these technologies. You should consider doing interviews with musicians, animators and film makers about AI music, animation and film. You might find those videos would be very insightful and popular, and the two way conversation might be a lot more constructive than you might think.
Much love y'all. Whatever you do, be creative... and don't be afraid to work hard to make things happen. A tool is just a tool.
Good luck on not getting carpal tunnel.
@@ViralThing ???? From what? What an odd comment.
It's a really cool model and it can be used to interpolate older photos (not just anime) and bring old moments back to life.
@@saudiphilippines Any positives out of it, I have no complaints. I come from the angle that even with this tech, it doesn't remove the mountain of work to make something of significance. In a way, I'm trying to say that finding a quick fix might be a detriment to someone who has a story to tell. Learning to do it yourself can be generally faster and more controllable (and improves you as a person).
However, there are case scenarios that fall outside what is worth trying to do yourself, or is even possible. An example would be with AI music. I've created some amazing pieces that replicate the sound and feel of 1930s music, that would simply be inhibitively expensive to recreate, especially on a small budget. That has value.
But I'd never stop making music myself that I know I can achieve. The creation process is the reward, and not the final result. The fact is, if this technology is going to remain, then we need to find a balance that doesn't reduce human creativity into a headline. It's too boring and puts a person into a permanent consumption state, and that's bad.
@@shaketelevision I make music too. But I'm not intimidated by AI that can create music. In fact, I'm amazed and want better control over it.
As an animation film producer, I am thrilled by this. Because of the hefty price of good animation, I keep waiting for do it yourself animation like this to get better. Its moving fast and getting closer to being able to make it cheaper to do a full animated film.
This will definitely helped the traditional 2d animators
This along with the news that SD3 is to be released 12th of this month made my day! I hope this works well on 3D art too, I'd love trying it on mine, it's a lot of work animating all the pretty details like hair, clothing and vegetation.
The main issue im noticing with this is the timing of things, it seems very robotic. Dont get me wrong, extremely impressive stuff, but once again, the computer isnt accounting for the 12 principles of animation.
now this is what I like to see
10:18 causetics?
I can see this being used for especially simple animations like Family Guy, American Dad and South Park. Dude! What a time saver. Because most of the time is spend on animation movement. This is so cool.
I'd like to see somebody this with consistent characters and controlnet to try and do the whole animation process
What's amazing is the coherence through all the frames
It's something I seriously want to get into but it needs a huge system though. maybe 64gb of ram minimum and maybe 24gb video card minimum. It's seriously system resource-heavy.
The coloring frames option is not out there..
Incredible! but-
The amount of work it would take to fix that many frames you might as well just do it yourself to begin with. Is it amazing? YES. but we're not 100% there yet. For animatic type workflow and prototyping scenes this is off the charts useful though;
Testing emotional feedback to certain timings etc. without actually having to invest countless hours- crazy
I see a lot of weebos worried about this but I don't think it's as bad as you might think, and I'm pro traditional animation.
Interpolating = in-between (in animation terms). That's like an apprentice job, the kind studios source to China or some South Eastern Asian country when limited in budget and/or time. I don't think this changes a lot how animation studios have been running for decades now. This can help getting more shows out, if anything.
hope that will help to finish Berserk and animate vagabond
this is massive news for the advancement of technology but a apocalyptic one for the artists, AI once in its maturity can replace thousands of jobs and i dont think we are ready to take this massive leap as it will affect millions.
Hey Matt, is there a good way to convert footage to sketches to use this tech effectively by starting with a green screen video filmed on anything? would love to see you try to connect these technologies end to end!
Even though the currently generated results are not fully polished, I expect that AI will replace "in between" animators in the future.
Animation programs already had automatic inbetweening interpolation for years.
Obviously? @@Skitskl33
This isn't that though
@@jumpy2783 It's pretty similar to most interpolation tech we've seen over the years, including what is in modern tvs with motion smoothing.
It also seems to have alot of the same issues like artifacting and deforming the image.
I couldn't use the rough drawings to guide the animation, did anyone figure out how to do this?
Make a anime movie will come from really expensive and lots of working hours to some hours....
Finally.
instant manga to anime mayhaps
As an artist, I would be extremely unsatisfied with the output if I had to use this thing
Nice one Matt! Thank you ✨🙏✨
😂bruh every time I come to ur page…I leave feeling like I got new super powers 😂
Oh hey, what about that one anime with like 5 frames total. Nora I think? Maybe it can fully animate it.
Mappa animators: were saved! 😭😭😭
Yall dont understand lol these companies are just going to fire people till the workload gets back to basically the same
Wake me up when you can tell a complete story with it
If you're expecting Keanu Reeves from Cyberpunk, I don't think he's within the MattVidPro budget.
@@80sGameGuy No i want to see tools I can use right now to make money with
@@dwainmorris7854if you were a artist, then this could. Or advanced at consistent characters maybe.
@@Samuel-wl4fw I am a artist and I have been using Zbrush : Corel Draw: and Clip stuido Paint for Years ,but I need AI to make my work flow faster
@Samuel-wl4fw Yes, I am an artist. I have been using Zbrush, Corel draw and Clip. Studio paint, but I need AI to make my workflow faster.
I love that small companies and individuals will be gaining tools to flesh out their projects and make awesome things.
I don’t think more traditional animation will die, but I will lament its fading to a sub-sect of people and companies that appreciate legacy animation. If I want the little guy to have tools to flesh out awesome things, I have to accept the larger companies using them too when they could make something wholly unique and impressive without it.
Title: "I cant tell its AI!" Video: "I can see some warping...."
So much excitement for a potential threat to people's dream jobs. It's unreal.
it will kill the magic of hand drawing animation
I want to use this to go and make my manga have "still animations" like in a harry potter photo... I think that would be cool. While also colorizing the pages.
3d practically killed hand drawn animation and most artists are against AI but I think this may be the tool to bring back "traditional" animation back into the spotlight, so disney can go make shitty hand drawn movies instead of another generic pixar flic and 2d animators will be once again employed to do what they spent all those years learning to do.
Thanks Matt,I was wondering what I was going to do this weekend.
I like that it’s open source.
I've been waiting for this, finally no more drawing for long durations of time
Fucking finally😂 the world is mine. I cant waot until this is legit
I see a lot of potential with this not for finished products, but rather auto-generated in-betweens and rough drafts to speed up the animation process. Weirdness doesn't matter then because you can just erase it or draw over it like they were guidelines.
All that money and time I invested into learning animation seems kinda worthless now. Its not at the point of it being a problem yet, but it looks like its getting close :(
Don't worry. You will still be hired.
Also, it will actually help ease your work with the sketch feature so that you don't need to color or shade all frames by hand anymore.
@@saudiphilippines Hopefully that's the case 🤞
@@Caspian. It depends on the circumstances and greed.
@Dookiestain-jw8wy Exactly!
Wow bruh, so amazing bruh, its fire bruh! Now on to a more serious reply, Professional Animators "decide" where frames and inbetweens are placed and how the lines are drawn for the feel/flow of the animation, its neither a set rule nor is it "calculated". AI is good for rendering frames and repetetive tasks based on calculations, not for making traditional 2d animation, it wil always look/feel weird in terms of timing/follow-though and overlap. But yeah, i do agree on this video on one thing, for sh*tty anime its good enough, the bar is set pretty low anyway so AI can keep squirting out generic bad looking crap for the masses... AI as a tool for auto coloring or animating liquids could be very usefull though. AI doesnt create, it will never make a movie like for example Ren and Stimpy, where all the core animation techniques are pushed to the max. AI==input -> output in a warping, copy/paste kind of way... that is tweening, not animating, and based on how it works, this will (luckily) never change.
Will this put animators out of their jobs or will it help them improve their working conditions in Japan?
Their working conditions likely won't improve any, they will probably just demand more from people now
Looking good Matt!
i find the lineart interpolation and autocoloring the most impressive and usefull