PERFECT ROTO with After Effects & RunwayML
ฝัง
- เผยแพร่เมื่อ 25 ก.พ. 2023
- For amazing rotoscoping in less time you can combine the strengths of RunwayML and After Effects.
Rotoscoping can be a time consuming task especially if you want to get really nice looking edges and hair. Luckily there are some great tools out there and some are better than others. RunwayML is an AI based solution that can do really fast rotoscoping but only for rough roto. Meanwhile After Effects has great edge refinement tools, which are also AI assisted. So why not mix the two?
🔥BECOME A PATRON - for exclusive content and credit on new videos
🔗bit.ly/3pcBuBn
---------------------------------------------------------------
😁 Subscribe to Joshua M Kerr
🔗bit.ly/3l4U9wo
---------------------------------------------------------------
🎦 JOIN THE NEXT GEN FILMMAKERS DISCORD
🔗bit.ly/3kUzqvj
---------------------------------------------------------------
👀WHAT TO WATCH NEXT..
📺 An Unreal Engine Short Film In Three Days
bit.ly/3DwvPtI
📺 The Smallest Virtual Production Studio
bit.ly/3WW3KCY
📺 Virtual Production Playlist
bit.ly/3HnUeDe
---------------------------------------------------------------
Let me know your preferred rotoscoping tools in the comments
I love these tutorials. Would love to see them for Davinci Resolve.
Green Screen is not equal to ROTO !!!!!!!!!!!!!!!!!!!!!!!
I don't belive that I said it was. That's a lot of exclamation marks.
Hi Joshua, I exported the pre-comp from the original and the Runwayml mask, however how do I proceed, I can’t overlay this precomp on my background.
Which version of After Effects are you using on this? The one i have is different to the one in the video as its not the same. Thanks.
That looked really amazing in the final result. I only saw a few places where there was some odd behavior, for example at 6:44 when the cat's paw moves near your shoulder, it somehow causes your shirt to melt away. But this was barely noticeable.
Amazing dude. I love your workflow. Such an inspiration and a plethora of knowledge. Absolutely brilliant mate.
Hey thanks, Sometimes I wonder whether my videos are useful at all... so it's good to know you liked it.
Great tutorial. Thank you. Subbed.
Looks great! Thanks for sharing such valuable tips 👍
No problem
As it happens, I am working on a project that I need to rotoscope and I was thinking about doing exactly what you did, using Runway for the rotoscoping and after effects for the edges. Thanks for affirming my workflow.
Hey no worries. What I would definitely suggest is using the chatter reduction too. Very useful.
Wow! What a life saver!! In my case the footage was filmed so badly with the green screen and any other keying effects or plugin would ruin the person in the footage. That's when I thought of using a roto brush but as its time consumes, I was almost gonna give up. But your process made my process way more easier and made worth trying to fix the errors made when filming. Thank you for your tutorial and have a good one!
Rotobrush 3 is a good way to go too
so good!
thank you very much
Thanks to share great knowledge with us.
My pleasure
Nice one! I just use the 3D keyer in DaVinci Resolve Studio and it works great, good edge detection and definition. It's also less work than this workflow. But whatever works for you 🙂 love seeing everything you put out 😁👍
Exactly, whatever tool you're used to can work. I'm always looking toward new methods and the best possible result (which is rarely the quickest). I always enjoy your comments.
What a cool video ❤. We didn’t even know about the web app, but will implement it for sure. Love that you download the mask so you don’t have to worry about duplicate HQ video files
Hey! Glad you found it useful. Its a super quick workflow. I'm going to be using it for Unreal Engine Filmmaking for sure.
We need AI doing perfect rotoscoping and boring technical stuff like this, but they create AI doing creative tasks to replace humans...
I hear you man.
Pretty awesome!
Thanks!
I'm going to give this a crack. Nice tips dude. Thankee :)
Let me know how it goes
Thank you very much, can you create a video with this procedure exported to Unreal Engine 5? For example, to make videos for news or event presentations?
How about this... th-cam.com/video/uWWmdFO79fQ/w-d-xo.html
As a Davinci Resolve user I was curious to try RunwayML masking, but from what you have shown, it seems to be pretty similar to Magic Mask feature in Resolve, Almost identical to Runway, with the built in tools for refining that you did in After Effect. One thing that AF I think does a bit better is the refine hair brush for rotoscoping, For fine hair detials, but I'm less impressed by the Runway masking than I originally imagined it to be. Thanks for the tutorials.
I've been fiddling with magic mask in Davinci 18.5. There's some impressive stuff going on there. But once ive mastered it I'll probably do a comparison.
@@JoshuaMKerr Yeah. I think there is some user experience involved in all these tools, since AI can only take us so far, or at least give a decent starting point.
For example, Magic Mask, was originally not supposed to be a rotoscoping tool, but just a mask for some effects or to help in color grading. Which is still what is best to use it. However rotoscopoing is what gets most attention for obvious reasons. And in my experience it works somewhat similar how you show Runway works in video, plus some fine tuning options.
Where I think Magic Mask and Runway roto struggle is when there are some unexpected gaps between limbs or staff behind the person, it can get confused.
That needs to be manually adjusted. But on the flip side, Magic Mask does offer lot of fine tuning and selection of certain body parts, so its impossible to do more than one pass with different settings and combine masks later. This appraoch off course works with all of the methods, Runway, Magic Mask and RotoBrush2 or any number of other selection methods, like keying or plain old school roto wish splines.
One can use various Magic Mask modes, like person or object, and track only face, legs or body or arms etc. Sometimes gives better track and one can later combine the masks.
@@KrunoslavStifter Some really great detail here. Thanks for this comment, I think you have hit the nail on the head.
@@JoshuaMKerr No problem. Looking forward to see what you do in your videos. Also here are some good stuff from another guy, might serve as inspiration for new videos ideas.
Davinci Resolve relight and depth map
th-cam.com/video/rwgC5Svdt-w/w-d-xo.html
Magic Mask (roto) + CG room.
th-cam.com/video/IKGXvAaXZDw/w-d-xo.html
감사합니다.!
You're welcome
wow, end result was a flex
Haha thanks
Hey looks amazing, I would like to know how this works if I want to roto and export sparetad objects
Give it a try. What sort of objects?
@@JoshuaMKerr like for instance, separate the cat and you in different masks
Oh I think so. But probably not seperate parts of the same subject... like cats body and it's ears
That’s pretty flawless! Now I just need a few hundred dollars for an after effects license haha unfortunately I’m still doing everything by hand
I'm trying to avoid tracking a video clip of a room where there are screens where I need to put custom made videos in each display. Can masks replace the need for tracking in this situation if I have the green screen+masks from runway?
doing it with masks sounds like an awful lot more work to me
Gotcha, appreciate the tip! @@JoshuaMKerr
I'm on after effects 2020, I used refine matte but the pre comp becomes black. It kinda works... If i turn off decontaminate edge color but the whole video becomes a darker but i do see the edges lessen
It's probably not a one size fits all solution but it works for me most of the time.
nice, i was very lazy for rotoscoping, but i believe this tec is faster.
Fun too
You have a greenscreen in your background. Is this process also more efficient if the object is not good seperated from the background?
Good separation is always advised. But I do know that Runway is quite good at isolating objects in non greenscreen scenes. It always depends on the scene and you never really know what will confuse the AI.
is there any way to make the brush have a rougher outline on purpose? I want to make a stop motion video where it looks like the characters have been cut out of paper, but the super defined rotobrush wouldn't give that "rough edge" look.
I would imagine you would do your roto first and then add an effect to roughen the alpha mask
@@JoshuaMKerr do you know of any specific effects I could try? That's what I was thinking to do but I'm not to AE and I'm not sure exactly how to apply the necessary effects
@@JEBLego have you tried the roughen edges effect?
@@EzDyt no wait i wasnt able to find an effect like that. do you know where it is/how to apply it to a roto?
How are you liking this workflow now that Rotobrush 3.0 is now in Beta?
Exploring this now
Can't wait for the vid~!@@JoshuaMKerr
can the mask be used in davinci resolve ?
Probably, but I don't have enough experience with Davinci to know what result you can get
Hey man I’m a big fan of your work and this is the first video I’ve watched of yours!😵😅 I have some crazy big buck ideas for shorts if anyone wants a buddy to roto with and make some banger clips, I have a TH-cam channel (not this one) and trying to do some big things (with such easy work of roto and after effects😅)
What's your channel?
Hi how can I add my rotoscope into unreal engine from run way
I have a couple of videos on that.
th-cam.com/video/uWWmdFO79fQ/w-d-xo.html
th-cam.com/video/WWjloYDCXd0/w-d-xo.html
Hey... I just tried this out for roto. Its pretty slow on my machine. I didn't know this was all web based. Its possible I need to be on a faster M1/2 system for web, currently on my imac pro. Also I'm using the sample footage and its not very quick at all with a lot of extra paint work. I will try another file thats 4K with more pixel info. And also do it on a faster machine. Report back later.
How did it go?
@@JoshuaMKerr I found a way to download Runway via Webcatalog, this is all on iMac Pro btw. I did this thinking I can get it all working on my machine, but I still think it's working as a web app. Either way working with it as an "desktop app" is much faster. Also, I worked with my own 1080x1920 clip with a simple person taking up most of the frame with fairly decent separation from the background. It went through the clip like butter. The export from Runway is only 720p (free version) and the black and white matte worked very well. I will try this out again with a tougher shot and report back.
One thing...I wish this had a stand-alone desktop app. I would love to try this with footage from work but since we are uploading clips online I really can't use this. We have highly sensitive materials and it can't be floating around on the internet.
I feel like now with rotoscope 3.0 runway isn't needed anymore, right?
Experimenting with this now
thats not enough. you need to generate a spline path that can be cleaned up by hand. In essence let the AI do 99% of the roto and then you just step in to adjust a few areas.
I'm just used to doing the cleanup with rotobrush. Results look good to me.
RUNWAY ML is a good step in the right direction but the control in the UI aren't very good. I still get a better roto in AE unfortunately.
100 percent. Have you tried rotobrush 3
your cat is named brisket?
It is! That was the name he had in the shelter but I stuck with it
You are a genius, still looking through these, could you send me the invite to your discord
You're too kind. Here's an invite link... discord.gg/5aFQGGBS