FREE AI Video Upscaling for Windows Mac and Linux
ฝัง
- เผยแพร่เมื่อ 1 ต.ค. 2024
- IMPORTANT TIP: Before you get started make sure your video drivers are fully updated otherwise they have older Vulkan support and not work
FOR THIS IS FOR WINDOWS & LINUX - FOR MAC INSTRUCTIONS SEE THIS VIDEO: • Free AI Video Upscalin...
Original video on using a GUI instead of the command line: • AI Video Upscaling for...
Links:
Waifu2x ncnn vulkan project: github.com/nih...
WINDOW ONLY ffmpeg: github.com/Btb...
IF NECESSARY! video format converter: handbrake.fr/
Other sites not used in this tutorial:
Single photo at a time processing online: waifu2x.udp.jp/
Main waifu2x project: github.com/nag...
NOTE: // lines are COMMENTS and not commands, don't paste these in!
==== WINDOWS =====
// Get info about video: i.e. frame rate and dimensions
ffprobe video/video.mp4
// Extact images to our frames folder
ffmpeg -i video/video.mp4 frames/frame%06d.png
// Extract audio to our video folder
ffmpeg -i video/video.mp4 video/audio.mp3
// Scale images: -s valid options are 1 2 4 8 16 or 32
waifu2x-ncnn-vulkan.exe -i frames/ -o scaled/ -n 2 -s 2
// Put video back together lower -crf means better quality but larger file sizes
// MAKE SURE YOU CHANGE THE FRAME RATES FROM 24 TO THE ACTUAL VIDEO FRAME RATE!
ffmpeg -f image2 -framerate 24 -i scaled/frame%06d.png -i video/audio.mp3 -r 24 -vcodec libx264 -crf 16 video/upscaled.mp4
==== LINUX ====
// Open a terminal and CHANGE THE DIRECTORY to where you have the waifu2x-ncnn-vulkan program copied
// Make the 3 additional subdirectories we will be using
mkdir video
mkdir frames
mkdir scaled
// Install ffmpeg directly - no need to download it like with Windows!
sudo apt install ffmpeg
// Get info about video: i.e. frame rate and dimensions
ffprobe video/video.mp4
// Extract images to our frames folder
ffmpeg -i video/video.mp4 frames/frame%06d.png
// Extract audio to our video folder
ffmpeg -i video/video.mp4 video/audio.mp3
// Scale images: -s valid options are 1 2 4 8 16 or 32
./waifu2x-ncnn-vulkan -i frames/ -o scaled/ -n 2 -s 2
// Put video back together lower -crf means better quality but larger file sizes
// MAKE SURE YOU CHANGE THE FRAME RATES FROM 24 TO THE ACTUAL VIDEO FRAME RATE!
ffmpeg -f image2 -framerate 24 -i scaled/frame%06d.png -i video/audio.mp3 -r 24 -vcodec libx264 -crf 16 video/upscaled.mp4
Hi, thank you very much for the wonderfully explained video, I manage to convert some really old 352x240 videos to 720p! I got a question you might be able to help me with though. The upscaled video plays just fine with mpv (in linux), but the screen is black if I play it with VLC. Any ideas?
Edit: I just though to copy and paste the video to a Windows PC and try to play it there, same issues with all the players. So it looks like the ffmpeg conversion from frames to video did not work as expected! I will investigate further.
Edit 2: I just had to add `-pix_fmt yuv420p` in the final ffmpeg command before the `output/`. Hope this helps somebody one day!
Thank You bro, You saved me!
Awesome instructions. It worked all the way, but does it only work on VLC player? What codec will allow it to play on QuickTime Player?
I figured it out. after vcodec I put prores and at the end I put .mov instead of mp4 so the entire code is
>ffmpeg -f image2 -framerate 29.97 -i scaled/frame%06d.png -i video/audio.mp3 -r 29.97 -vcodec libx264 -crf 16 video/upscaled.mp4
I use waifu2x, and it'll upscale without all this extra steps
I did a video on waifu2x but it is windows only and some people weren’t sure about running software they didn’t know what it was doing in the background. So I put together what waifu2x is doing behind the scenes so you can run it on any platform and know exactly what is going on
Hi thanks for you tutorial and the tool of course. I followed every step but when making the resize step i got the following error: vkCreateInstance failed -9
Hard to say exactly but it looks like a Vulcan driver issue. See if you can update your graphics drivers and try it again
How did you get your background like that? It's so cool. Thanks for the tutorial.
That is a program called Wallpaper Engine, it is like $4 on Steam. Thousands of community submitted animated wallpapers to choose from, highly recommended if you suffer from wallpaper boredom syndrome like I do ;-)
So for some reason at 13:00 i have stream #0:2 whereas you have only stream #0:0 and 0:1. Is it possible to delete the extra stream in the commands?
I haven’t seen that before. Are you getting multiple lines with different frame rates or just the one like like mine that is just labeled 0:2? If you just have one like with one frame rate then I am guessing you can just ignore the labeling carry on with the instructions. If not I would use something like handbrake to re-encode the video to H264 to get rid of any funkiness in the formatting. My only guess on that is sometimes DVDs have multiple tracks and perhaps that is embedded in the video? Just a guess though
@@freewarefocus yes it is weird, it is not labeled as a video nor audio and ffprobe is not able to recognize it which leads to the error "codec input 0 is unsupported for stream 2" but i did find a workaround.. other than that, your video has been a BIG help!! TY
Glad you found a workaround. Video encoding “standards” are kind of a hot mess
Hi! Is there a difference between this one and Waifu2x GUI from your vid in November 2022?
The GUI, by default, uses the same engine (waifu2s Vulcan). However it is Windows only, a really large download and some people were worried if they couldn’t trust it. So these videos are to show that you can just run that same AI upscaling engine without any GUI directly on whatever OS you want.
@@freewarefocus thanks man! get a like from me
million thanks for the cmd tip you gave at around 10:00, never knew that after being soooo many years in windows, shame for me haha.
It also took me about a decade to learn that trick too, glad I could pass it along :)
Hey man! Nice video! What is the main difference between this and the other tutorial you made 6 months back?
The other video used a visual front end program (GUI) which as large & Windows only, although it made just a couple of clicks to use. This is how to run Waifu2x directly on any platform without a GUI, much smaller, more flexible and multi platform
@@freewarefocus Thank you for replying. Just watched both of your videos and I am going to try the first method since I have windows and high spec pc. Do you think I should skip the old method and try the new one which is in this video? Thank you in advance.
Do you have any suggestions for interlaced video? I've got some laserdisc captures of anime.
I would de-interlace it first. Download the free, open source video encoder HandBrake (handbrake.fr/) and drag your video in. Then on Filters tab you can play around with "Interlace Detection" and "Deinterlace" if needed but the defaults seem to work pretty well in my limited testing. Check your "Save As" path and name at the bottom and then click the green "Start Encode" button at the top. BTW you can also use ffmpeg (Google ffmpeg deinterlace Yadif) but HandBrake makes it even easier.
How can I do this in VirtualBox? Do I have to download and install some other video driver?
Usually VMs have horrendously slow GPU performance (here waifu2x is using Vulkan to interact with a GPU). Maybe if you could setup GPU passthrough on a VM you might get acceptable performance but just like you don't really hear of VM gaming you likely will be disappointed at AI upscaling on a VM. You could use a CPU only version of waifu2x, here is a Github project (but I have zero experience with this): github.com/Rorical/waifu2x-ncnn-cpu
will this upgrade a 1080 video to 4k ? and is there a noticeable difference in quality?
Yes it can up scale video of really any resolution. Of course the results of the AI processing vary depending on the content of the video. With AI upscaling it actually attempts to identify items in the video and scale them based on what they are. On some videos it does a great job and on some it is kinda 'meh'. Best to try it on a short sample since processing times can be long.
@@freewarefocus Appreciate the feedback.. ill try it on an old short. I tend to use videos on YT in compilations that are 720 but i cant stand 720 haha
@@PigTownSports BTW you can change the model it is using (it defaults to an animation oriented model called cunet) by using this command line parameter: -m model-path to change the AI model to the photo realistic (ie real life) one named "models-upconv_7_photo"
works perfect. thx!
Any different with this vs the GUI one?
The GUI supports several different engines and this is only waifu2x Vulcan but a fraction on the size, you are in full control, runs on wWin, Mac & Linux and not everyone felt comfortable about using a freemium software. Functionality between the two is basically identical, however so if you like the GUI stick with it
failed.... what the problem...??
Exactly what step are you having the issue with?
Does it work on Windows 11
I haven't tried it but I have seen zero reports of issues on Windows 11. Waifu2x-ncnn-vulkan and ffmpeg depend very little on the operating system they are ran on, hence the wide compatibility for Mac, Linux and Windows.
Can we convert 30 to 60fps in this?
Yes you can use ffmpeg to change the framerate. That is -r option. You can also use a free tool like handbrake to get a nice GUI to do the same thing
@@freewarefocus thank you so much
@@freewarefocus What is the method by which ffmpeg generates new frames? Is it copy, blend, motion estimation, any way to check and change it? If I don't know that it uses the best possible method by default nor that and how I can set it, I would just leave the frame rate as original.
I have this error "../src/intel/isl/isl.c:2216: FINISHME: ../src/intel/isl/isl.c:isl_surf_supports_ccs: CCS for 3D textures is disabled, but a workaround is available."
Try upgrading your graphics driver. Are you on Windows or Linux?
@@freewarefocus thank you. I use Linux. I restart it and worked great now
I always trust a video with no examples.
I think you may have the wrong video - I literally take a sample video through the entire process and show the before and after at the end