Video Upscaling tools (Patreon)
Content
As long as I'm fielding questions, I was also asked about my upscaling workflow for my videos so I thought I'd break down the tools that I use and offer some advice based on my experiences with it.
Early on I was almost exclusively using Kling to generate footage in 720p (and occasionally 1080p) so it was pretty easy to maintain a crisp and detailed look. Now that I'm using Seedance for just about everything, I've switched to generating 480p clips in order to control costs, so maintaining that high quality detailed look is a bit trickier.
So my workflow is basically to generate in 480p and then use a couple of tools to upscale the clips 2x before editing in Davinci Resolve Studio, usually fit to a 1080p timeline. In Davinci I use a couple of effects to claw back a little more detail and apply some sharpening before exporting. If I feel like the quality of the exported full movie is good enough to support more upscaling I'll run it through the upscaler again to generate a 4K version.
In most cases the results are pretty solid - I think it's hard to tell looking at my work when I switched over to the lower resolution clips (there are a few places in the Offering video where things get a bit soft but I think that's more down to the clips than the scaling).
The Tools
I have three tools at my disposal to handle the various upscaling tasks (one free/OSS, two proprietary):
Lifecast Upscale Video
Topaz Video
Davinci Resolve AI Super Scale
Lifecast Upscale Video
Lifecast.ai was apparently a company that made a lot of tools for making gaussian and volumetric video using iPhones. When they shut down they generously open-sourced all of their tools (MIT license). Among the tools is the prosaically-named Video Upscaler.
Which does just what it says: it upscales video 1.5x or 2x, optionally de-noising and "enhancing". You just drag and drop a video or a batch of videos, set a couple of options and click the big green "enhance" button and that's it. It's quick and simple and in my experience does a pretty great job of scaling videos without requiring a ton of technical knowledge.
Video upscaler is the primary tool that I use, running every 480p clip through it prior to editing. Not a ton of fiddly options - these are the settings that I use:
That's it. Drop a bunch of files, set four or five options, get bigger videos.
One caveat: make sure that all videos in a batch are the same aspect ratio: if you drop, for example, a 4:3 aspect ratio clip and a 16:9 clip in the same batch, they'll both come out 16:9.
Download Link (Github). Has installers for both M1 Macs and Windows.
Topaz Video
Topaz video is a proprietary/paid app, and while it isn't crazy expensive as far as these things go, it ain't exactly cheap: $299/year, or as part of the Topaz Studio that includes all of their apps for $399 per year. I've been using TopazAI for image upscaling in my day job for years, so adding their video tools to the workflow was an easy sell for me. And for the most part, they work as well as the image tools although they are a bit fiddly.
Topaz Video is more of a Swiss Army Knife tool than just an upscaler: it does upscaling (2x, 4x, or arbitrary), but also frame interpolation (to change a 24FPS video to 30 or 60 for example or to add slow mo), and also has a ton of features to add or remove noise, correct focus, add grain, or otherwise enhance video using a variety of AI (really ML) models tailored to different tasks.
Which is where the fiddly part comes in. Glancing at the interface right now I count 10 different AI models for general enhancement, and 5 just for frame interpolation, with only short tool tip messages to hint at what each model might be best at. So expect to do some experimentation or research to really get the most out of the program. It's insanely flexible and powerful, and capable of taking a crappy low res clip and making it look great at 4x the initial res, but it's also possible to waste hours of your life to get to that point before you really learn all of the ins and outs.
I use Topaz for upscaling when I have a problem clip that Video Upscaler just doesn't deal with well, or when I need greater than 2x upscaling (in my tests running 4x upscaling in Topaz looks better than 2x in Upscaler run twice). And lately I've been using it for frame interpolation on my final videos to generate the higher frame rate versions. The ability to either run single functions (just upscaling, just enhancing etc.) or combining them together in a single pass is powerful. I do all of my processing locally, but Topaz also supports cloud rendering so you could run it on a machine with a sub-par GPU.
If I had to choose just one tool out of the three to keep, this would probably be the one just because of its flexibility. It does everything, and mostly does it well.
Davinci Studio
I use Davinci Resolve Studio for all of my editing, and one of the features that the paid version adds is "AI Super Scaling" of footage using a variety of models. You can scale 2x or 4x, with or without "enhancement", and there are two "NVIDIA RTX" models that presumably use NVIDA-specific tech. Davinci also does frame interpolation, but I haven't used it at all.
I haven't used this for much yet, but I do activate it when I want to do a zoom in an edit that would scale the video past the timeline size. It obviously works pretty well as I can't really tell from the output where I used it (one example that springs to mind is the cunnilingus section of the Tribute rough cut/preview - some of those cuts zoom in 50% or so, and they look just as sharp as the rest of the footage to my eye. I plan on doing some testing to see if it's viable as a primary upscaler - skipping the pre-processing and just editing the 480p footage directly - which would save a bit of time for every video going forward.
Not upscaling-specific, but the paid version of Davinci also enables the "Texture Pop" and "UltraSharpen" effects which can add back some detail in upscaled footage that looks a bit soft... Texture Pop in particular can have an incredible effect if used judiciously. It's also incredibly easy to over-use, though.
Bonus Tools
Not specifically upscaling tools, but I also find myself making use of a couple of other standalone apps in my process.
SVP 4 Pro
SVP is a weird program. It's a video player that interpolates any video to higher frame rates in real time, but it can also be used to do a variety of frame rate interpolation and transcoding tasks. It's a useful tool, but the interface is designed by a crazy person, and no matter what settings I choose I seem to end up with a 120FPS video.
Adobe Media Encoder
If you are already in the Adobe Creative Cloud prison ecosystem, this is a simple but extremely powerful tool to transcode videos for a variety of use cases. Literally hundreds of presets to optimize videos for everything from YouTube to mobile devices to professional broadcast and even VR use, all applied with a single click. It also has a ton of other potentially useful things (e.g. automation using watch folders, time tuning to invisibly shorten videos, etc.) that I'll likely never use.
Lossless Cut
Does what you'd expect: it lets you cut videos into segments, and gives you the option of saving, discarding, and/or combining the segments on save. I like using this to make quick edits (e.g. to cut down a video for R34) rather than feeding it back into Davinci Resolve, but it does have one flaw: the timeline selector is somewhere between pretty and wildly inaccurate. If you need frame-accurate clipping this is not the tool, but for less-accurate cutting it gets the job done quickly and easily.
That's it - hope this is useful. Any suggestions for related tools I might want to check out?