Home Creators Posts Import Register Favorites Logout

Downloads

Content

Ultimate SECourses Video Upscaler preview: the diffusion-based STAR model upscales context aware, 32 frames at once, for top quality; feedback is requested.

Note: this is an older post kept for reference. The current, maintained version is SECourses Upscaler Pro: SeedVR2 & FlashVSR+ Video and Image Upscaler.

Update 1 adds RIFE FPS interpolation. Planned: 720p presets for 6 GB to 96 GB GPUs, saving and loading presets, an instant cancel button and fast GAN upscalers, aiming to restore entire movies automatically. The released app is SECourses Upscaler Pro.

Update 1: RIFE FPS Increase Interpolation Implemented

  • Please check the screenshots extremely carefully and what other features you would like to see please tell me

  • Hopefully I will add presets for 1280x720p (0.92 mega pixel) for each GPU like 6 GB, 8 GB, 12 GB, 16 GB, 24 GB, 32 GB, 48 GB, 80 GB, 96 GB

  • You will be also able to save and load your presets, every setting simply

  • Also hopefully I am gonna add cancel button which will cancel process immediately and generate final video from so far generated frames

  • Hopefully I will add fast GAN based upscalers too that upscales each frame

    • Currently our STAR model upscales context aware (like upscaling 32 frames at once) with diffusion based thus it is highest quality but slower and VRAM demanding

  • But other than these do you see we lack anything?

  • I have also attached screenshots as zip file to see in full screen

  • By the way, Bronze tier will be raised to 6.99 from 6.00 before the release of the app

  • Existing subscribers will not be impacted and will continue from their subscription price as long as they don't cancel

  • With this app, you will be able to upscale an entire movie fully automatically at industry level hopefully, like restoration of older movies

Related posts

All posts in this topic: Image and Video Upscaling and Restoration · New to SECourses? Start Here: Best and Most Popular SECourses Posts

Files

Comments

cool1

In the Topaz video AI upscaler (an older version) there's an option to upscale to 7680x4320 (and it's possible to upscale to that on my 3090). The max target width slider in the Star video upscaler screenshot shows it goes up to 4096 wide. Could it go up to 7680x4320 on normal GPUS (eg. RTX3090)? Maybe it could also allow you to change the frame rate like the topaz one could (eg. convert 24 fps to 60 fps with interpolation, convert a video to slow motion etc.). Also in the GUI it's got "max target height" then "max target width". Normally I think things ask for width first so it might be a bit easier if those two were re-ordered. From the screenshots it seems to split it by scene and has an option to describe scenes (using CogVLM2). I think it could be good and helpful if there was something like this (maybe another app) that like the "critique" option in JoyCaption beta 1 (for critiquing images as if by an art critic) if you could have an option where it critiqued a whole short film based on multiple scene descriptions and could suggest changes that could make it better (like re-ordering scenes, cutting sections out, adding things etc.).

Steve

Looks awesome! One suggestion for the output tab- include option to input custom filenames for each file type with duplicate filename checking!

J D

My main concern would be to preserve likeness on a face while also fixing slight artifacts or adding detail. Sometimes it can be good at adding detail, but then lose some of the likeness. So a setting that is designed to balance those two things, with an emphasis on preserving facial likeness

cool1

Maybe lora options could be added to help keep the likeness, with it trained on whatever ai it uses.

Zow

I think this is looking perfect, it would be a great first release, we can then compare its results with Topaz Starlight (this release might be even better already) and vs other tools to improve it if needed. Just to add ideas in case you think they fit (now or in the future): Check "SVFR: AI Video Upscaling: Enhancing Low-Res Footage", maybe any of the SVFR like the video impainting (to remove unwanted stuff like watermarks or other) and also has black and white video coloring. That said, there's also a 2D video to 3D converter (https://github.com/nagadomi/nunif/tree/master/iw3) in case it also fits (not sure if this is the best current 2D to 3D solution though). Thanks for the update.

SECourses

thanks nice ideas. i plan to release hopefully soon

SECourses

this model doesnt support LoRA but i think face likeliness will be preserved great

SECourses

can you elaborate more ? batch processing will save same file name as input. single processing saves as 0001 0002 0003 etc

SECourses

ye those are all fake upscales. this is real diffusion based upscale. perhaps you can upscale very big rest 1 frame by 1 frame but that reduces quality I plan to add fake upscales as well like ESRGAN based which you can use to upscale further after diffusion upscale

Draxel

I'm excited to try this out. Thank you for putting it together.

Anshul Gupta

why are you increasing price of subscription and fastly. you are forcing person like me to leave you, who really and keenly interested in AI and your wark. I AM your subscriber from beginning of 2024

Steve

Yes I mean it would be nice to be able to input custom filenames e.g. "Bluecar" instead of default "0001". The Wan video app does not use the same file name as the input which can become confusing when running large batches

cool1

Though are you sure Topaz AI upscales are "fake"? Even though they aren't (I assume) using diffusion based, aren't they all using deep learning (which is still AI)? Perplexity.ai said "Most modern AI upscalers, including Topaz Video AI and ESRGAN, use deep learning models (often GANs or similar architectures) to generate high-resolution images from low-resolution inputs. So if they're using deep learning AIs to upscale isn't that still true AI upscaling (I assume they're taking more than one source frame into account too when generating new ones, at least one of their interpolation models does)? And one help page about Topaz Video AI (a newer version than the one I own probably) also says it can upscale to 16K (though I don't know if I'd have the VRAM for that). And from what I read, their Starlight AI model seems to be a lot better than their other AI models (though I've not tried it). Perplexity.ai said "Diffusion models can sometimes hallucinate details that weren't present in the original, which can be an advantage or disadvantage depending on the use case.". Though Topaz's AI models will probably also be adding some 'fake' details sometimes. But yes if ERSGAN is more standard deep learning and can do the higher resolutions that diffusion ones can't, that would help if there's an option for that, thanks (eg. for when higher res than 4K is needed).

SECourses

thanks a lot. adding dynamic FPS reduce first feature so the time it takes to upscale can be shortened and later we can do fast RIFE FPS increase

SECourses

subscribers wont be affected. only for new people

SECourses

thanks. and also subscribers wont be affected. only for new people

SECourses

ye but this is really hard. it breaks so many stuff.

SECourses

GAN based upscales are fake compared to diffusion based upscales that is what I mean. I think STAR doesnt have a limit but big resolution really increases VRAM. consider you are upscaling 32 images at once into 16K. compare what it would require in flux or SDXL upscale.

Draxel

Rife is really good. Actually far better than the paid products I've used. You would think they would just license rife.

SECourses

yep. RIFE is perfect in my opinion as well

Lou

This looks promising. The cancel option you mentioned will be an important addition. I would have to get familiar with the program before making any suggestions. I do quite a bit of video editing and have work flows based on what I'm trying to achieve. I also have 6 or so paid programs I use on a regular basis. I think with AI upscaling, the most important feature for me is the ability to toggle face upscale on or off. I would normally add detail and grain well before considering AI for upscailing facial features.

SECourses

awesome so your feedback will be immensely important to me.

Erwin Hartsuiker

Very much looking to trying out this video upscaler on my RTX5090!

SECourses

Thanks a lot for comment. Yep I develop on RTX 5090 but it works on all GPUs . I have RTX 3090 ti as well to test