Stable WarpFusion v0.5.16 - stable build (Patreon)

Hey there, fellow derp learners! Thanks to all the L and XL patrons' feedback, the cumulative v0.5.13-v0.5.16 build is live. 

Changelog since the last stable build (v0.5.12):

• add channel mixing for consistency masks
• add multilayer consistency masks
• add jpeg-only consistency masks (weigh less)
• replace RAFT install with a precompiled model 
• add flow_lq switch for flow generation
• add preview flow images
• add reverse consistency checking mode
• clean up text descriptions
• remove forward flow maps after consistency checking
• add save as pickle option (model weighs less, loads faster, uses less CPU RAM)
• add clip guidance (ViT-H/14, ViT-L/14, ViT-B/32)
• remove unnecessary image resizes, that caused a feedback loop in a few frames, kudos to everyoneishappy#5351 @ Discord
• fix progress bar
• change output dir name to StableWarpFusion

Channel mixing for consistency masks

Consistency masks are now saved and used as 3-channel jpg files. This means they weigh less, process faster, and you can edit them as you see fit and then put them back and warp!

Channel mixing allows you to control the weight of each part of a consistency mask:

missed pixels, overshoot pixels, and motion edges.

Missed pixels are the ones whose movement was predicted incorrectly. We warp a frame to the next frame and then back, and the pixels that didn't get to their expected places are deemed missing and thus inconsistent.

Overshoot or out-of-image pixels are the ones that are being warped into the frame from outside of the frame. Instead of using this channel, you can increase padding.

Edges - are the lines between moving and static objects, i.e. an edge of an object and background, for example.

Save as pickle

You can now avoid CPU out-of-memory crashes by resaving the model as a pickle object and using it for smaller size, faster loads, and a smaller memory footprint.

Replace RAFT install with a precompiled model

Getting rid of that malevolent RAFT install process.

I've converted it to a jit-traced model, and now it's being downloaded from my repo without installing anything for it. Seems to work like a charm on colab, needs testing locally.
Compiled with pytorch v1.12.*, but works for v1.11.*, lower version needs to be tested, but it's not that hard to recompile with a lower pytorch version for compatibility if needed.

Add flow_lq option

This switches between half and full-precision models. The half one uses 50% less memory, thus letting you use 2x larger images.
This comes at some quality cost, of course, but you can now eyeball the difference and check if it's causing any trouble because you will know where the affected areas are located in the actual frame.

Flow_save_img_preview

This saves flow maps as images along with the motion vectors. Not very informative to the human eye, but you can compare quality and investigate the final result artifacts' sources easier now.

Reverse_cc_order

A test feature, allow calculating consistency maps in reverse. Testing it now, not sure it'll help, but still worth testing. reverse_cc_order=True is the default value that replicates older notebooks

Remove forward flow maps after consistency checking

Those maps aren't used after consistency checking, so it makes sense to remove those.

Clip guidance

CLIP guidance is in its infant\embryo state, no multi-crops yet, because even the single sample slows down the rendering a lot, so furter testing is needed to see that it's worth the extra time and VRAM

Remove unnecessary image resizes

Removed unnecessary resizes, now the feedback loop effect should be less prominent in cases where video size and output size don't match.