sxela

Stable WarpFusion v0.5.16-nightly - consistency channel mixing (Patreon)

Published:

2022-09-23 17:15:59

Imported:

2026-06

Tags:

colab notebook stable diffusion warp warpfusion

haven't archived this post yet. have a subscription? use the importer!

Downloads

stable_warpfusion_v0_5_162_cc_layers.ipynb

Missing 1 full-res photo, 1 file, as this post has not yet been imported.

Content

Long time no see my Warp Sisters and Brothers!

Here's a new nightly build for you to test.

The changelog:

add channel mixing for consistency masks
add multilayer consistency masks
add jpeg-only consistency masks (weigh less)

Consistency masks are now saved and used as 3-channel jpg files. This means they weigh less, process faster, and you can edit them as you see fit and then put them back and warp!

Channel mixing allows you to control the weight of each part of a consistency mask:

missed pixels, overshoot pixels, and motion edges.

Missed pixels are the ones whose movement was predicted incorrectly. We warp a frame to the next frame and then back, and the pixels that didn't get to their expected places are deemed missing and thus inconsistent.

Overshoot or out-of-image pixels are the ones that are being warped into the frame from outside of the frame. Instead of using this channel, you can increase padding.

Edges - are the lines between moving and static objects, i.e. an edge of an object and background, for example.

add save as pickle option (model weighs less, loads faster, uses less CPU RAM)

You can now avoid CPU out-of-memory crashes by resaving the model as a pickle object and using it for smaller size, faster loads, and a smaller memory footprint.

Please leave your feedback on discord, as usual. https://discord.gg/j6NXSGZwJ2

Best, Alex.

upd: fixed Clip guidance not working with init or latent scales

upd: add community guide, session termination

Files

Previews only

Stable Warpfusion v0.5.16-Nightly settings study

testin different Consistency map mixing settings. Settings are provided in the same order as in the notebook, so 1-1-1 corresponds to "missed_consistency_weight": 1, "overshoot_consistency_weight": 1, "edges_consistency_weight": 1 Basic settings: { "text_prompts": { "0": [ "a highly detailed matte painting of a No Man's Sky Screenshot, a tall grass field, broken machinery, Simon Stalenhag , featured on Artstation." ], "40": [ "a highly detailed matte painting of a derelict starship, No Man's Sky Screenshot, a tall grass field, broken machinery, Simon Stalenhag , featured on Artstation." ] }, "user_comment": "testing cc layers", "image_prompts": {}, "range_scale": 0, "sat_scale": 0, "max_frames": 75, "interp_spline": "Linear", "init_image": "/content/p27735_p_v8_aj.jpg", "init_scale": 0, "skip_steps": 20, "frames_scale": 0, "frames_skip_steps": "70%", "clamp_grad": true, "clamp_max": 0.7, "seed": 4275770367, "fuzzy_prompt": false, "rand_mag": 0.05, "eta": 0.55, "width": 512, "height": 768, "diffusion_model": "stable_diffusion", "steps": 50, "diffusion_steps": 1000, "video_init_path": "/content/sqr1.mp4", "extract_nth_frame": 1, "flow_video_init_path": null, "flow_extract_nth_frame": 2, "video_init_seed_continuity": false, "turbo_mode": false, "turbo_steps": "3", "turbo_preroll": 1, "flow_warp": true, "flow_blend": 0.999, "check_consistency": true, "turbo_frame_skips_steps": null, "forward_weights_clip": 0, "forward_weights_clip_turbo_step": 0, "padding_ratio": 0.2, "padding_mode": "reflect", "consistency_blur": 2, "inpaint_blend": 0, "match_color_strength": 0, "high_brightness_threshold": 180, "high_brightness_adjust_ratio": 0.97, "low_brightness_threshold": 40, "low_brightness_adjust_ratio": 1.03, "stop_early": 0, "high_brightness_adjust_fix_amount": 2, "low_brightness_adjust_fix_amount": 2, "max_brightness_threshold": 254, "min_brightness_threshold": 1, "enable_adjust_brightness": false, "init_latent_scale": 0, "frames_latent_scale": 0, "cfg_scale": 15, "dynamic_thresh": 2, "warp_interp": 1, "fixed_code": false, "blend_code": 1, "normalize_code": false, "mask_result": false, "reverse_cc_order": true, "flow_lq": true, "use_predicted_noise": false, "clip_guidance_scale": 0, "clip_type": "ViT-H-14", "clip_pretrain": "laion2b_s32b_b79k", "missed_consistency_weight": 1, "overshoot_consistency_weight": 1, "edges_consistency_weight": 1 }