Home Creators Posts Import Register Favorites Logout
haven't archived this post yet. have a subscription? use the importer!

Content

Hi there! 

It's not Christmas yet, but I come bearing gifts!

The changelog: 

New stuff:

  • add xformers for colab
  • add latent init blending
  • add verbose mode
  • add 4 channels (4d) support to user-defined latent stats
  • add 4 channels (4d) support to fixed code mode
  • track latent stats across all frames
  • print latent norm average stats

Fixes:

  • fix frame correction for non-existent reference frames
  • fix init scale loss to use 1/2 sized images

Colab link: https://www.patreon.com/posts/74651376

xformers for colab

Add xformers optimization fro colab (or linux).
Increases diffusion performance by about 30%, decreases memory requirements. 

For example, 16bg card can now run 720x1280 with init and latent scale on. Without xformers than won't be possible.

Currently works on colab or linux machines, at it only has binaries compiled for those architectures. You can try building it on windows if you feel lucky :D 

Latent init blending

stable-settings -> danger zone -> blend_latent_to_init

You can now blend the latent vector to current frame's raw latent vector. Helps stay closer to the init video, but not in a pixel-perfect way like fdecreasing flow blend does. 0.2 is a good value to begin with. 

Verbose mode

stable-settings -> danger zone -> verbose

Toggles printing out various stuff during render. Turn off if you're not experiencing any errors.

4 channels (4d) support to user-defined latent stats

Frame correction -> latent_fixed_mean, latent_fixed_std

User defined latent norm stats can now be entered in 4-channel format, like [0.8, 0.7, 0.5, 0.1]

Check printed latent stats for reference values for your video during render.

Colab link: https://www.patreon.com/posts/74651376

Files

Previews only

Stable WarpFusion v0.5.20.1 Nightly - Flow blend experiments

Colab: https://www.patreon.com/posts/74650475 Settings: { "text_prompts": { "0": [ "a beautiful breathtaking highly-detailed intricate portrait painting of a blissful ignorant enlightened young woman with long hair, against a backdrop of stars in the style of alphonse mucha and ilya kuvshinov and peter mohrbacker, ross tran rossdraws, watercolor, featured on artstation, 70mm, rendered in octane" ], "550": [ "a highly detailed matte painting of a robot, No Man's Sky Screenshot, tall grass field, broken machinery, Simon Stalenhag , featured on Artstation." ], "1000": [ "a beautiful breathtaking highly-detailed intricate portrait painting of a blissful ignorant enlightened young sxelaqwertyop dancing against a backdrop of stars in the style of alphonse mucha and ilya kuvshinov and peter mohrbacker, ross tran rossdraws, watercolor, featured on artstation, 70mm, rendered in octane" ], "1500": [ "a highly detailed matte painting of a robot sxelaqwertyop, No Man's Sky Screenshot, tall grass field, broken machinery, Simon Stalenhag , featured on Artstation." ] }, "user_comment": "testing cc layers", "image_prompts": {}, "range_scale": 0, "sat_scale": 0, "max_frames": 461, "interp_spline": "Linear", "init_image": "", "clamp_grad": true, "clamp_max": 0.7, "seed": 4275770367, "fuzzy_prompt": false, "rand_mag": 0.05, "eta": 0.55, "width": 704, "height": 1280, "diffusion_model": "stable_diffusion", "diffusion_steps": 1000, "video_init_path": "init_images/120461054_383336673751268_6796056890868473576_n.mp4", "extract_nth_frame": 1, "flow_video_init_path": null, "flow_extract_nth_frame": 2, "video_init_seed_continuity": false, "turbo_mode": false, "turbo_steps": "3", "turbo_preroll": 1, "flow_warp": true, "check_consistency": true, "turbo_frame_skips_steps": null, "forward_weights_clip": 0, "forward_weights_clip_turbo_step": 0, "padding_ratio": 0.2, "padding_mode": "reflect", "consistency_blur": 2, "inpaint_blend": 0, "match_color_strength": 0, "high_brightness_threshold": 180, "high_brightness_adjust_ratio": 0.97, "low_brightness_threshold": 40, "low_brightness_adjust_ratio": 1.03, "stop_early": 0, "high_brightness_adjust_fix_amount": 2, "low_brightness_adjust_fix_amount": 2, "max_brightness_threshold": 254, "min_brightness_threshold": 1, "enable_adjust_brightness": false, "dynamic_thresh": 50, "warp_interp": 1, "fixed_code": false, "blend_code": 0.2, "normalize_code": true, "mask_result": false, "reverse_cc_order": true, "flow_lq": true, "use_predicted_noise": false, "clip_guidance_scale": 0, "clip_type": "ViT-H-14", "clip_pretrain": "laion2b_s32b_b79k", "missed_consistency_weight": 1, "overshoot_consistency_weight": 1, "edges_consistency_weight": 1, "style_strength_schedule": [ 0.5, 0.2 ], "flow_blend_schedule": [0.7]*65+[0.85]*30+[0.8], "steps_schedule": [ 50, 50 ], "init_scale_schedule": [ 0, 0 ], "latent_scale_schedule": [ 0, 0 ], "latent_scale_template": "", "init_scale_template": "", "steps_template": "", "style_strength_template": [ 0.2, 0.5, 0.55, 3 ], "flow_blend_template": [ 0.999, 0.0, 0.55, 3 ], "make_schedules": false, "normalize_latent": "init_frame_offset", "normalize_latent_offset": 0, "colormatch_frame": "off", "use_karras_noise": false, "end_karras_ramp_early": false, "use_background_mask": true, "apply_mask_after_warp": true, "background": "init_video", "background_source": "red", "mask_source": "init_video", "extract_background_mask": true, "mask_video_path": "", "negative_prompts": { "0": [ "nakked, nsfw, nude, text, logo, cropped, two heads, four arms, lazy eye, blurry, unfocused" ] }, "invert_mask": false, "warp_strength": 1, "flow_override_map": [], "cfg_scale_schedule": [ 15 ], "respect_sched": true, "color_match_frame_str": 1, "colormatch_offset": 0, "latent_fixed_mean": 0.0, "latent_fixed_std": 0.9, "colormatch_method": "LAB", "colormatch_regrain": false }

Comments

gary chan

that is just amazing, keep up with the good stuff !

Joel Gautraud

Is diffusion performance increased by ~30% in terms of speed? Or just less memory required? Or both?