Home Creators Posts Import Register Favorites Logout
haven't archived this post yet. have a subscription? use the importer!

Content

Notebook: https://www.patreon.com/posts/74899007


Have a nice week everyone! Here's a new version with some interesting tweaks and fixes.

The changelog:

  • add patchmatch inpainting for inconsistent areas
  • add warp towards init (thanks to Zippika from deforum team)
  • add grad with respect to denoised latent, not input (4x faster) (thanks to EnzymeZoo from deforum team)
  • add init/latent scale towards real frame option (thanks to Zippika from deforum team)
  • add json schedules (thanks to brbbbq)
  • add settings comparison
  • save output videos to a separate video folder (thanks to Colton)
  • fix xformers not loading until restart


Patchmatch inpainting

stable-settings->patchmatch_inpainting

You can now inpaint inconsistent areas via patchmatch algorithm. Really slow and haven't really impressed me, but I will leave it there for you to test. 

Warp towards init

(thanks to Zippika from deforum team)

stable-settings->warp_towards_init

This options calculates flow between real frame and stylized frame and warps stylized frame back too real frame. May produce really interesting results, though the 2nd frame will be messy because of the large difference between real and stylized frames, but after it will be okay because the frames won't be that different. 

grad with respect to denoised latent

(thanks to EnzymeZoo from deforum team)

stable-settings->grad_denoised

Calculates gradients with respect to denoised latent, not input latent as it was before. Works x4 faster and better.

init/latent scale towards real frame

 option (thanks to Zippika from deforum team)

stable-settings->init_grad

Add option to calculate init/latent scale against real frame, not warped previous frame.

json schedules

(thanks to brbbbq)

Schedules now support json (dictionary) format, like the one used in discodiffusion

Format example: 

{
 0: 1,
 10: 10,
 50: 5
}

will use value of 1 for 0-9 frames, value of 10 for 10-49 frames, and value of 5 after 50 frames. 

You can enable blend_json_schedules to blend intermediate frames' values,  so that in the example above the first 10 frames will have values from 1 to 10.


You can use list schedules as well. 

settings comparison

Compare settings->

Insert paths to two settings.txt files to compare values. It will print the differences and missing/new keys.

save output videos to a separate video folder 

(thanks to Colton)
Output videos are now saved to /video folder insire your frames' output folder.

Fixes

  • fix xformers not loading until restart

Notebook: https://www.patreon.com/posts/74899007

Files

Previews only

Stable WarpFusion v0.5.21 Nightly - init/frame scale fixes

Notebook: https://www.patreon.com/posts/74899593 Settings: { "text_prompts": { "0": [ "a beautiful breathtaking highly-detailed intricate portrait painting of a young woman with long hair, against a backdrop of stars in the style of alphonse mucha and ilya kuvshinov and peter mohrbacher, ross tran rossdraws, watercolor, featured on artstation, 70mm, rendered in octane" ], "550": [ "a highly detailed matte painting of a robot, No Man's Sky Screenshot, tall grass field, broken machinery, Simon Stalenhag , featured on Artstation." ], "1000": [ "a beautiful breathtaking highly-detailed intricate portrait painting of a blissful ignorant enlightened young sxelaqwertyop dancing against a backdrop of stars in the style of alphonse mucha and ilya kuvshinov and peter mohrbacker, ross tran rossdraws, watercolor, featured on artstation, 70mm, rendered in octane" ], "1500": [ "a highly detailed matte painting of a robot sxelaqwertyop, No Man's Sky Screenshot, tall grass field, broken machinery, Simon Stalenhag , featured on Artstation." ] }, "user_comment": "testing cc layers", "image_prompts": {}, "range_scale": 0, "sat_scale": 0, "max_frames": 187, "interp_spline": "Linear", "init_image": "", "clamp_grad": true, "clamp_max": 2, "seed": 4275770367, "fuzzy_prompt": false, "rand_mag": 0.05, "eta": 0.55, "width": 1280, "height": 704, "diffusion_model": "stable_diffusion", "diffusion_steps": 1000, "video_init_path": "/content/Untitled_design_11.mp4", "extract_nth_frame": 1, "flow_video_init_path": null, "flow_extract_nth_frame": 2, "video_init_seed_continuity": false, "turbo_mode": false, "turbo_steps": "3", "turbo_preroll": 1, "flow_warp": true, "check_consistency": true, "turbo_frame_skips_steps": null, "forward_weights_clip": 0, "forward_weights_clip_turbo_step": 0, "padding_ratio": 0.2, "padding_mode": "reflect", "consistency_blur": 0, "inpaint_blend": 0, "match_color_strength": 0, "high_brightness_threshold": 180, "high_brightness_adjust_ratio": 0.97, "low_brightness_threshold": 40, "low_brightness_adjust_ratio": 1.03, "stop_early": 0, "high_brightness_adjust_fix_amount": 2, "low_brightness_adjust_fix_amount": 2, "max_brightness_threshold": 254, "min_brightness_threshold": 1, "enable_adjust_brightness": false, "dynamic_thresh": 50, "warp_interp": 1, "fixed_code": false, "blend_code": 0.2, "normalize_code": true, "mask_result": false, "reverse_cc_order": true, "flow_lq": true, "use_predicted_noise": false, "clip_guidance_scale": 0, "clip_type": "ViT-H-14", "clip_pretrain": "laion2b_s32b_b79k", "missed_consistency_weight": 1, "overshoot_consistency_weight": 1, "edges_consistency_weight": 1, "style_strength_schedule": [ 0.3, 0.2 ], "flow_blend_schedule": [ 0.9 ], "steps_schedule": { "0": 50, "1": 50 }, "init_scale_schedule": [ 0, 10000 ], "latent_scale_schedule": [ 0, 100 ], "latent_scale_template": "", "init_scale_template": "", "steps_template": "", "style_strength_template": [ 0.2, 0.5, 0.55, 3 ], "flow_blend_template": [ 0.75, 0.9, 0.55, 3 ], "make_schedules": false, "normalize_latent": "init_frame_offset", "normalize_latent_offset": 0, "colormatch_frame": "off", "use_karras_noise": false, "end_karras_ramp_early": false, "use_background_mask": true, "apply_mask_after_warp": true, "background": "init_video", "background_source": "red", "mask_source": "init_video", "extract_background_mask": true, "mask_video_path": "", "negative_prompts": { "0": [ "text, logo, cropped, two heads, four arms, lazy eye, blurry, unfocused" ] }, "invert_mask": false, "warp_strength": 1, "flow_override_map": [], "cfg_scale_schedule": [ 15 ], "respect_sched": true, "color_match_frame_str": 1, "colormatch_offset": 0, "latent_fixed_mean": 0.0, "latent_fixed_std": 0.9, "colormatch_method": "LAB", "colormatch_regrain": false, "warp_mode": "use_image", "use_patchmatch_inpaiting": 0, "blend_latent_to_init": 0, "warp_towards_init": "off", 'init_grad':True, 'grad_denoised':True }

Comments

No comments found for this post.