Stable WarpFusion v0.8.0 Nightly - Masked guidance/diffusion (Patreon)

An update to celebrate the end of winter :D

This update requires running it in a new folder, or to delete k-diffusion folder in you current warp dir.

Changelog:

New:

• add masked diffusion callback
• add masked latent guidance
• add option to offload model before decoder stage
• add fix noise option for latent guidance
• add noise, noise scale, fixed noise to masked diffusion

Fixes:

• fix frame_range starting not from zero not working thanks to Oleg#8668

Upd:

• add controlnet_preprocessing switch to allow raw input
• fix sampler being locked to euler
• add separate base model for controlnet support
• add smaller controlnet support
• add invert mask for masked guidance
• add use_scale options to use loss scaler (guidance seems to work faster)

Separate base model for ControlNet / small controlnet support

You can now specify any v1.x model checkpoint with any of controlnet_v1.5_* model_version. It will assume that the checkpoint is correct and load it as the base of the controlnet model. It will then look for a small controlnet, and download it if it's not being found.

Masked diffusion

You can now use masked diffusion to stylize masked areas for more steps compared to the whole image. The mask source for now is the consistency mask.

Stable-settings -> Non-GUI -> mask_callback

mask_callback:

0 - off. 0.5-0.7 are good values. Value is a % of actual diffusion steps being made.
Diffuse inconsistent area for only before this % of actual steps, then diffuse whole image. So With 50 steps, 0.5 strength and 0.7 mask_callback you will diffuse the masked area for 50*0.5*0.7  = 17 steps, and the wholde image will be diffused for 50*0.5*(1-0.7) = 8 more steps to smoothen the transition.

cb_noise_upscale_ratio - noise upscale in masked diffusion callback

cb_add_noise_to_latent - add noise to latent in masked diffusion callback

cb_use_start_code - fix noise per frame in masked diffusion callback

cb_fixed_code - fix noise across all animation in masked diffusion callback (overcooks fast af)

cb_norm_latent - normalize latent stats

Masked guidance

Stable-settings -> Non-GUI -> masked_guidance

Use mask for init/latent guidance to ignore inconsistencies and only guide based on the consistent areas

guidance_use_start_code - fix noise per frame for guidance (default - True)

A reminder:

Changes in GUI will not be saved into the notebook, but if you run it with new settings, they will be saved to a settings.txt file as usual.

You can load settings in misc tab.

You do not need to rerun the GUI cell after changing its settings.

Local install guide:
https://discord.com/channels/973802253204996116/1067887206125015153/1067888756910215278
https://github.com/Sxela/WarpFusion/blob/main/README.md

Youtube playlist with settings:
https://www.youtube.com/watch?v=wvvcWm4Snmc&list=PL2cEnissQhlCUgjnGrdvYMwUaDkGemLGq

For tech support and other questions please join our discord server:
https://discord.gg/sayE6j2sdP

Discord is the preferred method, because it is nearly impossible to provide any decent help or tech support via Patreon due its limited text formatting and inability to add screenshots or videos to comments or DMs.
Error reports in comments will be deleted and reposted in discord.