Stable WarpFusion v0.34 - VidToMe, variable cfg, adiff, rec noise updates (Patreon)

Local install guides:
Windows: https://github.com/Sxela/WarpFusion#local-installation-guide-for-windows-venv
Linux: https://github.com/Sxela/WarpFusion/blob/main/README.md#local-installation-guide-for-linux-ubuntu-2204-venv

If something's not working, make a new env and grab a fresh install.bat from that repo first, ask questions later :D

Changelog: 

• fix predicted noise randomness not working in animatediff mode

• turn off depth logs

• fix controlnets and ipadapters in 'stylized' source mode sometimes using raw init frame during the 1st frame render

• fix ip-adapter_sd15_light URL

• fix animatediff mode only processing the last scene thanks to usergenic issue #105

• fix animatediff not working with scenes smaller than the context size

• fix mask extraction error in animatediff mode thanks to #thenightsfall1

• add mask in animatediff mode (applied after the render)

• enable rec noise in animatediff mode with batch length > context length

• remove error trace when exporting video with no available audio

• add ToMe 3d to animatediff mode

• fix rec noise in animatediff mode with batch length > context length triggered on the long total length

• fix rec noise in animatediff mode for multiple batches

• disable guidance during noise reconstruction

• add VidToMe

• fix VidToMe not working with batch_length > context_length

• add CFG scheduling during diffusion

• add rec noise blending in animatediff mode with context_length < batch_length

• add sliding context for rec noise in animatediff mode

• add dev_settings for temporary settings

• add dev_settings comparison to compare settings cell

• add ffmpeg downloader to install.bat

• add VidToME to GUI

• add delayed_adapter_apply to apply ipadapters after controlnet preprocessing

0.34.1, 3.6.24:

• import vidtome only when it's on

• fix ipadapter not working on colab

28.7.2024

• move to L tier

Animatediff mode updates 

in animatediff mode: GUI -> animatediff -> rec_sliding_ctx

Added rec_sliding_ctx - sliding context for noise reconstruction in animatediff mode. You can now use batch size > context length with it. Keep on by default.

in animatediff mode: GUI -> animatediff -> stop_early_adiff

Added stop_early_adiff. You can use it to eject animatediff motion module after some diffusion steps and continue as img2img to add more detail at the cost of flickering. You can also use use_vidtome_with_adiff together with this setting to apply VidToMe after ejecting animatediff motion module

VidToMe
More about it here - VISION-SJTU/VidToMe: [CVPR 2024] VidToMe: Video Token Merging for Zero-Shot Video Editing (github.com)

in animatediff mode: GUI -> animatediff -> VidToMe group

use_vidtome - enable VidToMe
use_vidtome_with_adiff - used together with stop_early_adiff - enable to apply VidToMe after ejecting animatediff motion module. need to disable use_vidtome, or it will be applied during the whole diffusion process
vidtome_rand - randomness
vidtome_local_merge - % of tokens to merge per frame
vidtome_global_merge - % of tokens to merge across 4 frames
vidtome_max_down - max downscaling layers to be affected by VidToMe
1 means only merge tokens at full resolution layers, 2 - full+half, and so on.
vidtome_control - merge controlnet tokens


VidToMe (Video Token Merging) is used to merge attention tokens to smooth video and lower vram usage. It's still inferior to animatediff or traditional warp in terms of flickering, but can provide interesting ebsynth-like results when pushed to its limits. it can also be used with animatediff at the end of diffusion to add more detail at the cost of some flicker :D

Variable CFG Scale

You can now schedule cfg scale during the diffusion process. It will be automatically interpolated to your number of steps.

Formats:
[7] - 7 for the 0th frame and the rest
[7,5] - 7 for the 0th frame, 5 for the rest.

{0:7} - 7 for the 0th frame and the rest
{0:7, 1:5} - 7 for the 0th frame, 5 for the rest.
{0:7, 10:5} - 7 for 0-9 frames, 5 for frame 10 and the rest.

Instead of a single number value like 7, you can now use a list like [3,7,4] which will be interpolated used during the diffusion process like this: [4, 4.316, 4.632, 4.947, 5.263, 5.579, 5.895, 6.211, 6.526, 6.842, 6.789, 6.368, 5.947, 5.526, 5.105, 4.684, 4.263, 3.842, 3.421, 3.0] for 20 steps.
Value is bein taken from the end of the list, so the 1st step gets cfg scale = 3, the second - 3.421 and so on.

This allows to get stronger prompt adherence without oversaturation, especially with reconstructed noise.

Reconstructed Noise Updates 

in animatediff mode: GUI -> animatediff -> rec_sliding_ctx

Added rec_sliding_ctx - sliding context for noise reconstruction in animatediff mode. You can now use batch size > context length with it. Keep on by default.
Guidance has also been disabled for rec noise, so you can use latent\image scale with it.

GUI -> contronet tab -> delayed_adapter_apply

Added delayed_adapter_apply, so that ipadapters are applied after rec noise is calculated. Before that most ipadapters were unusable with reconstructed noise. Keep on by default.

Local install guide:
https://github.com/Sxela/WarpFusion/blob/main/README.md

Guides made by users:

• 05.05.2023, v0.10 Video to AI Animation Tutorial For Beginners: Stable WarpFusion + Controlnet | MDMZ

• 11.05.2023, v0.11 How to use Stable Warp Fusion

• 13.05.2023, v0.8 Warp Fusion Local Install Guide (v0.8.6) with Diffusion Demonstration

• 14.05.2023, v0.12 Warp Fusion Alpha Masking Tutorial | Covers Both Auto-Masking and Custom Masking

• 23.05.2023, v0.12 STABLE WARPFUSION TUTORIAL - Colab Pro & Local Install

• 15.06.2023, v0.13 AI Animation out of Your Video: Stable Warpfusion Guide (Google Colab & Local Intallation)

• 17.06.2023, v0.14 Stable Warpfusion Tutorial: Turn Your Video to an AI Animation

• 21.06.2023, v0.14 Avoiding Common Problems with Stable Warpfusion

• 21.06.2023, v0.15 Warp Fusion: Step by Step Tutorial

• 04.07.2023, v0.15 Intense AI Video Maker (Stable WarpFusion Tutorial)

• 15.08.2023, v0.17 BEST Laptop for AI ( SDXL & Stable Warpfusion ) ft. RTX 4090 - Make AI Art FREE and FAST!

• 25.08.2023, ComfyWarp v0.1 WarpFusion: Warp and Consistency explanation in ComfyUI

• 2.09.2023, ComfyWarp v0.2 WarpFusion: ComfyWarp iteration 2.

• 3.09.2023, v0.16+ WarpFusion - Multiple Masked Prompts Guide

• 10.10.2023, ComfyWarp v0.4 WarpFusion: ComfyWarp v0.4.2 (schedulers, flow_blend)

• 4.11.2023, ComfyWarp v0.5 WarpFusion: ComfyWarp v0.5 - FixedQueue

• 13.11.2023, v0.27 Stable WarpFusion v0.27 - Changelog

• 2.03.2024 v0.16 Create a INSANE Video Using AI Animation | Stable Warpfusion (youtube.com)

YouTube playlist with settings:
https://www.youtube.com/watch?v=wvvcWm4Snmc&list=PL2cEnissQhlCUgjnGrdvYMwUaDkGemLGq

Setting on GitHub: 
WarpFusion/examples at main · Sxela/WarpFusion (github.com)

For tech support and other questions please join our discord server:
https://discord.gg/58Rd73uAe7

Discord is the preferred method because it is nearly impossible to provide any decent help or tech support via Patreon due to its limited text formatting and inability to add screenshots or videos to comments or DMs.
Due to the recent Patreon comments update it's impossible to reply to comments from notifications anymore, so if your comment hasn't been replied to for a while, DM me.