Stable WarpFusion v0.29 - QoL fixes, wav2lip, FaceFusion installers (Patreon)

Merry Christmas Incoming!

Local install guides:
Windows: https://github.com/Sxela/WarpFusion#local-installation-guide-for-windows-venv
Linux: https://github.com/Sxela/WarpFusion/blob/main/README.md#local-installation-guide-for-linux-ubuntu-2204-venv

If something's not working, make a new env and grab a fresh install.bat from that repo first, ask questions later :D

This notebook needs an updated transformers 4.31.0 package. Make a new env with a fresh install.bat or update the package by running !pip install transformers==4.31.0 in a new code cell.

Changelog:

• add scheduling for controlnet\ipadapter sources

• add mouth mask to consistency masked diffusion and inpaint controlnet mask

• add fill_lips setting to dilate the mask

• add fill_lips to gui/settings

• rewrite PDF colormatching in pytorch (x20 faster)

• add input frames support for video init

• add input frames support for background mask source

• add input frames support for RIFE, SAMTrack

• add fast multiprompts for non-animatediff mode. visually different sometiems from the slow mode, doesn't work well with lcm lora

• move fast multiprompt option to diffuse tab

• fix uc_mask_shape error when using predicted noise

• fix masks not working in None blend_mode video export

• add FaceFusion install - https://github.com/facefusion/facefusion

• add simple cli interface for FaceFusion (gradio is still available)

• disable optical flow blend_mode for animatediff video export

• add wav2lip installer & interface - https://github.com/snpranav/wav2lip-HD-dockerized?tab=readme-ov-file

• add fill lips to saved settings

• add flow maxsize

• add flow resizing and rescaling to support flow maxsize

• fix errors in animatediff/controlnet preview only modes

24.12.2023, v0.29.1:

• add fill lips to saved settings

• add flow maxsize

• add flow resizing and rescaling to support flow maxsize

• fix errors in animatediff/controlnet preview only modes

26.12.2023, v0.29.2:

• fix unboundvariable cc error thanks to #yuraelkin

• hide pdf colormatch debug output spam

9.01.2024, v0.29.3:

• fix face controlnet error when no faces detected

• upd link to linux installer author

12.1.2024, v0.29.4:

• fix dwpose annotator weights access denied error

17.01.2024:

• move to L tier

23.01.2024, v0.29.5:

• fix face_cc error in non-controlnet model version thanks to #infinitevibes

v0.29.6, 30.01.2024:

• fix pool not running error when exporting video with missing flow files

11.4.2024:

• move to M

Flow Maxsize

Sets max size for flow maps (by the longest side, keeping the aspect ratio)
Can be useful when you render 1080p, given that the flow detection model was trained on 720p for example. Also useful if you're running OOM during flow stage.

Controlnet \ Ipadapter scheduling

You can now schedule  Controlnet \ Ipadapter source images. For example, if you need to change ipadapter styles on a keyframe basis. Uses the same syntax as all json \ list schedules.

Face Controlnet consistency mask

gui -> controlnet - controlnet annotator settings -> fill_lips

When face controlnet is enabled, the mouth area is added to the consistency mask to help with lipsync. Openpose Controlnet is still working better with lipsync as a controlnet, but an extra mouth mask helps.

Enabled by default when face controlnet is on and fill_lips > 0

Add input frames support to init video cell

You can now use frames\folders with frames\glob patterns as inputs in video init cell, background mask source, RIFE, SAMTrack.

FaceFusion

Faceswaps based on a single face reference. Works best with faces 512x512 and smaller. Added installer. You can run a fast cli with minimum settings, or launch the FaceFusion Gradio UI.

Wav2Lip

Syncs lips to audio, comes with GFPGan enhancer. Works best with faces 256x256 and smaller.

Local install guide:
https://github.com/Sxela/WarpFusion/blob/main/README.md

Guides made by users:

• 05.05.2023, v0.10 Video to AI Animation Tutorial For Beginners: Stable WarpFusion + Controlnet | MDMZ

• 11.05.2023, v0.11 How to use Stable Warp Fusion

• 13.05.2023, v0.8 Warp Fusion Local Install Guide (v0.8.6) with Diffusion Demonstration

• 14.05.2023, v0.12 Warp Fusion Alpha Masking Tutorial | Covers Both Auto-Masking and Custom Masking

• 23.05.2023, v0.12 STABLE WARPFUSION TUTORIAL - Colab Pro & Local Install

• 15.06.2023, v0.13 AI Animation out of Your Video: Stable Warpfusion Guide (Google Colab & Local Intallation)

• 17.06.2023, v0.14 Stable Warpfusion Tutorial: Turn Your Video to an AI Animation

• 21.06.2023, v0.14 Avoiding Common Problems with Stable Warpfusion

• 21.06.2023, v0.15 Warp Fusion: Step by Step Tutorial

• 04.07.2023, v0.15 Intense AI Video Maker (Stable WarpFusion Tutorial)

• 15.08.2023, v0.17 BEST Laptop for AI ( SDXL & Stable Warpfusion ) ft. RTX 4090 - Make AI Art FREE and FAST!

• 25.08.2023, ComfyWarp v0.1 WarpFusion: Warp and Consistency explanation in ComfyUI

• 2.09.2023, ComfyWarp v0.2 WarpFusion: ComfyWarp iteration 2.

• 3.09.2023, v0.16+ WarpFusion - Multiple Masked Prompts Guide

• 10.10.2023, ComfyWarp v0.4 WarpFusion: ComfyWarp v0.4.2 (schedulers, flow_blend)

• 4.11.2023, ComfyWarp v0.5 WarpFusion: ComfyWarp v0.5 - FixedQueue

• 13.11.2023, v0.27 Stable WarpFusion v0.27 - Changelog

YouTube playlist with settings:
https://www.youtube.com/watch?v=wvvcWm4Snmc&list=PL2cEnissQhlCUgjnGrdvYMwUaDkGemLGq

For tech support and other questions please join our discord server:
https://discord.gg/58Rd73uAe7

Discord is the preferred method because it is nearly impossible to provide any decent help or tech support via Patreon due to its limited text formatting and inability to add screenshots or videos to comments or DMs.
Due to the recent Patreon comments update it's impossible to reply to comments from notifications anymore, so if your comment hasn't been replied to for a while, DM me.