Stable WarpFusion v0.7.0 Nightly - ControlNet (Patreon)
Downloads
Missing 2 files, as this post has not yet been imported.
Content
An update bringing the power of ControlNetworks to Warp!
Changelog:
- add ControlNet models from https://github.com/lllyasviel/ControlNet
- add ControlNet downloads from https://colab.research.google.com/drive/1VRrDqT6xeETfMsfqYuCGhwdxcC2kLd2P
- add settings for ControlNet: canny filter ranges, detection size for depth/norm and other models
- add vae ckpt load for non-ControlNet models
- add selection by number to compare settings cell
- add noise to guiding image (init scale, latent scale)
- add noise resolution
- add guidance function for init scale
- add fixed seed option
Hotfix:
- fix image_resolution error for controlnet models
- fix controlnet models not downloading (file not found error)
- fix settings not loading with -1 and empty batch folder
- fix prettytable requirement
- fix blip generationg ccaptions for n-th frame even with a different setting
- fix load settings not working for filepath
- fix norm colormatch error
- fix warp latent mode error
- fix prompts not working for loaded settings thanks to Euclidean Plane#1332
ControlNet
define SD + K functions, load model -> model_version -> control_sd15_*
You can select one of ControlNet models. Be aware that warp settings are vastly different across all of those models.
For example, depth/normal map models work best with style strength 0.9-1.0, lower values tend to just return back the input image. Canny model on the other hand is closer to v1.5 warp settings (style strength 0.5, 0.2 is okay with it)
download_control_model:
check to download the checkpoint file for the selected ControlNet model together with it's conditioning counterpart (like midas ddept estimatior for normal map model)
force_download:
check to force overwrite existing model file
Extra settings for ControlNet were added to the stable-settings cell.
detect_resolution: size of the image being fed into ControlNet companion models like midas. Keep it the same size as you output image, or as high as you can get to it.
low_threshold and high_threshold: canny filter parameters
You can specify the source for ContolNet guidance just like you do for depth model: depth_source parameter in GUI -> diffuse tab.
You can also use video as a source for ControlNet Conitioning.
Just put it here: cond_video_path -> Video Input Settings and select GUI -> diffusion -> depth_source -> cond_video
Models tested: all.
Separate vae support
define SD + K functions, load model -> vae_ckpt
Use this to load a standalone variational autoencoder (vae) file.
Compare settings by number
Propagated the load settings fix to the compare settings cell. You can now specity only run numbers as well.
Additions to latent conditioning
This one is for people familiar the with diffusion process.
Added some options for init scale:
- Add noise corresponding to current diffusion timestep to "ground truth image" that we are comparing out generated result to
- Add noise scaling options so that the noise won't add, well, visible noise to the image :D
- Add criterion selection for init scale loss
A reminder:
Changes in GUI will not be saved into the notebook, but if you run it with new settings, they will be saved to a settings.txt file as usual.
You can load settings in misc tab.
You do not need to rerun the GUI cell after changing its settings.
Local install guide:
https://discord.com/channels/973802253204996116/1067887206125015153/1067888756910215278
https://github.com/Sxela/WarpFusion/blob/main/README.md
Youtube playlist with settings:
https://www.youtube.com/watch?v=wvvcWm4Snmc&list=PL2cEnissQhlCUgjnGrdvYMwUaDkGemLGq
For tech support and other questions please join our discord server:
https://discord.gg/sayE6j2sdP
Discord is the preferred method, because it is nearly impossible to provide any decent help or tech support via Patreon due its limited text formatting and inability to add screenshots or videos to comments or DMs.
Error reports in comments will be deleted and reposted in discord.