Home Creators Posts Import Register Favorites Logout
haven't archived this post yet. have a subscription? use the importer!

Downloads

  • stable_warpfusion_v0_5_27_10_embeddings.ipynb
  • LICENSE.MD

Missing 1 full-res photo, 2 files, as this post has not yet been imported.

Content

Lo and behold: Embeddings are now supported!

Changelog:

  • add embeddings, prompt attention weights from AUTOMATIC1111 webui repo
  • bring back colab gui for compatibility
  • load default settings option
  • add frame difference manual override for content-aware scheduling
  • fix PDF color transfer being used by default with LAB selected
  • fix content-aware schedules


Embeddings

Added embeddings support from the awesome AUTOMATIC1111 repo.

Place embeddings into embeddings folder that should be ccreated in your warp folder. Embeddings are loaded right after the model itself, so if you add some more files mid-run, make sure to reload the notebook. 

You can use embeddings by specifying their filename in the prompt. For example, if you have some_embedding.pt, add some_embedding to your prompt as text to call it. 


Prompt attention weights

Added keyword weights support from AUTOMATIC1111 repo.
You can weight keywords and embeddings as well with the following syntax:
(someword:1.1) - increases the weight of that word by 10%,
(someword:0.9) - decreases the weight by 10%.
You can also use () without a number to increase the weight by 10%, or use [] without a number to decrease the weight by 10%


Colab gui is back

Brough back the old colab gui (by popular demand). The new gui is still there, but now its defaults are taken from the old gui, so you can see what defaults you're gonna get.

Load default setings

GUI -> default_settings_path

You can now specify the default settings file that will be loaded. It will override the old-fashioned gui settings.


Frame difference manual override

Content-aware scheduling -> Create schedules from frame difference -> diff_override

You can override the keyframes that were inffered during frame difference analysis with your own list of keyframes, for example obtained from audio to keyframe script.


Fix default color transfer
The default color transfer is now LAB, as is stated in the interface (was PDF even if LAB was selected)

Fix content-aware schedules
Content-aware schedules were applied before the GUI and were effectively overridden by the GUI.

Local install guide:
https://github.com/Sxela/WarpFusion/blob/main/README.md

Quickstart guide if you're new to google colab notebooks: https://docs.google.com/document/d/1gu8qJbRN553SYrYEMScC4i_VA_oThbVZrDVvWkQu3yI

A reminder:

Changes in GUI will not be saved into the notebook, but if you run it with new settings, they will be saved to a settings.txt file as usual.

You can load settings in misc tab.

You do not need to rerun the GUI cell after changing its settings.

A more thorough guide (a bit outdated, but covers most of the warp-specific stuff and local\cloud install): https://docs.google.com/document/d/11xxHyvkCBBUwT73lWHQx-T_FU_rC7HWzNylftyCExcE

For tech support and other questions please join our discord server: https://discord.gg/YrpJRgVcax

Discord is the preferred method, because it is nearly impossible to provide any decent help or tech support via Patreon due its limited text formatting and inability to add screenshots or videos to comments or DMs.

upd:
- fix xformers for a100
- remove PatchMatch to fix jacinle/travis error on colab 

  • print settings on error while saving settings :D
  • reload embeddings in Diffuse! cell (you can add new without reloading the whole model)
  • fix pkl not saveable after loading embeddings
  • fix xformers install (no need to downgrade torch thanks to TheFastBen)
  • add ffmpeg instal for windows
  • add torch install/reinstall options for windows
  • add xformers install/reinstall for windows
  • disable shutdown runtime for non-colab env
  • fix error saving settings with content aware schedules
  • fix legacy normalize latent = first latent error
  • fix xformers install in colab A100
  • fix 'load default settings' not working in GUI cell
  • fix mask_clip not loading from settings
  • fix error with loaded settings' schedules 
  • fix another error with loaded schedules ("0" vs 0)


Files

Previews only

Beastie Boys - Sabotage feat WarpFusion v0.5.27

Settings: { "text_prompts": { "0": [ "a beautiful highly detailed cyberpunk 2077 city, pixelart-hard" ] }, "user_comment": "testing cc layers", "image_prompts": {}, "range_scale": 0, "sat_scale": 0.0, "max_frames": 4312, "interp_spline": "Linear", "init_image": "", "clamp_grad": true, "clamp_max": 3.0, "seed": 4275770367, "width": 1344, "height": 832, "diffusion_model": "stable_diffusion", "diffusion_steps": 1000, "video_init_path": "init_images/video_2023-01-11_22-56-58.mp4", "extract_nth_frame": 1, "flow_video_init_path": null, "flow_extract_nth_frame": 2, "video_init_seed_continuity": false, "turbo_mode": false, "turbo_steps": 3, "turbo_preroll": 1, "flow_warp": true, "check_consistency": true, "turbo_frame_skips_steps": null, "forward_weights_clip": 0.0, "forward_weights_clip_turbo_step": 0.0, "padding_ratio": 0.2, "padding_mode": "reflect", "consistency_blur": 1.0, "inpaint_blend": 0, "match_color_strength": 0.0, "high_brightness_threshold": 180, "high_brightness_adjust_ratio": 0.97, "low_brightness_threshold": 40, "low_brightness_adjust_ratio": 1.03, "stop_early": 0, "high_brightness_adjust_fix_amount": 2, "low_brightness_adjust_fix_amount": 2, "max_brightness_threshold": 254, "min_brightness_threshold": 1, "enable_adjust_brightness": false, "dynamic_thresh": 30.0, "warp_interp": 1, "fixed_code": false, "blend_code": 0.1, "normalize_code": true, "mask_result": false, "reverse_cc_order": true, "flow_lq": true, "use_predicted_noise": false, "clip_guidance_scale": 0, "clip_type": "ViT-H-14", "clip_pretrain": "laion2b_s32b_b79k", "missed_consistency_weight": 1.0, "overshoot_consistency_weight": 1.0, "edges_consistency_weight": 1.0, "style_strength_schedule": [0.4, 0.2] "flow_blend_schedule": [ 0.7 ], "steps_schedule": [ 50 ], "init_scale_schedule": [ 0, 0, 0, 0, 5000 ], "latent_scale_schedule": [ 0, 0, 0, 0, 100 ], "latent_scale_template": "", "init_scale_template": "", "steps_template": "", "style_strength_template": [ 0.2, 0.5, 0.42, 2 ], "flow_blend_template": null, "make_schedules": true, "normalize_latent": "off", "normalize_latent_offset": 0, "colormatch_frame": "off", "use_karras_noise": false, "end_karras_ramp_early": false, "use_background_mask": false, "apply_mask_after_warp": true, "background": "init_video", "background_source": "red", "mask_source": "none", "extract_background_mask": false, "mask_video_path": "", "negative_prompts": { "0": [ "text, naked, nude, logo, cropped, two heads, four arms, lazy eye, blurry, unfocused" ] }, "invert_mask": false, "warp_strength": 1.0, "flow_override_map": [], "cfg_scale_schedule": [ 15 ], "respect_sched": true, "color_match_frame_str": 0.2, "colormatch_offset": 0, "latent_fixed_mean": 0.0, "latent_fixed_std": 0.9, "colormatch_method": "LAB", "colormatch_regrain": null, "warp_mode": "use_image", "use_patchmatch_inpaiting": 0.0, "blend_latent_to_init": 0.0, "warp_towards_init": "off", "init_grad": false, "grad_denoised": true, "colormatch_after": false, "colormatch_turbo": false, "model_version": "v2_512", "depth_source": "init", "warp_num_k": 128, "warp_forward": false, "sampler": "sample_euler", "mask_clip": [ 0, 255 ], "inpainting_mask_weight": 1.0, "inverse_inpainting_mask": false, "model_path": "v2-1_512-ema-pruned.ckpt" }

Comments

Pierre Farah

my runtime keeps stopping at "Install and import dependencies" claiming import error. what could be the reason?

Pierre Farah

Oh.. which version should I try?