Minimax H3 R2V VDN Multi-Stages 4K Pipeline 20260905 — For Patreon Supporters (Patreon)

Hey everyone!

The public post has the two free test workflows https://www.patreon.com/aifuturetech/posts/vdn-h3-in-faster-168607785 — the text-to-video A/B and the three-branch reference testlab.

This post has the full production version I ran for the final demo: a two-stage reference-to-video pipeline where stage 1 runs VDN hybrid attention with NO community Turbo LoRA, and stage 2 adds the Turbo LoRA only for the latent upscale pass. I also attached the actual output video from this workflow so you can judge stage 2 quality before wiring anything.

Why the split matters: the Turbo LoRA is a distillation. It buys speed by pushing the model away from its original weights, and fast-motion areas pay for that with data loss. VDN-H3 gets its 8-step speed a different way — it patches attention itself (windowed softmax for nearby frames, a linear delta-rule branch for distant frames) and barely touches the base weights. So stage 1 gives you clean motion that still looks like the base model, and the only place we accept a distilled pass is stage 2, a light 1-step re-sample whose whole job is upscaling.

----------------------------------------

WHAT'S INCLUDED

----------------------------------------

File attached:

MiniMax-H3 R2V VDN (Full) -20260905 and the Example Prompt.

Plus, as media: the finished demo clip (vid-vfi_00046-audio.mp4) and a workflow screenshot from this exact run.

1. VDN-H3 Two-Stage R2V (62 nodes + Latent Upscaler subgraph)

The demo is a Mortal Kombat style fatality scene: two character sheets wired as references (CharSheet_00157_.png and CharSheet_00158_.png), an 8-shot Ref2VA prompt, 10 seconds at 16:9, 0.8 megapixels stage 1.

Stage 1 — VDN hybrid attention, no Turbo LoRA:

- UNETLoader: minimax-h3\minimax_h3_ref2va_int8_convrot.safetensors

- Realism LoRA (the only LoRA in this stage): h3-realism-people-t2v-i2v-r2v (r34l1sm).safetensors at strength 1, with the trigger prefix r34l1sm , ultra realistic video. prepended to the prompt

- ApplyVDNH3Advanced: vdn_checkpoint stage-dmd-step-250, apply_turbo_adapter ON (that is the VDN checkpoint's own 8-step adapter — NOT the community Turbo LoRA), stage_b_strength 1, turbo_strength 1, lora_mode merge, branch_weights stream, attention_backend grouped, window_radius 1, window_chunk 5, anchor_frames both, text_state ON, linear_branch ON, fast_kernels OFF

- MiniMaxChunkFeedForward: 2 / 4096

- MiniMaxH3SigmaShift: 12 / 3

- Sampling: SamplerCustomAdvanced, euler, simple scheduler, 8 steps, CFG 1 (guider, no negative branch in stage 1), seed 42 fixed

- MiniMaxH3ReferenceToVideo at 24 fps; frame length auto-rounds your duration input (10s default) up to satisfy the 17-frame chunk alignment

- Reference images pass through ImageResizeKJv2 at 1024x1024 with nvidia_rtx_vsr padding

- No SageAttention patch in this stage — deliberate, see tip 2

Stage 2 — Turbo LoRA for the latent upscale:

- Inside the Latent Upscaler subgraph, the model gets minimax_h3_ref2v_turbo_4step_v0.1_comfyui_bf16.safetensors at strength 1 — this is the ONLY place the community Turbo LoRA touches the pipeline

- MMH3LatentUpscaleWithModelParams: minimax_h3_latent_upscaler_3d_bf16.safetensors, target 1280x704, cuda, bf16, keep_models_resident True

- Spatial split 864x480 tiles with 128 overlap, temporal split 85 frames / 17 overlap / 0.999 fade

- Re-sample: sa_solver, 1 step, denoise 0.2, SigmaShift 12 / 3, seed 42 — a touch pass to re-anchor detail, not a regeneration

- Conditioning width and height are computed from stage 1 output times two, so they stay in sync automatically

- PathchSageAttentionKJ: auto (safe here — VDN is not applied in stage 2)

Finish:

- Video VAE fp16 + audio VAE fp32 decode, audio carried through untouched

- FrameInterpolate: FILM (film_net_fp16.safetensors) x2

- RTXVideoSuperResolution: multiplier 2, quality ULTRA

- Export: 48 fps h264, CRF 19, prefix FULL-THROTTLE/vid-vfi

- Stage 1 and mid-stage encodes save to temp only (save_output off) so your output folder stays clean

----------------------------------------

WHAT YOU NEED

----------------------------------------

Model files:

models/minimax-h3/

- minimax_h3_ref2va_int8_convrot.safetensors

- qwen3vl_32b_minimax_h3_int8_convrot.comfy.safetensors (CLIPLoader, type: minimax)

- minimax_h3_video_vae_fp16.safetensors

- minimax_h3_audio_vae_fp32.safetensors

- minimax_h3_ref2v_turbo_4step_v0.1_comfyui_bf16.safetensors (stage 2 only)

- h3-realism-people-t2v-i2v-r2v (r34l1sm).safetensors https://huggingface.co/fal/MiniMax-H3-Realism-People-LoRA/blob/main/h3-realism-people-t2v-i2v-r2v.safetensors

Source: https://huggingface.co/Comfy-Org/MiniMax-H3

models/vdn/

- stage-dmd-step-250/ (the whole directory — model_spec.json, linear_branch/, adapters/)

Download: hf download OpenVDN/vdn-minimax-h3 --include "stage-dmd-step-250/*" --local-dir <ComfyUI>/models/vdn

models/latent_upscale_models/

- minimax_h3_latent_upscaler_3d_bf16.safetensors

Source: https://huggingface.co/LBH-123-AI/Minimax_h3_latent_Upscaler

models/upscale_models/ (or wherever FILM lives in your setup)

- film_net_fp16.safetensors

Custom nodes:

Video DeltaNet: Hybrid Attention to Speed Up Video Models with Near-Lossless Quality

vdn-minimax-h3

https://github.com/OpenVDN/vdn-minimax-h3

Weights

https://huggingface.co/OpenVDN/vdn-minimax-h3

ComfyUI-VDN-H3

https://github.com/Saganaki22/ComfyUI-VDN-H3

Install

Clone into ComfyUI/custom_nodes/ and restart ComfyUI:

cd ComfyUI/custom_nodes

git clone https://github.com/Saganaki22/ComfyUI-VDN-H3

Download the VDN checkpoint stage you want into ComfyUI/models/vdn/:

hf download OpenVDN/vdn-minimax-h3 --include "stage-dmd-step-250/*" --local-dir <ComfyUI>/models/vdn

Keep the release directory layout intact (model_spec.json, linear_branch/, adapters/). Nothing is converted on disk — the node re-keys the diffusers-format tensors onto ComfyUI module paths in memory.

No new Python dependencies. The node runs the official math in eager PyTorch that ships with ComfyUI (torch + safetensors). No Triton, no flash-attn-4, no CUDA builds, no pip install.

GPU:

- No hard VRAM floor. Run the MiniMax-H3 base as a GGUF quant (Q4/Q5 fit 12-16GB cards easily) and let ComfyUI Dynamic VRAM handle the rest — the VDN linear branch and the stage 2 upscaler swap in and out instead of blowing up your job. What you pay on a smaller card is time, not out-of-memory. - On tighter cards expect slower generation: every step runs the VDN branch on top of the quantized model, and Dynamic VRAM adds offload transfers between stages. The clearCache / cleanGpu nodes between stages keep that swapping tidy — leave them wired.

- RTX card required for RTX Video Super Resolution. If you are not on RTX, bypass that node — FILM output is fine.

----------------------------------------

QUICK START

----------------------------------------

1. Install ComfyUI-VDN-H3 and the latent upres fork, restart ComfyUI.

2. Download the VDN stage-dmd-step-250 directory into models/vdn — the whole folder, not just the 4.3 GB linear branch file.

3. Confirm every model file above exists, then import the JSON. Nothing should be red.

4. Wire your two character sheets into the two VHS_LoadImagePath nodes, in the order your prompt's Picture 1 / Picture 2 expect.

5. Replace the demo prompt with your own Ref2VA prompt (subject_definitions / summary / retention_analysis / detailed_description / soundscape). Keep the r34l1sm trigger prefix.

6. Set Duration (default 10s) and ResolutionSelector (default 16:9 at 0.8 MP). Queue and leave the cache nodes alone.

7. You get three files: stage 1 at 24 fps, stage 2 at 24 fps, and the final vid-vfi clip at 48 fps. Compare stage 1 against stage 2 before you trust any single one.

----------------------------------------

TIPS

----------------------------------------

1. Do NOT add the community Turbo LoRA to stage 1. That is the entire design of this pipeline. The 8-step speed in stage 1 comes from the VDN turbo adapter, and the VDN README is explicit that the two distillation routes replace each other — stacking a Turbo LoRA on a stage-dmd checkpoint re-introduces exactly the data loss VDN is here to avoid. Realism LoRA at 1 is fine, it is not a distillation.

2. Keep SageAttention out of stage 1. VDN patches the attention forward pass itself; if another attention patch is applied first, VDN replaces it and you get a confusing hybrid. The workflow applies Sage only inside the stage 2 subgraph, where there is no VDN on the model.

3. lora_mode merge is required on stage-dmd checkpoints, not a preference. Bypass applies the adapter deltas in activation space, bf16 rounding noise grows to about 10 percent of feature magnitude by block 49, and the 8-step output visibly degrades. Merge folds the deltas into the weights and reproduces the validated model exactly.

4. branch_weights: stream is shipped because it is safe on smaller cards (weights stream to GPU per block per step). If you have ~4.3 GB VRAM to burn, cache_gpu is meaningfully faster on long renders.

5. The 17-frame alignment on the duration expression is not optional — it keeps the latent frame count compatible with the chunk/window geometry. Do not simplify it to duration*24 even if that divides nicely.

6. Stage 2 at 1 step / denoise 0.2 is intentional. The upscale pass should re-anchor detail on latents VDN already sampled cleanly. Push denoise above ~0.3 and you pay for it in drift from stage 1 motion — which defeats the point of running VDN in the first place.

7. If you want pure bf16 quality on the VDN branch (no int8 base), swap the UNETLoader to the bf16 stage and keep everything else wired. On the 5090 benchmark rig, bf16 stage ran about 111 s and int8 convrot about 95 s for an 8-step 1280x736/61-frame clip. Both are near-lossless versus dense H3 — that fidelity is why stage 1 can skip the Turbo LoRA entirely.

8. fast_kernels OFF as shipped. It torch-compiles the branch hot spots (same math, verified unit tests), first run pays compile time. Flip it ON if you batch-render the same config repeatedly.

----------------------------------------

COMMON ISSUES

----------------------------------------

"ApplyVDNH3Advanced dropdown is empty / says place a VDN stage directory under models/vdn"

→ The node wants the stage FOLDER (model_spec.json + linear_branch/ + adapters/ together) inside models/vdn. A loose .safetensors or a nested stage-dmd-step-250/stage-dmd-step-250 layout both read as empty.

"This MODEL already has VDN-H3 applied"

→ You chained the VDN node twice (usually by re-wiring an old node). VDN applies once per model. Check the model chain: UNET → realism LoRA → ApplyVDNH3Advanced → ChunkFeedForward → TorchSettings → SigmaShift → Guider.

"stage-dmd output looks degraded / noisy"

→ Almost always lora_mode bypass. Set it back to merge (tip 3). The community distillation noise and the DMD 8-step schedule do not mix.

"Generation is slow on my card"

→ Normal, and the honest trade of this pipeline. The VDN branch adds compute every step, and if you are on a GGUF quant with Dynamic VRAM, the offload transfers between stage 1 and stage 2 add more. Options in order of impact: drop stage 1 to 0.5-0.6 MP, keep branch_weights on stream (cache_gpu needs ~4.3 GB free — only reach for it when you have the room), or run the two stages as separate queues so the caches fully flush between them. You should not be hitting actual OOM with a quant + Dynamic VRAM; if you still crash, check that the easy clearCacheAll nodes are wired and active.

"Second pass drifts from first pass / faces change"

→ Denoise snuck above 0.2, or the subgraph width/height link got broken and the conditioning no longer equals stage 1 size x2. Re-check both before changing anything else.

"Final video looks stuttery"

→ Check the output combine frame_rate is wired to src_fps x2 (48). The FILM x2 doubles frames; rendering that clip metadata at 24 plays it at half speed.

----------------------------------------

The honest bottom line after a month on this: VDN in stage 1 leans harder on compute than the Turbo LoRA route, and on a quantized model with Dynamic VRAM that surfaces as slower generation rather than OOM — the payoff is fidelity. Fast motion keeps its detail because the original weights are still doing the thinking. That is the trade this workflow is built around.

Questions? Drop them in the comments and I will help you sort it out.

Thanks for supporting the channel. You guys make these deep dives possible.

— Benji