YuE 2 ComfyUI With LoRA Trainning— Full Run Guide (Patreon)

Yue 2 ComfyUI + LoRA Trainer — Full Run Guide

Hey everyone. If you watched the Yue 2 video https://www.patreon.com/aifuturetech/posts/yue2-generate-in-169485916 , you saw the style prompt to ABC score to full song pipeline.

This post has the extended workflow — yue2_comfyui_With_Trainer.json — which adds the LoRA trainer on top of the generation graph. Same file, two jobs: generate songs, and train Yue 2 on your own music so you can push the audio quality up, lock in a consistent style, or produce clean instrumental and background music for your content.

----------------------------------------

WHAT TO DOWNLOAD (MODEL FILES + NODES)

----------------------------------------

Custom node (required for the trainer group):

ComfyUI-YuE2-Trainer

https://github.com/Starnodes2024/ComfyUI-YuE2-Trainer

Put it in ComfyUI/custom_nodes/ (git clone or Download ZIP), then

install its requirements.txt into your ComfyUI python environment.

Restart ComfyUI after. Note: these nodes are still experimental —

rough edges are possible.

Model files (same two as the base workflow):

1. yue2_3b_bf16.safetensors

Folder: ComfyUI/models/checkpoints/

Source: https://huggingface.co/Comfy-Org/YuE2

2. sheetsage2_bf16.safetensors

Folder: ComfyUI/models/audio_encoders/

Source: https://huggingface.co/m-a-p/SheetSage2

Optional, if your VRAM is under 24 GB:

GGUF quantized weights: https://huggingface.co/audio-cpp/Yue2-3B-GGUF

(q8_0 runs on roughly 9-10 GB, q4_0 on 8-9 GB)

Other nodes used: rgthree (Any Switch), comfyui-custom-scripts (ShowText),

VideoHelperSuite (VHS_LoadAudio). Most ComfyUI installs already have these.

----------------------------------------

QUICK START — GENERATE A SONG

----------------------------------------

1. Load yue2_comfyui_With_Trainer.json

2. Check the CheckpointLoaderSimple node points to yue2_3b_bf16.safetensors

3. In the Inputs group, fill the style node and the lyrics node

4. Leave the Lora Training group bypassed (grey) — it is off by default

5. Set duration on EmptyYuE2LatentAudio if your song is longer than 2 minutes

6. Queue

If you have a trained LoRA: the LoraLoaderModelOnly node sits between the

checkpoint and the sampler. Pick your yue2_*.safetensors file from

models/loras/ and keep strength at 1. That is the whole generation loop.

----------------------------------------

QUICK START — TRAIN A LoRA

----------------------------------------

1. Prepare your dataset folder. MP3, WAV or FLAC files. One song per file,

or split into chunks — more on that below.

2. Un-bypass the three trainer nodes: YuE2TrainingDataset, YuE2LoRATrainer,

YuE2TrainingCurve (select each, Ctrl+B toggles bypass)

3. YuE2TrainingDataset node:

- audio_folder = your dataset folder

- clip_seconds = 8 (see why below)

- caption_mode = none (trigger word only, simplest path)

4. YuE2LoRATrainer node:

- trigger_word = pick one, short and unusual (the shipped file uses mystyle)

- lora_name = your output file name — it saves to models/loras/

5. Queue. First run encodes your audio into VAE latents and caches them

(one-time cost), then training starts. Watch the loss curve on the

PreviewImage.

6. When it finishes, load the new LoRA in LoraLoaderModelOnly and generate.

Dataset prep: I trained on one full song split into 10-second chunks

(ffmpeg -f segment -segment_time 10). Twenty files, two and a half minutes

of audio. Works fine for style. The ShowText node in the shipped file shows

what the dataset node reported: 20 source files, 20 clips of 8 seconds each.

Why clip_seconds = 8 and not 10: clips must be strictly shorter than the

source files. My 10-second chunks decode as 9.98-10.00 seconds, and the

node raised "All source files are shorter than one clip" at 10. Eight

clears every file with headroom. If you train on full songs, you can raise

this to 20-30 for longer context.

----------------------------------------

WHAT TO INPUT (STYLE + LYRICS FORMAT)

----------------------------------------

Style prompt — comma-separated tags, lowercase, specific. The model reads

it like a producer brief:

k-pop, edm trap, hip-hop, girl group, four female vocals, aggressive

female rap, 808 bass, synth stabs, stomp-clap percussion, high energy, 140 bpm

Lyrics — section tags in square brackets:

[intro] / [verse] / [pre-chorus] / [chorus] / [post-chorus] / [bridge] / [outro]

For instrumental or background music, leave lyrics empty or feed minimal

vocal sections depending on what you want. The trainer is the way to get

consistent instrumental quality out of base Yue 2.

When you use a LoRA, put your trigger word in the style prompt. Example

with a trained style LoRA:

mystyle, k-pop, girl group, heavy 808 bass, high energy, 140 bpm

----------------------------------------

KEY SETTINGS (AND WHY)

----------------------------------------

From the shipped file, all values below are what it loads with.

YuE2TrainingDataset:

clip_seconds: 8 — must be shorter than your shortest audio file

caption_mode: none — trigger word only. Same-named .txt captions per file

are supported if you want them, but none keeps it simple and avoids

garbage captions from auto-taggers

force_reencode: False — set True only if you changed the audio files in the

folder and want the latent cache rebuilt

YuE2LoRATrainer:

steps: 3000 — shipped value. The repo README reports 5000 steps working

best for their tests, so if 3000 feels undercooked, push to 5000

rank: 32 / alpha: 32 — leave as shipped, standard capacity for a style LoRA

learning_rate: 0.0001 with cosine schedule, 50 warmup steps — leave as is

target_preset: nar_attn_mlp — this is correct and important: training hits

the NAR (acoustic rendering) branch only. The AR composer branch stays

frozen because no audio-to-token encoder is released. So this is a

style and timbre LoRA, not a voice clone

optimizer: adamw, ema_decay 0.999, t_sampling logit_normal — leave as shipped

LoraLoaderModelOnly:

strength_model: 1 — shipped. The repo author reports weight 2.0 works best

with 5000 steps, so test 1, 1.5 and 2 on your own LoRA

Generation path (unchanged from the free workflow):

KSampler 32 steps, dpm_2, sgm_uniform, cfg 1 — all leave as shipped

YuE2GenerateMusic: temperature 1, top_p 0.95, top_k 100, repetition_penalty 1.2 — leave

----------------------------------------

TIPS

----------------------------------------

1. Train on your best-sounding material. The LoRA learns style,

instrumentation and vocal character from whatever you feed it, so a

dataset of tracks you actually love beats a large mixed bag.

2. For background music production, build a small dataset of the exact

instrumental mood you use across videos — one 2-3 minute equivalent per

style is enough to start. Then every video's BGM renders in the same

house sound with the trigger word.

3. Keep the trigger word weird and unique. mystyle works as a placeholder,

but something like bbchoom collides with nothing in the base model's

vocabulary and binds faster.

4. Do not use ABC score input together with your LoRA. The repo README is

explicit: supplied ABC code may fight the LoRA. Let the trained model

plan its own score.

5. First training run caches the VAE latents of your dataset. Runs two and

onward start almost immediately. Do not edit files inside the dataset

folder between runs unless you flip force_reencode.

6. Loss curve down and flat is normal; loss spiking upward means drop the

learning rate or lower rank.

----------------------------------------

COMMON ISSUES

----------------------------------------

"All source files are shorter than one clip" — clip_seconds is longer than

your audio files. Lower it (8 for 10-second chunks), or re-split with

longer segments.

Trainer nodes missing / red after loading — ComfyUI-YuE2-Trainer not

installed or dependencies missing in your ComfyUI environment. Check the

console for import failed on startup.

LoRA loaded but nothing sounds different — trigger word missing from the

style prompt, or strength too low. Put the trigger first in the style

string, raise strength toward 2.

Voice does not match the training tracks — expected. Voice cloning does not

work with this trainer; only the acoustic/style branch trains. It captures

timbre character, not identity.

Out of memory during training — close other models, make sure generation

graph is not queued in the same batch, and keep batch clips short (8

seconds). The 3B model plus training overhead still fits comfortably on

24 GB; below that, expect it to be slow rather than impossible.

Audio glitches in output right after training — let it run the full steps.

A half-trained checkpoint (save_every defaults to off, file appears at the

end) will not exist mid-run anyway, but an interrupted run can leave a

partial file. Delete and rerun if that happens.

----------------------------------------

LICENSE REMINDER

----------------------------------------

Yue 2 weights are CC BY-NC 4.0, non-commercial. Personal videos and

learning are one thing, but before you sell music or run this in a

commercial production pipeline, check the license terms and the repo's

THIRD_PARTY_NOTICES.md.

----------------------------------------

The workflow file is attached, along with screenshots of the training group

and the loss curve from my run. 20 chunks of one song, 3000 steps, trigger

word, done — that is the shortest honest path to a working style LoRA.

Questions? Drop a comment with your GPU, VRAM, dataset size, and the exact error.

Thanks for supporting the channel. You guys make these deep dives possible.

— Benji