SDXL Kohya LoRA Training With 12 GB VRAM Having GPUs - Tested On RTX 3060 (Patreon)

Patreon exclusive posts index

Join discord and tell me your discord username to get a special rank : SECourses Discord

Upgrade Kohya to latest version. Open a CMD and do a git pull.

Kohya repo : https://github.com/bmaltais/kohya_ss

SDXL training video : https://youtu.be/AY6DMBCIZ3A

SDXL training github file : click

I tested on my second GPU - RTX 3060 (12 GB). However since it is my second GPU it has 0 VRAM usage. Therefore I was able to use 96 Network Rank (Dimension). The more Network Rank (Dimension) means it will be able to learn more information.

But since you will use it on your main GPU there will be already some VRAM usage. Therefore start with Network Rank (Dimension) 8 and then increase until you get error as I have shown in video.

Sorry that it doesn't have audio because I am on vacation. Therefore I recorded via remote connection to my main computer.

Training command (training command.txt) with 32 Network Rank

32 Rank uses 11.5 GB VRAM

accelerate launch --num_cpu_threads_per_process=2 "./sdxl_train_network.py" --enable_bucket --min_bucket_reso=256 --max_bucket_reso=2048 --pretrained_model_name_or_path="F:/0 models/sd_xl_base_1.0.safetensors" --train_data_dir="F:/kohya sdxl tutorial files\img" --reg_data_dir="F:/kohya sdxl tutorial files\reg" --resolution="1024,1024" --output_dir="F:/kohya sdxl tutorial files\model" --logging_dir="F:/kohya sdxl tutorial files\log" --network_alpha="1" --save_model_as=safetensors --network_module=networks.lora --text_encoder_lr=0.0004 --unet_lr=0.0004 --network_dim=32 --output_name="tutorial_video" --lr_scheduler_num_cycles="10" --no_half_vae --full_bf16 --learning_rate="0.0004" --lr_scheduler="constant" --train_batch_size="1" --max_train_steps="5200" --save_every_n_epochs="1" --mixed_precision="bf16" --save_precision="bf16" --cache_latents --cache_latents_to_disk --optimizer_type="Adafactor" --optimizer_args scale_parameter=False relative_step=False warmup_init=False --max_data_loader_n_workers="0" --bucket_reso_steps=64 --gradient_checkpointing --xformers --bucket_no_upscale --noise_offset=0.0

Training json file : test_video_lowVRAM.json