LoRA / PEFT

LoRA: low-rank adapters

Freeze W, train small matrices A·B so ΔW ≈ BA — millions of trainable params instead of billions.

h = W @ x + (B @ A) @ x   # rank r ≪ d