Training
LoRA: Low-Rank Adaptation of Large Language Models
Microsoft·June 17, 2021
Edward J. Hu, Yelong Shen, Phillip Wallis
View on arXivTL;DR
Introduces LoRA: freeze the base model and train small low-rank adapter matrices, cutting fine-tuning cost by orders of magnitude with no quality loss.
Why it matters
Made fine-tuning large models affordable and democratized customization. LoRA and its descendants underpin most of the open-source fine-tuning ecosystem.