QLoRA: Efficient Finetuning of Quantized Large Language Models

Tim Dettmers, Artidoro Pagnoni, Ari Holtzman|May 23, 2023arXiv

Key Takeaway

4-bit quantization plus LoRA makes fine-tuning 65B models possible on a single GPU — QLoRA democratized large model customization for individual researchers.

Summary

Enables fine-tuning a 65B parameter model on a single 48GB GPU by combining 4-bit quantization with LoRA adapters. Introduces NormalFloat4 quantization and double quantization to reduce memory without quality loss.

training efficiency