Fine-Tuning LLMs: LoRA, QLoRA, and Full Fine-Tuning

Descrizione del corso

A practical course on parameter-efficient fine-tuning. Covers when to fine-tune vs prompt engineer, dataset curation for instruction tuning, LoRA fundamentals (rank, alpha, target modules), QLoRA with 4-bit quantization, supervised fine-tuning with HuggingFace TRL, DPO and RLHF alignment, LoRA adapter merging, evaluation on domain benchmarks, and deploying fine-tuned models with Ollama and vLLM.

Cosa imparerai

• Curated from public resources. Fine-tune open-source LLMs with LoRA and QLoRA: dataset preparation, training, evaluation, and deployment.

Programma del corso

1. When to fine-tune vs prompt engineer (25 min)
2. Dataset curation for instruction tuning (35 min)
3. LoRA mathematics: low-rank decomposition explained (30 min)
4. QLoRA: 4-bit quantization + LoRA on consumer GPUs (45 min)
5. SFTTrainer: supervised fine-tuning with TRL (40 min)
6. DPO: Direct Preference Optimization (38 min)
7. Merging LoRA adapters and pushing to HuggingFace Hub (25 min)
8. Evaluating fine-tuned models with EleutherAI harness (30 min)
9. Deploying fine-tuned models with Ollama and vLLM (28 min)

Struttura del corso

3 sezioni, 9 lezioni, 14h 0m di durata complessiva. Il corso si segue interamente online, al proprio ritmo.

Domande frequenti