LLM Evals: RAGAS, DeepEval, and Automated Testing

Descrizione del corso

Learn how to systematically evaluate LLM-powered applications beyond vibes. This course covers the evaluation crisis in LLM apps, building ground-truth datasets, RAGAS metrics (answer relevancy, faithfulness, context recall, context precision), DeepEval framework setup, LLM-as-judge patterns, regression testing CI/CD pipelines, A/B evaluation of prompts, and monitoring evaluation drift in production.

Cosa imparerai

• Curated from public resources. Build automated evaluation pipelines for LLM applications using RAGAS, DeepEval, and custom metrics.

Programma del corso

1. The evaluation crisis: why vibes don't scale (20 min)
2. Building ground-truth datasets for your domain (30 min)
3. RAGAS metrics: faithfulness, answer relevancy, context precision (35 min)
4. Running RAGAS on your RAG pipeline (25 min)
5. Synthetic test set generation with RAGAS (28 min)
6. DeepEval setup: unit tests for LLM outputs (32 min)
7. LLM-as-judge: GPT-4 and Claude as evaluators (30 min)
8. Regression testing in CI/CD with DeepEval (25 min)

Struttura del corso

3 sezioni, 8 lezioni, 8h 0m di durata complessiva. Il corso si segue interamente online, al proprio ritmo.

Domande frequenti