LangSmith: Tracing, Debugging, and Monitoring LLM Apps

Descrizione del corso

LangSmith provides full-stack observability for LLM applications. This course covers LangSmith setup, instrumenting Python and TypeScript apps with automatic tracing, building datasets from production traces, running evaluations against datasets, prompt management and versioning, A/B testing prompts in LangSmith Hub, monitoring dashboards, alerting on hallucinations and latency, and integrating LangSmith into CI/CD pipelines.

Cosa imparerai

• Curated from public resources. Master LangSmith for full observability of LLM applications: tracing, prompt management, and production monitoring.

Programma del corso

1. LangSmith overview: tracing and evaluation platform (18 min)
2. Instrumenting LangChain apps with automatic tracing (22 min)
3. Manual tracing with @traceable and RunTree (20 min)
4. Creating datasets from production traces (28 min)
5. Running automated evals with LangSmith Evaluate (32 min)
6. LangSmith Hub: sharing and versioning prompts (18 min)
7. Setting up production dashboards and alerts (25 min)
8. Online evaluations: monitoring quality in real traffic (30 min)

Struttura del corso

3 sezioni, 8 lezioni, 7h 0m di durata complessiva. Il corso si segue interamente online, al proprio ritmo.

Domande frequenti