LangSmith: Observability + Evals per App LLM Production
«LangSmith: Observability + Evals per App LLM Production» è un corso online di Agenti AI e framework di livello principiante, tenuto da InsightLearn Curated AI su InsightLearn. Il percorso comprende 7 lezioni organizzate in 3 sezioni, per una durata complessiva di 5h 0m. LangSmith traccia ogni LLM call, costo, latenza, errori. Evals automatiche, datasets, comparison runs. Il monitoring che serve in production.
- Docente: InsightLearn Curated AI
- Categoria: Agenti AI e framework
- Livello: principiante
- Durata: 5h 0m
- Accesso: gratuito in questa fase di validazione, in cambio di un feedback al termine
- Certificato: certificato di completamento verificabile online
Descrizione del corso
InsightLearn è in fase di sviluppo (work in progress): i contenuti di questa pagina sono pubblicati esclusivamente a fini di verifica e test.
LangSmith traccia ogni LLM call, costo, latenza, errori. Evals automatiche, datasets, comparison runs. Il monitoring che serve in production.
Cosa imparerai
- Curated from public resources. LangSmith traccia ogni LLM call, costo, latenza, errori. Evals automatiche, datasets, comparison runs. Il monitoring che serve in production.
Programma del corso
- Trace tree, latency, token usage (10 min)
- Projects + filtering avanzato (10 min)
- Creare dataset da trace (10 min)
- LLM-as-judge evaluator (10 min)
- Custom evaluator Python (10 min)
- Comparison run prompt A vs B (10 min)
- Prompt Hub versioning (10 min)
Dalle lezioni
Estratto della trascrizione di «Trace tree, latency, token usage»
So many devs are working with LLMs these days and they don't really know about the fundamentals. I was teaching an AI workshop recently in Poland and I asked people to put up their hands if they knew what a token was and I don't know whether it's my AI bubble on X or something but I was pretty shocked because only about a third of the people in the workshop put up their hand and so I wanted to put together a token deep dive for those of you who are using LLMs but kind of need to catch up on what's going on under the hood and of course we're going to explain all of these concepts using…
Estratto della trascrizione di «Projects + filtering avanzato»
Welcome back to Real Terms for AI where we breakdown modern AI concepts for professional developers. We did one episode on RAG and it's a powerful architecture for improving the quality of responses from an LLM. But sometimes basic RAG just isn't enough to reach the quality you want. Luckily when that happens there are many advanced techniques based on RAG that you can try. One of the most... ways it can be useful here. So when you add re-ranking to your rag application, you're adding a step between retrieving the chunks and sending those chunks to the LLM with the prompt. In the re-ranking step, you use an algorithm of some kind to score the chunks by which ones…
Estratto della trascrizione di «Creare dataset da trace»
Hi, I'm Matt Gaska, manager here with Siemens, and I'm going to talk about how to use the trace tool and operate today. So the first thing we're going to do is we're going to just quickly drive my axes to some known position, just my x-axis, because I have cop enabled on this axis, so I'm going to show you some positioning traces. So we're exactly at 8th to start. So we come into the diagnostics area, and I'm going to make a brand new trace. The name doesn't need to be anything unless you're going to be saving it. We're going to start off saving NC variables. I'll explain what the rest are later. And now I'm going…
Altri corsi di Agenti AI e framework
Domande frequenti
Quanto dura il corso LangSmith: Observability + Evals per App LLM Production?
Il corso dura 5h 0m in totale, con 7 lezioni suddivise in 3 sezioni, da seguire online al proprio ritmo.
Serve esperienza pregressa?
Il corso è di livello principiante e non indica requisiti specifici di ingresso.
Il corso rilascia un certificato?
Sì: al completamento del corso ricevi un certificato di completamento verificabile online.