Local LLMs: Ollama, vLLM, and Self-Hosted AI
«Local LLMs: Ollama, vLLM, and Self-Hosted AI» è un corso online di RAG e AI engineering di livello intermedio, tenuto da InsightLearn Curated AI su InsightLearn. Il percorso comprende 8 lezioni organizzate in 3 sezioni, per una durata complessiva di 11h 0m. Run large language models locally and on private servers with Ollama, vLLM, LM Studio, and llama.cpp.
- Docente: InsightLearn Curated AI
- Categoria: RAG e AI engineering
- Livello: intermedio
- Durata: 11h 0m
- Accesso: gratuito in questa fase di validazione, in cambio di un feedback al termine
- Certificato: certificato di completamento verificabile online
Descrizione del corso
InsightLearn è in fase di sviluppo (work in progress): i contenuti di questa pagina sono pubblicati esclusivamente a fini di verifica e test.
The complete guide to self-hosted LLMs. Covers model formats (GGUF, GGML, safetensors), quantization techniques (GPTQ, GGUF Q4/Q8, AWQ), Ollama setup and modelfile customization, vLLM for high-throughput serving, llama.cpp compilation and optimization, LM Studio for desktop use, OpenAI-compatible API servers, model benchmarking (MMLU, HumanEval), privacy-first RAG with local models, and hardware sizing for 7B/13B/70B models.
Cosa imparerai
- Curated from public resources. Run large language models locally and on private servers with Ollama, vLLM, LM Studio, and llama.cpp.
Programma del corso
- GGUF, GGML, safetensors: model format guide (28 min)
- Quantization: GPTQ, AWQ, GGUF Q4/Q5/Q8 tradeoffs (35 min)
- Ollama setup: models, Modelfile, and API (30 min)
- Ollama custom models with Modelfile FROM/SYSTEM (25 min)
- Open WebUI: ChatGPT-like interface for local models (22 min)
- vLLM: PagedAttention and continuous batching (38 min)
- OpenAI-compatible server with vLLM (20 min)
- Hardware sizing: GPU VRAM requirements per model size (25 min)
Dalle lezioni
Estratto della trascrizione di «GGUF, GGML, safetensors: model format guide»
Welcome to this lesson of the local LLMS course. Orama, VLLM, and Self-hosted AI. In this session we will explore GOOF, GGML, Safe and Source, Model Format Guide. We will start with the fundamental concepts and the basic theoretical principles. We will then see practical applications that are almost universal in the world of AI development. We will analyze them in detail. and the best practices in the sector. At the end of this lesson, you will be able to apply autonomously the techniques of GOOF, GGML, Saftet & Source in your projects. We will start with the fundamental concepts and the basic theoretical principles. We will then see practical and almost universal applications in the world of development. We…
Estratto della trascrizione di «Quantization: GPTQ, AWQ, GGUF Q4/Q5/Q8 tradeoffs»
Welcome to this lesson of local local LLMS. Hola, my name is LLMS, and I am a self-hosted AI. In this session, we will explore quantization, GPTQ, AUG, GOOF Q4 BARB, Q5 BARB, Q8 TOTRA DELOFS. We will start with the fundamental concepts and the theoretical principles. We will then see practical applications and practical cases of use in the world of development. We will analyze specific examples and the best practices of the sector. At the end of this lesson you will be able to apply autonomously the techniques of quantization in your projects. We will start with the fundamental concepts and the basic theoretical principles. We will then see practical applications and almost real uses in the world…
Estratto della trascrizione di «Ollama setup: models, Modelfile, and API»
Welcome to this lesson of the local LLMS course. Lama, VLLM, and Self-hosted AI. In this session, we will explore Lama Setup. Models, Modelfile, and API. We will start with the fundamental concepts and the basic theoretical principles. We will then see practical applications and almost practical cases in the development world. We will analyze them in concrete elements. of the sector. At the end of this lesson, you will be able to apply autonomously the techniques of the mass-set-up of your projects. We will start with the fundamental concepts and the basic theoretical principles. We will then see practical applications and almost real ones in the world of development. We will analyze concrete examples and the best practices of…
Altri corsi di RAG e AI engineering
- Multimodal AI: Vision, Audio, and Image Generation APIs
- Advanced Prompt Engineering: Chain-of-Thought, Few-Shot, and Beyond
- RAG from Scratch: Complete Retrieval-Augmented Generation
- Vector Databases in Production: Chroma, FAISS, pgvector
- Agentic AI Patterns: ReAct, Plan-and-Execute, and Multi-Agent
- AI Engineering: Building Production LLM Applications
Domande frequenti
Quanto dura il corso Local LLMs: Ollama, vLLM, and Self-Hosted AI?
Il corso dura 11h 0m in totale, con 8 lezioni suddivise in 3 sezioni, da seguire online al proprio ritmo.
Serve esperienza pregressa?
Il corso è di livello intermedio e non indica requisiti specifici di ingresso.
Il corso rilascia un certificato?
Sì: al completamento del corso ricevi un certificato di completamento verificabile online.