Customizing LLMs on Amazon EKS

If you're already serving GenAI models and want to make them yours, this is the workshop for you.

This is a focused path within the Generative AI on EKS Workshop series, designed specifically for practitioners who want to move past out-of-the-box inference and into model customization — fine-tuning and retrieval-augmented generation — all running on the same Amazon EKS and NVIDIA GPU infrastructure you'd use in production.

In this workshop, you will:

• Deploy and serve Ministral-3-8B on EKS using vLLM with Run:ai Streamer for direct S3 model loading
• Monitor GPU utilization and inference metrics with NVIDIA DCGM, Prometheus, and Grafana
• Build intelligent agents using the Strands Agents SDK with your self-hosted model
• Fine-tune the model into a domain-specific advisor using LoRA on a single GPU, then deploy the adapter alongside the base model and compare outputs side-by-side
• Build a RAG pipeline with Amazon S3 Vectors, a document processor, and a Gradio interface to compare RAG-enhanced and standard responses

Who is this for? Data scientists, ML engineers, and application developers who want to customize GenAI models for specific business needs. Basic familiarity with EKS and model serving is helpful but not required.

What this workshop does not cover: Cluster setup from scratch, distributed inference with Ray, or autoscaling strategies. For the full infrastructure foundation, see the companion Hosting and Scaling GenAI Workloads on Amazon EKS workshop.