Hosting and Scaling GenAI Workloads on Amazon EKS

This hands-on workshop teaches you how to deploy, serve, optimize, and scale large language model inference workloads on Amazon EKS with NVIDIA GPUs.

In this workshop, you will:

• Set up an EKS cluster optimized for NVIDIA GPU workloads
• Deploy and serve models efficiently using vLLM
• Implement distributed inference with Ray
• Monitor workloads using Prometheus and Grafana
• Apply production best practices for GenAI deployments on Kubernetes

Who is this for? Platform engineers, MLOps practitioners, and DevOps teams focused on deploying and scaling production GenAI inference on Kubernetes.

What this workshop does not cover: Fine-tuning or RAG. To customize models for domain-specific use cases, see the companion Customizing LLMs on Amazon EKS workshop.