Supercharging GenAI: Ray, Kubernetes, and TPUs for Lightning-Fast Inference
Talk at KubeCon 2024 on supercharging GenAI inference. **Key points:** - Distributed inference with Ray for scaling LLM serving - Kubernetes orchestration for ML workloads - TPU …
