OpenXLA Summer DevLab 2026 — Benchmarking vLLM on XLA Backends
Abstract
I was honored to be invited by Google’s ML team and the TPU team to speak at Dev Lab Summer 2026. My talk, “Benchmarking vLLM on XLA Backends: From CUDA to OpenXLA for LLM Serving,” covered what it takes to move LLM serving from CUDA to OpenXLA: why raw vendor throughput curves mislead, how to measure usable goodput under strict SLOs, where multi-accelerator sharding breaks down without NVLink bridges, and a look under the hood of the newly unified 2026 vLLM + tpu-inference backend stack. The lineup was largely from Google, with colleagues from NVIDIA and the Meta PyTorch team, so it was exciting to represent PayPal in the room.
