The Deployment Spectrum of Generative AI on Google Cloud

Every team has picked a GenAI model. Far fewer have deliberately picked where it runs, and that second decision is the one that shows up in your bill, your latency graph, and your on-call rotation.Google Cloud doesn't offer one way to run Generative AI. It offers a spectrum: the Gemini API on Vertex AI, dedicated Vertex endpoints, Cloud Run with GPUs, GKE inference, and raw GPUs and TPUs on Compute Engine. Each rung trades control for operational burden, and the trade is rarely the one teams expect.This session walks all five rungs and gives you a decision path you can defend in a design review. We'll work through a cost curve that surprises most people (self-hosting isn't cheaper, it's cheaper only if you keep the accelerator busy), name the three ways this commonly goes wrong, and cover what you inherit on the way down: cold starts, the observability signals that actually matter, and quota as your new single point of failure.You'll leave able to:Map a workload to the right rung and justify itJudge when self-hosting genuinely pays offDesign a fallback for the day capacity runs outFor platform engineers, architects, and engineering leaders running GenAI in production.