Cloud Run GPUs: serve a model without running a cluster

The operating problem

A serving path has to answer four questions before any platform choice helps. How many requests arrive, and how bursty are they? What latency does the product promise? What does the model cost to hold in memory? And who operates the service at 2 a.m. when it fails? GPU serverless answering shifts the last question. Nobody patches the node, configures drivers, or schedules GPU nodes in a pool. The trade is a fixed hardware menu, instance-based billing, and fewer tuning controls than a dedicated deployment.

System boundary

A Cloud Run GPU service is a container image with a declared NVIDIA L4 accelerator. Google sets the rest of the boundary: That last point matters for Indian workloads. A serving path through Singapore or Virginia adds round-trip latency that a Mumbai or Delhi user will notice on every request.

One L4 GPU per instance. No other accelerator is offered on Cloud Run today.
A minimum configuration of 4 vCPUs and 16 GiB of memory alongside the GPU.
Instance-based billing. There are no per-request fees for the GPU itself, and the GPU is billed for the entire instance lifecycle, not only while a request executes.
Region availability is narrower than CPU-only Cloud Run. As of late August 2026, generally available L4 regions include us-central1, us-east4, europe-west1, europe-west4, and asia-southeast1. asia-south1, the Mumbai region closest to most BluePi customers, is invitation-only.

The cost model, with numbers

Published L4 GPU pricing is 0.011202 dollars per minute without zonal redundancy and 0.017454 dollars per minute with it. Those are GPU-only prices; CPU, memory, and network are billed separately, and usage rounds up to the nearest 100 milliseconds. The minimum instance therefore costs roughly: The minimum works out to about 0.0174 dollars per instance-minute, or near 1.05 dollars per instance-hour, before network charges and at Tier 1 published rates. The calculation changes two design choices. First, scale-to-zero is the feature that makes the price tolerable for sparse traffic. An internal tool that handles 40 requests a day pays nothing between requests, provided cold starts are acceptable. Second, minimum instances cost the full rate while idle. A team that sets one minimum instance to avoid cold starts commits to about 750 dollars per month before a single request arrives. The GPU is billed for the instance lifecycle, so warm capacity and GPU spend are the same line item.

L4 GPU: 0.0112 dollars per minute.
4 vCPUs: about 0.0043 dollars per minute.
16 GiB of memory: about 0.0019 dollars per minute.

When the L4 is the wrong answer

An L4 has 24 GiB of video memory. That fits tuned 7B to 13B parameter models comfortably, and larger models only with aggressive quantization. It does not fit a 70B parameter model in ordinary precision. If the model does not fit, the serving path is a GKE node pool or Vertex AI endpoints, not Cloud Run. Batch throughput workloads are also a poor fit. Cloud Run bills the instance, so a queue that keeps the GPU busy around the clock pays serverless rates for steady utilization. A reserved GKE commitment is cheaper at that duty cycle. The decision boundary is therefore traffic shape. Sparse or business-hours traffic, one model that fits in 24 GiB, and no operations team favor Cloud Run. Steady high utilization, larger models, or multi-GPU topologies favor a cluster or a managed inference service.

Failure modes worth planning for

Cold starts include model loading. A multi-gigabyte model pulled into video memory can add tens of seconds to the first request. Keep-alive pings spend GPU minutes, so budget for one or the other.
Instance-based billing surprises teams that expect request-based serverless pricing. Review the billing docs before the first deploy, not after the first invoice.
Region gaps bite late. Confirm L4 availability in the region closest to your users before writing the deployment config. Mumbai remains invitation-only as of this writing.
Concurrency per GPU instance is limited by memory, not by the CPU quota. Load test with realistic payloads to find the useful concurrency, and set max instances so a burst cannot create an expensive fleet.

Monitoring and operations

Watch instance count, GPU memory utilization, and request latency percentiles together. Instance count multiplied by the per-minute rate is the spend signal. GPU memory near the ceiling with low throughput usually means the concurrency setting is too conservative or the batch size is too small. Alert on cold-start latency if the product promises a response-time target. A serving path that meets its target only while warm is a product risk hiding inside an infrastructure metric.

What we would tell a team choosing today

Start from the traffic, not the platform. Count requests per day, the latency target, and the model size, then let those three numbers pick the serving path. For a model that fits an L4 and traffic that arrives in office hours, a Cloud Run GPU service removes most of the operational surface and prices fairly. For anything steady, large, or latency-hard, compare against GKE with committed use discounts and Vertex AI endpoints before committing.