Google Cloud flipped the switch on general availability for Ironwood TPU v7 this week, and the specs are not incremental: pods that scale to 9,216 liquid-cooled chips, 192GB of HBM3e per chip, and a fabric designed for one job above all others — serving frontier models at inference scale without a single Nvidia part in the critical path. Anthropic has publicly committed to accessing up to one million of these chips, the largest non-Nvidia frontier compute commitment anyone has made. That number matters less as a procurement fact than as a proof point: the second source is real, it is bookable, and it has a customer willing to bet a model roadmap on it. With Nvidia reportedly scaling back its $250B OpenAI datacenter guarantee this week, every buyer with a 2027 capacity plan is re-pricing accelerator supply right now.
What’s actually new about Ironwood TPU v7
Ironwood is Google’s seventh-generation Tensor Processing Unit and the first the company positions explicitly as an inference-first part rather than a training accelerator that happens to serve. The headline configuration is a 9,216-chip pod — roughly 24x the chip count of the largest v5p pod slice — connected by Google’s Inter-Chip Interconnect (ICI) in a 3D torus, with optical circuit switching handling reconfiguration and fault isolation. Each chip carries 192GB of HBM3e. Internalize that per-chip memory number: a 400B-parameter model in bf16 fits across a handful of chips rather than a rack, and KV cache headroom for long-context serving stops being the binding constraint on batch size.
Liquid cooling by default
Ironwood racks ship with direct-to-chip liquid loops, which is what makes the density story work — you cannot air-cool 9,216 chips in a coherent fabric at any sane footprint. Google has run liquid-cooled TPUs internally for several generations, so this is productized experience rather than a first attempt. The practical consequence: Ironwood is region-limited at launch in a way v5e never was. The capacity exists where the datacenter retrofit happened, and nowhere else. Check region availability before you architect around it.
The software gap has narrowed
JAX on TPU was always excellent and always a moat in the wrong direction — it kept PyTorch shops out. PyTorch/XLA has improved substantially, vLLM has a working TPU backend, and Google now ships an inference stack through GKE and Vertex AI that serves an open-weights model on Ironwood without a rewrite. That does not make Ironwood a drop-in Blackwell replacement. It does move the migration cost from “rewrite your serving stack” to “re-tune your serving config,” a category difference in enterprise procurement conversations.
Why it matters
- Nvidia’s pricing power now has a credible ceiling. A second source that can serve frontier-scale inference changes negotiation dynamics even for buyers who never actually switch. The Anthropic TPU deal is the existence proof that makes the threat credible in a procurement meeting.
- AI inference cost per token becomes the real battleground. Training runs are lumpy capex; inference is the recurring line item that scales with revenue. Ironwood’s perf-per-watt and 192GB HBM3e per chip both attack the same metric — tokens served per dollar of TCO, not FLOPs on a spec sheet.
- Memory capacity unlocks serving patterns, not just bigger models. 192GB per chip means larger batches, longer contexts, and more concurrent sessions per chip. If your workload is KV-cache-bound — long-context RAG, agentic loops, multi-turn chat — that is where the economics move.
- Supply timing is the actual constraint for 2027. Nvidia pulling back on the OpenAI datacenter guarantee signals that even the incumbent is calibrating commitments to real supply. Buyers who assumed guaranteed GB300-class capacity in 2027 should price a mixed fleet now.
- Vendor lock-in shifts, it does not disappear. Ironwood is Google Cloud only. You trade CUDA lock-in for GCP lock-in. The escape hatch is writing against vLLM or JAX rather than vendor-specific kernels — build that discipline in before you commit capacity.
- Liquid cooling is now table stakes for frontier inference. If you run any on-prem AI infrastructure, this is your signal that air-cooled accelerator racks have a finite roadmap.
How to use Ironwood TPU v7 today
Ironwood is consumable three ways: raw TPU VMs, GKE node pools, and Vertex AI endpoints. Most teams should start at the GKE or Vertex layer and drop to raw VMs only if they are writing custom kernels.
-
Check quota and regional availability first. Ironwood capacity is region-constrained at GA, and quota is not granted by default. Confirm before you plan anything:
gcloud auth login gcloud config set project YOUR_PROJECT_ID # List accelerator types available in a region gcloud compute accelerator-types list \ --filter="zone:( us-east5-a us-east5-b )" \ --format="table(name,zone,description)" # Check your current TPU quota gcloud compute regions describe us-east5 \ --format="table(quotas.metric,quotas.limit,quotas.usage)" -
Request a queued resource rather than a live VM. For scarce capacity, queued resources put you in line instead of failing on a stockout. This is the correct default for anything larger than a single slice:
gcloud compute tpus queued-resources create ironwood-qr-1 \ --node-id=ironwood-node-1 \ --project=YOUR_PROJECT_ID \ --zone=us-east5-a \ --accelerator-type=v7-256 \ --runtime-version=tpu-ubuntu2204-base \ --valid-until-duration=7d # Poll status gcloud compute tpus queued-resources describe ironwood-qr-1 \ --zone=us-east5-a \ --format="value(state.state)" -
Verify the topology from inside the VM before you spend anything. A surprising number of failed TPU projects trace back to a misread topology. Confirm chip count and memory:
gcloud compute tpus tpu-vm ssh ironwood-node-1 --zone=us-east5-a # On the TPU VM python3 -c " import jax print('devices:', jax.device_count()) print('local: ', jax.local_device_count()) d = jax.devices()[0] print('kind: ', d.device_kind) print('mem GB: ', d.memory_stats()['bytes_limit'] / 1e9) " -
Serve an open-weights model with vLLM on TPU. This is the fastest path to a real cost-per-token number. Do not benchmark with a toy prompt — use your actual traffic distribution:
pip install vllm-tpu VLLM_TARGET_DEVICE=tpu python3 -m vllm.entrypoints.openai.api_server \ --model meta-llama/Llama-3.3-70B-Instruct \ --tensor-parallel-size 8 \ --max-model-len 32768 \ --max-num-seqs 256 \ --download-dir /mnt/disks/models -
Benchmark against your real traffic, then compute cost per million tokens. The only number that matters is dollars per million output tokens at your p95 latency target:
python3 -m vllm.entrypoints.benchmarks.benchmark_serving \ --backend openai \ --base-url http://localhost:8000 \ --model meta-llama/Llama-3.3-70B-Instruct \ --dataset-name sharegpt \ --num-prompts 1000 \ --request-rate 20 \ --metric-percentiles "50,95,99"Then:
cost_per_1M_output_tokens = (hourly_slice_price / output_tokens_per_hour) * 1_000_000. Run the identical harness on your current Nvidia fleet. Skip this step and you are buying a spec sheet. -
For GKE, pin the node pool to the right topology. Autoscaling across TPU slices requires the topology labels to match your workload’s expectations:
apiVersion: apps/v1 kind: Deployment spec: template: spec: nodeSelector: cloud.google.com/gke-tpu-accelerator: tpu-v7 cloud.google.com/gke-tpu-topology: 4x4x4 containers: - name: inference image: YOUR_IMAGE resources: limits: google.com/tpu: 4
How it compares: Google TPU v7 vs Nvidia Blackwell
The honest comparison is not chip-to-chip — it is fleet-to-fleet at a fixed serving SLO. Here is how the options stack up for a team choosing accelerators for 2027 inference capacity.
| Dimension | Ironwood TPU v7 | Nvidia Blackwell (B200 / GB200) | AWS Trainium2 |
|---|---|---|---|
| Memory per chip | 192GB HBM3e | 192GB HBM3e (B200) | 96GB HBM |
| Max coherent domain | 9,216-chip pod (ICI, 3D torus) | 72-GPU NVL72 rack, scaled via InfiniBand | UltraCluster via NeuronLink |
| Cooling | Liquid, default | Liquid (GB200 NVL72), air option on B200 | Mixed |
| Primary software path | JAX, PyTorch/XLA, vLLM-TPU | CUDA, TensorRT-LLM, vLLM, everything | Neuron SDK, PyTorch |
| Ecosystem maturity | Good and improving; JAX-native | Dominant; every kernel targets it first | Narrower |
| Availability | Google Cloud only, region-limited | Every cloud plus on-prem | AWS only |
| Best fit | High-volume inference, long context, JAX shops | Anything requiring the newest kernels or on-prem | Cost-sensitive AWS-native workloads |
| Lock-in vector | GCP | CUDA | AWS |
Read that table as a portfolio decision, not a winner. If you are training a frontier model with custom kernels, Blackwell’s ecosystem still wins on time-to-first-result. If you are serving a stable model at high volume and your AI inference cost per token is a board-level line item, Ironwood is the first alternative worth a real evaluation rather than a courtesy one. On TPU pricing Google Cloud specifics: list prices move and committed-use discounts move more, so treat published on-demand rates as a ceiling and get a quote against a one- or three-year commit before you model anything.
What’s next
Watch three things over the next two quarters. First, whether Ironwood capacity lands outside the initial regions on schedule. Liquid-cooling retrofits are physical construction projects, and construction slips. If Google adds European and Asian regions on time, the “second source” story holds; if it does not, Ironwood remains a US-centric option that constrains anyone with data residency requirements.
Second, watch the software gap close or stall. The specific signal is whether new inference techniques — speculative decoding variants, novel attention kernels, quantization schemes — ship with TPU support at launch or six months later. Today they ship CUDA-first. If that changes, the moat erodes fast. If it does not, TPU stays a great option for stable workloads and a poor one for teams chasing the frontier of serving efficiency.
Third, watch what Nvidia does about the OpenAI guarantee pullback. A vendor recalibrating a $250B commitment is telling you something about its own supply confidence. The practical move for any buyer with 2027 capacity plans: stop assuming a single-vendor fleet, run the benchmark in step 5 on both stacks, and negotiate with real numbers in hand. Optionality has a price, and right now it is cheaper than it will be in twelve months. Among Google Cloud accelerator options, Ironwood is the one that changes the negotiation.
Frequently Asked Questions
Is Ironwood TPU v7 actually faster than Nvidia Blackwell?
For inference on well-supported model architectures, the two are close enough that the answer depends entirely on your workload and serving configuration. Ironwood’s advantage is perf-per-watt and coherent-domain scale; Blackwell’s is ecosystem maturity and kernel availability. Benchmark both on your actual traffic — vendor benchmarks are selected to flatter the vendor, and the honest answer here is genuinely workload-dependent.
Can I run PyTorch on Ironwood, or do I have to learn JAX?
PyTorch works through PyTorch/XLA, and vLLM has a TPU backend that covers most standard serving cases without model code changes. JAX remains the native and best-optimized path — if you are writing custom kernels or doing anything unusual, JAX is where the performance is. For serving a standard transformer, PyTorch/XLA plus vLLM is sufficient.
What does TPU pricing on Google Cloud look like versus GPU instances?
Published on-demand rates are the wrong number to compare. TPUs sell by slice, GPUs by instance, and the meaningful comparison is dollars per million tokens at your latency SLO — which depends on batch size, context length, and model. Compute that number yourself using the benchmark in step 5. Then negotiate: committed-use discounts on TPU capacity are substantial, and Google is motivated to win reference customers right now.
What does the Anthropic TPU deal actually mean for other buyers?
It means the capacity is real and the software stack is production-viable at frontier scale — you do not commit a model roadmap to hardware that does not work. It also means competition for that capacity. A commitment of up to one million chips is a large share of near-term supply, so if Ironwood is in your 2027 plan, get in the queue rather than assuming on-demand availability.
Does 192GB HBM3e per chip change how I should architect serving?
Yes, if your workload is memory-bound rather than compute-bound. Larger per-chip memory means bigger batches and longer KV caches without sharding overhead — exactly the profile of long-context RAG and agentic workloads. Re-tune your batch size and max sequence length rather than porting your existing configuration verbatim; settings tuned for 80GB chips leave most of the advantage unused.
Should I move my whole fleet to TPUs?
No. Move the workload that is stable, high-volume, and cost-sensitive — typically your highest-traffic production inference endpoint on a model you are not changing weekly. Keep GPU capacity for experimentation, custom kernels, and anything that needs a technique that shipped last month. A mixed fleet costs more in engineering discipline and buys real negotiating leverage plus supply resilience, which in 2027 will be worth more than the operational simplicity you gave up.
Go deeper than this article
This article covers the essentials. Our Technical & Coding eguide collection gives you the full step-by-step playbooks — prompts, workflows, and copy-paste recipes built for exactly this work.
