Nvidia spent the last two years telling the industry that if you wanted to sit inside its rack, you brought your own chip and rented its bandwidth. The expansion of NVLink Fusion to support NVHBM — custom high-bandwidth memory stacks tuned per partner — gives that arrangement a memory subsystem to match. Broadcom, Marvell and Fujitsu can now attach accelerators to Nvidia’s rack-scale fabric with memory capacity and bandwidth ratios they specify, rather than whatever HBM configuration Nvidia shipped in its own GPUs. It lands the same week as a Q2 FY2027 earnings blowout and an AWS commitment for roughly two million more GPUs, which is exactly the point: Nvidia is widening the moat while the numbers are loud enough to drown out the strategic read.
What’s actually new with NVLink Fusion NVHBM
NVLink Fusion, announced in 2025, opened Nvidia’s chip-to-chip interconnect to third-party silicon. A hyperscaler’s custom TPU-class accelerator or a partner-designed CPU could speak NVLink and join a rack-scale coherent domain — 72 devices, sometimes more, behaving like one large machine. Fusion did not let those partners define their own memory hierarchy. You got the fabric; the memory system remained a Nvidia-shaped hole.
NVHBM moves that boundary. It is Nvidia-designed custom high-bandwidth memory packaging and controller IP that partners integrate into their own accelerator silicon, with capacity and bandwidth negotiated per design rather than inherited from a GPU SKU. A Broadcom inference ASIC can carry two to three times the memory-per-compute ratio of a Blackwell-class GPU if its workload is KV-cache-bound, or invert that ratio if it is compute-bound. Marvell can build for a customer whose model fits in 400GB but needs punishing bandwidth. Fujitsu can pair its MONAKA Arm CPU line with memory sized for HPC residency rather than transformer decode.
Three details matter more than the headline. First, this is Nvidia memory IP, not a generic open standard — the partner buys into Nvidia’s packaging, controller and validation flow, so Nvidia captures margin on the memory tier it previously reached only through whole-GPU sales. Second, NVHBM sits alongside HBM4 rather than replacing it: HBM4 is the JEDEC substrate, NVHBM a semi-custom stack-and-controller layer built on top with Nvidia-specific signaling into the NVLink domain. Third, the qualification path runs through Nvidia. Nobody attaches to this fabric without Nvidia’s sign-off, and that gate is the entire strategy.
Why it matters
- Memory ratio becomes a design variable, not a constraint. Inference economics turn on KV-cache residency and batch size. Teams building custom AI accelerator memory configurations can tune GB-per-TFLOP to their actual serving profile instead of over-buying compute to get memory.
- It defends against the custom-silicon exodus. Google, Amazon and Meta all build their own accelerators. NVLink Fusion plus NVHBM makes the pitch: build your chip, but build it inside our rack. Nvidia keeps the fabric, the software, the networking and now a memory royalty even when it loses the compute die.
- The HBM supply chain gains a new gatekeeper. SK Hynix, Micron and Samsung already allocate HBM4 capacity against Nvidia’s forecast. A semi-custom tier on top gives Nvidia influence over stack configurations that competitors must queue behind.
- Rack-scale inference bandwidth stops being uniform. Schedulers and inference servers assume homogeneous nodes. A rack mixing Blackwell-class GPUs with partner ASICs on different memory profiles demands topology-aware placement — vLLM, SGLang and TensorRT-LLM all need work here.
- It compresses the semi-custom market. Broadcom and Marvell made real money designing accelerators for hyperscalers who wanted out of Nvidia’s pricing. Those same firms are now channel partners inside Nvidia’s fabric. The escape hatch and the moat became the same door.
- Total cost of ownership shifts toward memory. Buy exactly the memory you need and the marginal cost of serving long-context models drops. Models that were uneconomical at 128K context start penciling out at 512K and beyond.
How to use it today
NVHBM silicon is a 2027 story for most buyers. Instrument your workloads now so that when partner parts arrive you know which memory ratio you need — and make sure your serving stack tolerates heterogeneous racks.
-
Measure your real memory-to-compute ratio. Most teams guess. Pull it from the device and compare against sustained FLOP utilization.
nvidia-smi --query-gpu=name,memory.total,memory.used,utilization.gpu,utilization.memory \ --format=csv -l 5 # Sustained view over a serving window nvidia-smi dmon -s pucm -d 5 -c 720 > ratio-profile.log -
Profile KV-cache pressure, because that is what NVHBM is for. In vLLM, cache utilization above roughly 90% with queued requests means you are memory-bound, not compute-bound — the exact case a custom memory ratio fixes.
curl -s http://localhost:8000/metrics | grep -E \ 'gpu_cache_usage_perc|num_requests_waiting|num_preemptions_total|prefix_cache_hit_rate' -
Model the counterfactual. Compute what a 2x memory, same-compute part buys you in batch size. Rough but decisive:
import math hbm_gb, weights_gb = 192, 140 # per-device HBM, model weights resident kv_per_token_mb = 0.125 # measure from your own model config ctx_tokens = 131_072 for mult in (1, 2, 3): free_gb = hbm_gb * mult - weights_gb seqs = math.floor(free_gb * 1024 / (kv_per_token_mb * ctx_tokens)) print(f"{mult}x memory -> {free_gb:.0f} GB free -> ~{seqs} concurrent 128K seqs") -
Verify your NVLink domain topology now. Heterogeneous racks make this a first-class concern rather than a curiosity.
nvidia-smi topo -m nvidia-smi nvlink --status nvidia-smi nvlink -gt d # per-link data throughput counters -
Make placement memory-aware in your serving config. Do not assume every device in a future rack has identical capacity.
# serving-profile.yaml placement: strategy: memory_aware pools: - name: high-capacity # NVHBM-class partner devices min_device_memory_gb: 288 workloads: [long_context, batch_offline] - name: high-compute min_device_memory_gb: 141 workloads: [low_latency_chat, embeddings] fallback: high-compute -
Get on the qualification list. Access to NVLink Fusion silicon runs through Nvidia’s partner program and the relevant ASIC vendor. If you buy at rack scale, open both conversations before your 2027 budget cycle closes — allocation for NVLink Fusion partners 2026 designs is being negotiated now, not when parts ship.
How it compares
| Attribute | NVHBM (NVLink Fusion) | Standard HBM4 | AMD Instinct + Infinity Fabric | Google TPU + ICI |
|---|---|---|---|---|
| Memory configuration | Semi-custom per partner design | JEDEC-standard stacks | Fixed per AMD SKU | Fixed per TPU generation |
| Third-party silicon allowed | Yes, via Fusion qualification | Open to any buyer | Limited (UALink roadmap) | No, Google-internal |
| Fabric | NVLink, rack-scale coherent | Fabric-agnostic | Infinity Fabric / UALink | Optical ICI, torus |
| Who controls qualification | Nvidia | JEDEC + memory vendors | AMD / UALink consortium | |
| Software stack | CUDA, NCCL, NVIDIA NIM | N/A | ROCm | JAX / XLA |
| Best for | Partners wanting Nvidia’s rack without its die | Anyone building conventional accelerators | Open-ecosystem buyers | Google Cloud tenants |
The honest read on HBM4 vs NVHBM: they are not competitors. HBM4 is the underlying memory standard everyone will use through the late 2020s. NVHBM is a proprietary integration layer that makes HBM4-class memory addressable on Nvidia’s terms, with configuration flexibility as the carrot and qualification control as the stick. UALink remains the only credible open counter-proposal, and it trails on both silicon and software.
What’s next
Watch the qualification announcements before the silicon. Nvidia will name Fusion partners well ahead of parts shipping, and the identity of those partners tells you where the pressure is. If a hyperscaler with an existing in-house accelerator program signs on, that concedes the fabric and software moat is worth more than vertical independence. If only merchant ASIC vendors sign on, NVHBM is a supply-chain play rather than a strategic one.
The second signal is memory allocation. SK Hynix and Micron capacity commitments for 2027 will reveal whether NVHBM volumes are real or a positioning exercise. Semi-custom stacks fragment manufacturing runs, memory vendors hate fragmentation, and they will price it accordingly. If NVHBM parts carry a meaningful premium over standard HBM4, the flexibility argument weakens fast for anyone not operating at hyperscale volume.
Third, watch the software. Heterogeneous rack-scale AI inference bandwidth is a scheduling problem nobody has solved cleanly. Expect NCCL topology awareness, memory-tier-aware placement in vLLM and SGLang, and eventually a Nvidia-blessed abstraction that hides the difference between its own GPUs and partner accelerators in the same domain. Whoever ships that abstraction sets the terms for the next several years of Nvidia semi-custom silicon, and Nvidia intends to ship it first.
Frequently Asked Questions
Is NVHBM a replacement for HBM4?
No. NVHBM is a semi-custom packaging and controller layer built on HBM4-class memory, designed to integrate with NVLink Fusion. HBM4 remains the underlying JEDEC standard. NVHBM is a specific way to buy and integrate HBM4, not a competing memory technology.
Can any company get NVHBM, or is it invitation-only?
It runs through Nvidia’s NVLink Fusion partner program, which is effectively invitation-based and gated on qualification. The named partners so far — Broadcom, Marvell, Fujitsu — are established silicon designers with existing hyperscaler relationships. A startup accelerator company should not expect access on day one.
Does this help me if I only rent GPUs in the cloud?
Indirectly and later. If partner accelerators with higher memory ratios reach cloud fleets, long-context and high-batch serving gets cheaper per token. Until then, profile your KV-cache pressure so you can pick the right instance type when heterogeneous options appear.
How is this different from CXL memory expansion?
CXL adds capacity over PCIe-class links at meaningfully higher latency, suited to memory tiering and pooling. NVHBM is on-package high-bandwidth memory attached to the accelerator die itself. They solve different problems — CXL for capacity at a latency cost, NVHBM for bandwidth and capacity with no latency penalty.
What does this mean for AMD and the open ecosystem?
It raises the bar. UALink and open-fabric efforts now need to match not just interconnect bandwidth but per-partner memory customization, which requires coordination across multiple companies that Nvidia handles internally. AMD’s counter is openness and price, not feature parity on this axis.
Should this change my 2027 hardware plan right now?
Only if you buy at rack scale. If so, start the partner conversation early and instrument your workloads so you can argue for a specific memory ratio with data. Everyone else should treat this as a signal that memory, not FLOPs, is the axis to optimize for in inference-heavy deployments.
Go deeper than this article
This article covers the essentials. Our premium eguide library gives you the full step-by-step playbooks — prompts, workflows, and copy-paste recipes you can put to work today.
