Renting an H100 by the hour used to mean a sales call, a three-month minimum, and a credit check. In 2026 it means a CLI command and a spot bid. Prime Intellect and Modal have converged on the same customer — the solo builder or five-person team fine-tuning a 7B model on a Tuesday — from opposite directions, and July’s pricing shifts blew the gap between them wide open. If you are hunting cheap GPU rental for AI training right now, the difference between the right broker and the wrong one on identical silicon is not 15 percent; it is a multiple.
What’s new in cheap GPU rental for AI training
The headline change is structural, not numeric. Prime Intellect operates as an aggregator: it indexes capacity across dozens of underlying providers — neoclouds, decentralized contributors, second-tier datacenters — and surfaces them as one searchable market. Query for eight H100 SXM cards and you get a ranked list with hourly rates, interconnect specs, and region, then rent the cheapest thing that meets your bar. The Prime Intellect GPU marketplace is Kayak for accelerators, and like Kayak, its value comes from the spread between the cheapest and most expensive listing for a functionally identical seat.
Modal took the other road. It is a serverless container platform where GPUs attach to functions, billed per second with genuinely fast cold starts. You never see a marketplace. Decorate a Python function, declare gpu="H100", and Modal provisions, runs, and tears down. Modal serverless GPU pricing is published, flat, and per-second — no bidding, no preemption, no negotiation. You pay a premium per GPU-hour versus the marketplace floor, and in exchange you pay zero for idle time and zero in engineering hours spent on infrastructure.
Mid-2026 broke differently because B200 supply finally loosened. Blackwell parts that were allocation-gated through 2025 started appearing in spot pools, dragging H100 rates down across the board as older silicon competed with newer. Marketplace H100 spot listings now routinely clear in the low single-digit dollars per hour, on-demand marketplace rates sit meaningfully above that, and serverless per-second pricing sits higher still. Every tier is defensible. Paying serverless rates for a 40-hour uninterrupted training run is not.
Why it matters
- The arbitrage now exceeds most teams’ entire tooling budget. A 200-hour fine-tuning project at a marketplace spot rate versus the same job on a premium on-demand provider is a four-figure swing — a hire’s worth of runway for a seed-stage team.
- Workload shape dictates vendor, not the other way around. Long, checkpointable, latency-tolerant training belongs on spot. Bursty inference, evals, and data-prep jobs that idle between invocations belong on serverless. Teams that pick one vendor for everything overpay on half their workload by construction.
- Annual reservations became a liability. Signing a 12-month H100 commitment in 2026 locks you out of the B200 price decline you are about to benefit from. The spot GPU vs reserved instances calculus has inverted for anyone whose utilization sits below roughly 70 percent.
- Checkpointing is a cost lever, not hygiene. If your training loop resumes from an arbitrary step, preemption is an annoyance. If it cannot, preemption is a total loss and forces you into the expensive tier. Fifty lines of checkpoint code is worth thousands.
- Egress and storage quietly dominate small jobs. Moving a 400GB dataset to a cheap region three times costs more than the GPU-hour savings. The cheapest compute attached to the wrong storage is not cheap.
- Multi-node interconnect is the real differentiator. Marketplace listings vary wildly on whether nodes carry InfiniBand or plain Ethernet. For single-node work it is irrelevant; for anything sharded across nodes it is the whole ballgame.
How to use it today
-
Price the market before you commit. Prime Intellect’s CLI queries availability and rates directly, so you can see the H100 hourly rate 2026 spread across providers in one shot.
pip install prime prime login prime availability list --gpu-type H100_80GB --gpu-count 8 prime availability list --gpu-type B200 --gpu-count 1 --region united_states -
Launch a spot pod and treat it as disposable. Never store anything on the pod that you cannot lose in the next sixty seconds.
prime pods create \ --name ft-qwen-run3 \ --gpu-type H100_80GB \ --gpu-count 2 \ --disk-size 500 \ --image ubuntu_22_cuda_12 prime pods list prime pods ssh ft-qwen-run3 -
Make the training loop preemption-proof. This is the highest-ROI change for reducing fine-tuning compute costs. Checkpoint to object storage on a step interval, and always resume from the latest checkpoint on boot.
# train.py — resume-first pattern import os, glob, torch CKPT_DIR = "/mnt/ckpt" def latest_checkpoint(): files = sorted(glob.glob(f"{CKPT_DIR}/step_*.pt")) return files[-1] if files else None start_step = 0 ckpt = latest_checkpoint() if ckpt: state = torch.load(ckpt, map_location="cpu") model.load_state_dict(state["model"]) optimizer.load_state_dict(state["optim"]) start_step = state["step"] + 1 print(f"resuming from step {start_step}") for step in range(start_step, total_steps): loss = train_one_step() if step % 200 == 0: torch.save( {"model": model.state_dict(), "optim": optimizer.state_dict(), "step": step}, f"{CKPT_DIR}/step_{step:07d}.pt", ) os.system(f"aws s3 sync {CKPT_DIR} s3://my-bucket/ckpt/ --quiet") -
Put bursty work on Modal instead. Anything that runs for ninety seconds and then sits idle should never hold a rented pod. Declare the GPU on the function and let it scale to zero.
import modal image = ( modal.Image.debian_slim() .pip_install("torch", "transformers", "accelerate") ) app = modal.App("eval-harness", image=image) vol = modal.Volume.from_name("model-cache", create_if_missing=True) @app.function( gpu="H100", volumes={"/cache": vol}, timeout=1800, scaledown_window=60, ) def run_eval(prompt: str) -> str: from transformers import pipeline pipe = pipeline("text-generation", model="/cache/my-model", device=0) return pipe(prompt, max_new_tokens=256)[0]["generated_text"] @app.local_entrypoint() def main(): for out in run_eval.map(["explain LoRA", "explain FSDP"]): print(out)Deploy and invoke it with:
pip install modal modal setup modal run eval_harness.py modal deploy eval_harness.py -
Instrument cost per run, not cost per month. Log wall-clock GPU-seconds per experiment and divide. If you cannot answer “what did run 14 cost,” you cannot optimize anything.
#!/usr/bin/env bash # cost.sh — rough per-run GPU spend START=$1 # unix seconds END=$2 RATE=${3:-2.40} # $/GPU-hour GPUS=${4:-2} python - <<PY hrs = ($END - $START) / 3600 print(f"GPU-hours: {hrs * $GPUS:.2f} | cost: \${hrs * $GPUS * $RATE:.2f}") PY -
Co-locate data with compute. Pin your dataset bucket to the same region you rent in, and cache model weights on a persistent volume so you are not re-downloading 30GB of safetensors on every cold start.
How it compares
| Dimension | Prime Intellect | Modal | Hyperscaler on-demand |
|---|---|---|---|
| Model | Aggregated marketplace, spot + on-demand | Serverless containers, per-second | Reserved or on-demand VMs |
| Relative H100 cost | Lowest (spot); low-mid (on-demand) | Mid-high per GPU-hour, zero idle | Highest |
| Preemption risk | Real on spot; none on reserved listings | None | None on-demand |
| Time to first token of work | Minutes (pod boot) | Seconds (cold start) | Minutes to weeks (quota) |
| Multi-node training | Strong — cluster listings with InfiniBand | Limited; single-node oriented | Strong but expensive |
| Ops burden | You manage the box | Near zero | High |
| Best fit | Long training runs, sharded fine-tunes | Inference, evals, bursty pipelines | Compliance-bound enterprise |
| B200 cloud availability | Growing spot supply | Available on request tiers | Allocation-gated |
What’s next
Expect the spread to compress, then re-open. As B200 and successor parts land in volume, H100 spot pricing will keep sliding — good news if you are renting, bad news if you bought. The second-order effect is that aggregators get stronger as supply fragments: the more distinct providers exist, the more valuable a single search interface becomes. Prime Intellect’s bet is that compute becomes a commodity with a price feed. That bet is looking correct.
On the serverless side, watch cold-start times and the size of the models that can be held warm economically. Modal’s moat is engineering — snapshotting, image caching, container reuse — and every second shaved off cold start expands the set of workloads where paying a per-GPU-hour premium still nets out cheaper than idling a rented box. If warm-pool economics improve enough, serverless starts eating into medium-length training jobs, not just inference.
The third thing to watch is the middle: reserved capacity resold on secondary markets. Teams that signed 2025 annual commitments are sitting on capacity they no longer need, and marketplaces that let them offload it will pull a lot of supply into the spot pool. If that liquidity arrives, the spot GPU vs reserved instances question stops being about risk tolerance and becomes purely about whether your job can checkpoint.
Frequently Asked Questions
Is spot GPU capacity reliable enough for real training runs?
Yes, with one condition: your training loop must resume from a checkpoint without manual intervention. Preemption on a well-instrumented run costs you the minutes since the last checkpoint, nothing more. On an uninstrumented run it costs you everything. Build the resume path first, then rent spot.
Which is cheaper overall, Modal or Prime Intellect?
It depends on utilization. If your GPU stays busy more than about 60-70 percent of the time it is rented, the marketplace wins on raw rate. If your workload is spiky — an eval suite that runs twice a day, an inference endpoint with quiet nights — Modal’s per-second billing and scale-to-zero usually beat a pod you pay for while it sleeps. Most teams should use both.
What H100 hourly rate counts as a good deal in 2026?
Treat the marketplace spot floor as your benchmark and refuse to pay more than roughly double it for on-demand convenience. Anything approaching hyperscaler list pricing for a single-node fine-tune signals you are buying a brand, not silicon. Confirm the listing’s VRAM, interconnect, and host CPU before comparing — an H100 PCIe seat and an H100 SXM seat are not the same product.
Do I need B200s, or are H100s still fine for fine-tuning?
For most fine-tuning under 70B parameters with LoRA or QLoRA, H100s remain the value pick, and improving B200 cloud availability matters mainly because it is pushing H100 prices down. Reach for Blackwell when memory bandwidth or VRAM per card is the binding constraint, not out of reflex.
How do I avoid surprise costs beyond the GPU line item?
Audit three things: egress charges when moving datasets between regions or providers, persistent storage you forget to delete after a project ends, and pods left running overnight. Forgotten idle pods are the single most common source of wasted spend among small teams. Set a hard teardown in your launch script rather than trusting yourself to remember.
Can I run multi-node distributed training on these platforms?
On Prime Intellect, yes — filter specifically for cluster listings with InfiniBand or equivalent high-speed interconnect, because Ethernet-connected nodes bottleneck gradient synchronization badly. Modal is oriented toward single-node GPU functions and fits sharded multi-node training poorly; use it for the surrounding pipeline instead.
Go deeper than this article
This article covers the essentials. Our Technical & Coding eguide collection gives you the full step-by-step playbooks — prompts, workflows, and copy-paste recipes built for exactly this work.
