Spot vs reserved GPUs: when the math flips

The eviction tax

Spot GPUs are cheap because the provider can take them back. For batch training, the real cost is checkpoint frequency plus restart time.

Rule of thumb: if your effective interruption rate is above 8% per hour, on-demand almost always wins once you factor in engineer time.

A worked example

• 8xH100 spot at $18/hr, on-demand at $32/hr.
• Training run: 72 hours if uninterrupted.
• Average interruption: every 6 hours, with a 4-minute recovery.

Effective spot cost: $1,458 On-demand cost: $2,304

Spot wins here — but only because we assumed 4-minute recovery. Push that to 15 minutes and the gap closes to under $100.

When spot is a trap

• Distributed jobs > 32 GPUs.
• Anything with strict deadlines (customer demos, benchmark deadlines).
• Fine-tuning where checkpoint restore corrupts optimizer state.