Reserved GPU Capacity vs Spot: Getting the Accounting Right

A reserved instance or committed-use discount trades a term commitment, often one or three years, for a lower effective rate on guaranteed capacity. Spot capacity is unused inventory the cloud provider sells at a steep discount with no guarantee: it can be reclaimed with short notice. The reserved rate is a known number you can put in a budget. The spot rate floats, and the real cost of spot includes the checkpointing and retry work needed to survive interruption.

Reserved capacity earns its keep when your baseline usage is predictable: a production inference endpoint serving steady traffic, or a training schedule you run every week regardless of demand. If you can look back six months and see the same GPU hours showing up reliably, that's the volume to commit against. Committing against usage you hope will happen, rather than usage you've already seen, is how idle reserved capacity ends up sitting on the balance sheet as sunk cost.

Spot fits work that can tolerate interruption: batch training jobs with checkpointing, offline evaluation runs, and anything you can resume rather than restart. It does not fit a customer facing inference endpoint where a reclaimed instance means a failed request. The teams that get the most out of spot invest early in checkpoint logic, since the discount is only real savings if a reclaimed job resumes cleanly rather than starting over.

Most mature setups run a reserved floor sized to steady state demand, with spot absorbing everything above it. The allocation trap is letting that blend get invisible in the billing export, where reserved and spot usage land in the same line item. Tag workloads by commitment type at the point of launch, not after the invoice arrives, so finance can split the bill by type without archaeology.

Use this checklist to size and allocate a blended GPU setup:

• Set the reserved floor from the GPU hours that show up reliably in your usage history, and commit only against that steady baseline.
• Send interruption tolerant work such as checkpointed batch training and offline evaluation to spot capacity, and keep customer facing inference off it.
• Tag every workload by commitment type when it launches, so reserved and spot usage do not merge into one billing line item.
• Record upfront reserved payments as a prepaid asset amortized over the term, and expense spot and on-demand usage as it is incurred.
• Put the renewal date on the calendar and compare actual usage to the commitment one full billing cycle before it renews.

Reserved commitments paid upfront are typically a prepaid asset amortized over the term, not a lump expense in the month you pay. Spot and on-demand usage is expensed as incurred. Mixing these up either understates a quarter's true infrastructure cost or overstates it, and both versions make gross margin look wrong to whoever is reading it, including the board.

If you cancel a workload mid-term and stop using a reserved commitment you're still paying for, the remaining unamortized balance doesn't just disappear from the books; it typically still needs to be recognized over what's left of the term, or written off if the commitment truly has no further use. Flag any reserved commitment that's underused well before the term renews, so finance isn't the last to know.

Reserved commitments auto-renew far more often than teams expect, and a commitment sized for last year's workload can quietly overshoot this year's actual usage. Put the renewal date on a calendar the same way you would a lease, and revisit actual usage against the commitment at least one full billing cycle before it renews, so there's time to right-size it instead of auto-renewing into another year of an outdated bet.

Say your training workload runs steady in the background most weeks, with bursts around a launch that roughly double demand for a short stretch. Sizing the reserved floor purely on the steady weeks leaves the burst covered only by full on-demand pricing, since spot capacity isn't reserved by definition and can simply not be there when demand spikes hardest.

Model both pieces separately before signing anything: price the steady floor at the reserved rate, and price the burst at a blended assumption that mixes spot with a fallback to on-demand for whatever spot can't cover in time. If the burst is frequent enough to plan around, a smaller secondary reservation sized to the burst, rather than the full peak, usually beats paying full on-demand rates every time it happens.

The mistake to watch for is sizing the reserved commitment to the peak instead of the floor, on the logic that it covers everything. That looks safer on paper and is actually the more expensive path, since it pays the higher committed rate for capacity that mostly sits idle between bursts, which is the exact idle-capacity problem a reserved commitment is supposed to avoid in the first place.