Teams spend weeks negotiating a few cents off the hourly rate, then run their GPUs at 30% utilization. The biggest saving is usually not a cheaper GPU, but making the one you have do useful work.
Where idle capacity hides
- Allocated but quiet: a cluster reserved for peak traffic that sits mostly empty at night and on weekends.
- Busy but inefficient: a GPU that shows high utilization but serves small batches, so each token costs more than it should.
- Waiting on something else: data loading, network calls or a CPU preprocessing step that leaves the GPU stalled between bursts.
- Forgotten: notebooks and test pods that nobody shut down.
Make it visible first
Put three numbers on one dashboard per service: GPU hours paid, GPU utilization, and useful output such as tokens, images or training steps. Divide paid hours by useful output and you have a cost per unit of work that shows waste immediately.
Then close the gaps
| Gap | Fix |
|---|---|
| Night and weekend idle | Scale on demand, or move the base to a smaller reserved block plus on-demand peaks |
| Small batches | Continuous batching and request queuing in the serving engine |
| Stalls between steps | Faster data pipelines, caching, overlapping I/O with compute |
| Forgotten resources | Auto-stop rules and alerts for GPUs idle longer than an hour |
Price matters less than you think
Moving from 30% to 70% utilization cuts the cost per unit of work by more than half. No provider discount comes close. On Strata, idle on-demand servers can be stopped automatically when your balance or schedule says so, and reserved clusters can be sized to the baseline you actually use.
Need GPUs for this?Reserve capacity
