STRATA Reserve capacity
Strata/Use cases/LLM inference

GPU servers for LLM inference

Serving a language model is a memory problem first and a compute problem second. Pick the GPU by model size, context length and traffic.

Reserve capacityAll GPUs

Choose by model size

Software that gets the most out of the GPU

Use a serving engine with continuous batching and paged KV-cache, such as vLLM, SGLang or TensorRT-LLM. On the same hardware these can serve several times more requests than naive batching.

On demand or reserved

Steady production traffic is cheapest on a reserved cluster: 12-month terms are 10% below on-demand rates and 24–36 month terms 14% below. Keep on-demand GPUs for traffic peaks and experiments.

Recommended GPUs

NVIDIA H200 141GB
Hopper · 141 GB HBM3e

H200

from $4.31/GPU-hr

Specs and pricing →
NVIDIA H100 80GB
Hopper · 80 GB HBM3

H100

from $3.23/GPU-hr

Specs and pricing →
NVIDIA RTX PRO 6000 Blackwell 96GB
Blackwell · 96 GB GDDR7

RTX PRO 6000

from $2.03/GPU-hr

Specs and pricing →
FAQ
How many GPUs do I need for a 70B model?

One H200 in 8-bit precision, two H100s in 16-bit, or one RTX PRO 6000 in 4-bit. Add replicas as traffic grows.

Can my data stay in one country?

Yes. Choose the region when you reserve capacity; dedicated servers are not shared with other customers.

From the blog