Will it run on your machine?

Pick a machine and a model. The calculator does the arithmetic: whether the model fits in memory, and how fast tokens will come out.

Inputs

KV cache precision

8-bit halves the cache with a small quality cost. 4-bit quarters it and shows.

Affects the typical response time only. Thinking triples the output tokens.

Estimate

—

Loading data.

Decode —to—tok/s Estimate
Prefill —to—tok/s Estimate
Typical response —
Memory needed —of — GB usable

Estimates. Memory is arithmetic from the model's architecture; speed is bandwidth divided by active weights with a per-layer latency floor, shown as a range. The constants are explained on the About page.