Local AI, measured.
Will that model run on your machine, and how fast? Arithmetic, not opinion.
Pick a machine and a model.
Memory is a room, bandwidth is the door. The diagram is the calculator's arithmetic, live.
The one idea
Memory is a room. Bandwidth is the door. A model is what you are trying to get through the door.
-
Capacity is the room
Gigabytes decide whether the model fits at all. Weights first, then the cache, then what the system keeps for itself.
-
Bandwidth is the door
Every token means reading the active weights once. Gigabytes per second divided by those bytes is your speed.
-
Mixture of experts is a trick
It fills the room but pushes only a sliver through the door per token. Big model, small read, high speed.
-
The cache is furniture
Each token in the conversation leaves something behind. Long context fills the room even before the model answers.
The guide
Fourteen chapters that make the calculator's numbers mean something. Read in order or jump to what you need.