Tool
Local LLM Calculator
Will that model fit in your GPU? Is self-hosting cheaper than the API? Estimate the VRAM you need and the cost break-even point.
Will it fit your GPU?
– needed
–
–
- Model weights: –
- KV cache: –
Cheaper than the API?
Self-hosted
–
hardware – · power –
API
–
–
Break-even: –
How it’s calculated
The VRAM needed to serve a model is the sum of three things:
- Model weights = number of parameters × bytes per parameter. Quantization shrinks those bytes: FP16 uses 2 bytes; a Q4_K_M GGUF about 0.56.
- KV cache = memory for the context, proportional to context length, number of layers and attention heads (GQA makes the cache smaller).
- Overhead ≈ 1 GB for the CUDA/Metal context and activations.
Self-hosting vs the API
Self-hosting has a high fixed cost (the hardware, amortized) and a low marginal cost (just electricity). The API is the opposite: zero fixed, cost per token. That’s why self-hosting only pays off above a certain volume — the break-even point the tool computes.
Estimates are approximate: real throughput depends on the runtime (llama.cpp, vLLM), batching and cache policy. Treat them as an order of magnitude, not an exact figure.