Mac
Benchmarks and the memory ceiling in oMLX
oMLX ships a benchmark tool in the panel that measures time to first token, time per output token, tokens per second and peak memory, with prompts from 1,024 to 200,000 tokens. The default memory ceiling is system RAM minus 8 GB, and that is what decides which context fits.