LM Studio: Exploring AI Models from Your Desktop

LM Studio^[1] is a desktop app (Mac, Windows, Linux) that downloads and runs local LLMs with a polished UI: no terminal, no complicated setup. Open, pick model, chat. For exploratory developers, data analysts, journalists with sensitive data, and anyone wanting to try LLMs without sending queries to the cloud.

Key takeaways

• LM Studio runs local LLMs (llama.cpp under the hood) with a polished chat UI and no terminal required.

• The local OpenAI-compatible API allows existing code to work without changes pointing to localhost:1234.

• Integrated RAG with documents (PDF, TXT, DOCX) keeps everything local: zero cloud exposure.

• For personal and single-user use, LM Studio is superior to Ollama in UX. For teams, Ollama + OpenWebUI is more flexible.

• For production or simultaneous multi-user, neither works: use vLLM or TGI.

What LM Studio does

Main features:

• Model download from Hugging Face^[2] with one click.

• Local execution over llama.cpp^[3], the inference engine that does the work under the hood.

• Polished chat UI.

• Local OpenAI-compatible API that other apps can consume.

• RAG with your documents (PDF, TXT, DOCX): chat directly with your files.

• Side-by-side model comparison.

• GPU offloading configurable (CPU+GPU hybrid).

Installation and first models

Download from lmstudio.ai^[1]. DMG for Mac, MSI for Windows, AppImage for Linux. The first time, the setup wizard asks you to pick a model. Recommended starting points:

• Apple Silicon Mac: Llama 3 8B Q4_K_M (~5 GB) or Phi-3 Mini (3 GB).

• PC with 16 GB RAM: Mistral 7B Q4 (~4 GB) or Phi-3.

• PC with 32 GB+ RAM: Mixtral 8x7B Q4 (~25 GB) or a quantised Llama 3 70B.

For modest hardware: Phi-3 Mini (3.8B) has an excellent quality-to-size ratio; Gemma 2B is lightweight.

OpenAI-compatible API: the hidden value

LM Studio exposes an API compatible with OpenAI’s^[4] at localhost:1234. Existing code works without changes:

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:1234/v1",
    api_key="not-needed"
)

response = client.chat.completions.create(
    model="local-model",
    messages=[{"role": "user", "content": "Hi"}]
)

Useful for offline development, privacy-sensitive apps, or as a fallback if the cloud API is unavailable.

Local RAG with Your Documents

LM Studio integrates ingestion and RAG directly in the UI:

• Drag PDFs/docs to the chat.

• System extracts text and generates local embeddings.

• Chat uses relevant context from your docs.

For lawyers, doctors, and journalists who handle confidential data, the advantage is clear: nothing leaves your machine. The document store stays on your own disk.

Performance by hardware

On Apple Silicon M2/M3:

• Llama 3 8B Q4: 30-50 tokens/s on M2 Pro.

• Mixtral 8x7B Q4: 15-25 tokens/s on M3 Max 64 GB.

• Llama 3 70B Q4: 5-10 tokens/s if it fits in unified memory.

On Windows with an NVIDIA GPU:

• RTX 4090: Llama 3 70B Q4 at ~15 tokens/s.

• RTX 4070/4080: 7B-13B are the sweet spot.

• Laptop with a 3050/4050: limited; CPU inference works better.

LM Studio vs Ollama vs OpenWebUI

LM Studio wins on UX for non-technical, individual use. Ollama^[5] wins on dev/CLI stack integration and on being open source. OpenWebUI^[6] is the choice for teams that want a self-hosted, multi-user panel.

Performance tuning

Three settings matter most for getting the most out of your hardware:

• GPU layers: how many layers of the model go to the GPU. More is faster but needs more VRAM.

• Context length: maximum tokens. Lower is faster and uses less memory.

• Thread count: for CPU inference, match physical cores (not logical ones from hyperthreading).

Honest limitations

• Closed source, though free to use. Some lock-in risk.

• Update cadence depends on the LM Studio team.

• Not easy to integrate into CI pipelines.

• Single machine: it does not distribute inference across nodes.

• Telemetry is optional but worth checking in the settings.

Conclusion

LM Studio is the best option for people who want to explore local LLMs with a polished interface. For teams, Ollama + OpenWebUI offers more flexibility. For production, neither works: use vLLM or TGI.

LM Studio occupies a specific but important niche: democratising access to local LLMs for non-technical users. Being free and well-finished makes it the obvious choice in its category. For anyone handling private data or wanting to experiment without paying for APIs, it’s worth downloading.

Frequently asked questions

Can I use code written for the OpenAI API with LM Studio without changes?

Yes. LM Studio exposes an OpenAI-compatible API at localhost:1234, so you point the client at base_url="http://localhost:1234/v1" with any api_key, since none is needed. It is useful for offline development, privacy-sensitive apps, or as a fallback if the cloud API is unavailable.

Which model should I start with for my hardware?

On an Apple Silicon Mac, Llama 3 8B Q4_K_M (~5 GB) or Phi-3 Mini (3 GB); on a PC with 16 GB RAM, Mistral 7B Q4 (~4 GB) or Phi-3. With 32 GB or more, Mixtral 8x7B Q4 (~25 GB) or a quantised Llama 3 70B. On modest hardware, Phi-3 Mini has an excellent quality-to-size ratio and Gemma 2B is lightweight. As a reference, Llama 3 8B Q4 runs at 30-50 tokens/s on an M2 Pro.

Is LM Studio suitable for a team or for production?

No. It is single-user with no multi-user support, it does not distribute inference across nodes and it is not easy to integrate into CI pipelines. It is also closed source, though free, and the optional telemetry is worth checking in the settings. For teams, Ollama with OpenWebUI offers a self-hosted multi-user panel; for production or simultaneous multi-user load, use vLLM or TGI.

Sources

1. LM Studio
2. Hugging Face
3. llama.cpp
4. compatible with OpenAI’s
5. Ollama
6. OpenWebUI