Open models with tool calling: which to choose

Any chat model can hold a conversation, but only some open models emit reliable tool calls, and that is the difference between a demo and an agent that works. Tool calling (or function calling) is what lets a model decide when to query an API, a database or a calculator instead of making the answer up. In this guide you will see why not every open model does it equally well, which families stand out, how to read the Berkeley Function-Calling Leaderboard, why the template and the parser matter as much as the model, and how to choose based on your hardware. The same explanation is available in Spanish.

Key takeaways

• Tool calling depends on fine-tuning, not size: an 8B model trained for it beats a 70B model that never saw tool examples.
• Four open families stand out today: Qwen3 (Apache 2.0, from 0.6B to 235B-A22B), Nous Hermes 3 and 4, Llama 3.1 and Mistral.
• The Berkeley Function-Calling Leaderboard (BFCL), in its V4 edition, is the public reference: it scores calls with abstract syntax tree (AST) matching and with real execution.
• The template and the parser are critical: each family wraps the call differently, and servers like vLLM ship a specific parser (hermes, mistral, llama3_json) that you must switch on.
• You can test tool calling on your own machine: Ollama has supported tools since 25 July 2024 and runs Qwen3 with no hacks.

Why don’t all open models use tools the same way?

A model does not "run" anything itself. When you describe a function (name, description and parameters), all it does is generate structured text that says "call this function with these arguments"; your code runs it and returns the result. Whether that text comes out well formed, with the right name and the right types, depends entirely on whether the model was fine-tuned with examples of that format.

That is the catch. Many open models are excellent at conversation but never saw thousands of tool-call examples during training, so they improvise: they skip a required parameter, invent a function that does not exist, or return broken JSON. That is why a small model trained explicitly for tools, such as an 8B Qwen3, is usually more reliable in an agent than a much larger general-purpose model.

The practical upshot is that you should not choose by a model’s overall ranking, but by its concrete performance on function tasks. For that, it pays to look at two things: a specific benchmark and your inference server’s support.

Which open families handle tool calling best?

Four lineages concentrate almost all serious open tool work today. Qwen3, released on 28 April 2025 under the Apache 2.0 licence, is the safest bet: it carries tool calling built into its chat template, trains reasoning and tools at once, and spans from 0.6B to the 235B-A22B giant (a mixture-of-experts model with 22B active parameters). Its training on roughly 36 trillion tokens across 119 languages also makes it very capable in Spanish.

Nous Hermes (3 and 4) start from Llama’s weights and add neutral alignment and their own tool format with <tool_call> tags; they are the choice when you want a very steerable model. Meta’s Llama 3.1 brings its own tool-calling template and was one of the first open models Ollama supported. And Mistral (from Mistral 7B Instruct v0.3) offers native function calling with a [TOOL_CALLS] token and fast, compact models.

If you are building your first local agent, Qwen3 and Nous Hermes 3 are the two names to start with.

What is the Berkeley Function-Calling Leaderboard?

The Berkeley Function-Calling Leaderboard (BFCL), from the Gorilla team at the University of California, Berkeley, is the reference public benchmark for measuring how well a model calls functions. Rather than testing general knowledge, it evaluates exactly what an agent needs: given a prompt and a set of function schemas, does the model emit the correct call, with the right arguments and types?

Its current edition, V4, has grown in layers. The first version introduced abstract syntax tree (AST) matching; the second added enterprise and community-contributed functions; the third brought multi-turn conversations; and V4 adds an "agentic" layer with a web-search category. Inside, it measures simple, multiple and parallel cases, plus hard situations such as a missing parameter (the model should ask, not invent) or no function fitting (it should recognise that).

It scores in two complementary ways. AST matching parses the call as a tree and checks name, parameters, values and types regardless of order; it is fast and deterministic. Executable evaluation actually runs the call against real or simulated APIs and compares the result; it is slower, but proves the call works. Qwen-family models consistently lead the open ranking, though it pays to check the live table before deciding, because it changes with every release.

How do the template and parsers matter?

Here is the detail that trips many people up: the model emits the call wrapped in a specific format, and your server has to know how to unwrap it. Each family uses a different convention. Qwen and Hermes wrap the call in <tool_call> tags on the ChatML template; Llama 3.1 uses its own JSON template; Mistral emits a special [TOOL_CALLS] token. If you serve the model with the wrong parser, the call text arrives intact as if it were a normal reply and your agent breaks.

That is why vLLM, the most common production inference server, ships a parser per family that you switch on at start-up. For Qwen and Hermes you use hermes; for Mistral, mistral; for Llama 3.1, llama3_json. The documentation sums up the key point: for Qwen2.5 "the chat template already includes support for Hermes-style tool use, so you can use the hermes parser".

# Serve Qwen3 with the correct tool parser (Hermes style)
vllm serve Qwen/Qwen3-8B \
  --enable-auto-tool-choice \
  --tool-call-parser hermes

  # For a Mistral model, the parser changes
vllm serve mistralai/Mistral-7B-Instruct-v0.3 \
  --enable-auto-tool-choice \
  --tool-call-parser mistral

In Ollama this is simpler, because the correct parser is already enabled for compatible models. You can pass the functions directly and get the structured call back:

from ollama import chat

def temperature(city: str) -> str:
    """Return a city's temperature."""
    return {"Madrid": "22C", "London": "15C"}.get(city, "?")

resp = chat(
    model="qwen3",
    messages=[{"role": "user", "content": "What is the temperature in Madrid?"}],
    tools=[temperature],
)
print(resp.message.tool_calls)

If you work with fully deterministic outputs, this same problem is solved through constrained generation; it is explained in function calling with Ollama on your own machine.

How do you choose based on your hardware?

Tool calling does not demand a huge model, so what matters is how much VRAM you have. With a consumer GPU or even a CPU with quantisation, a 7B to 8B model is enough for agents with a few well-defined tools. Beyond that, going larger mostly improves reasoning when you have to pick among many tools or chain several steps.

The rule of thumb: start small, measure call reliability on your own tasks, and scale up only if you fall short. An 8B Qwen3 with Ollama is a sensible starting point, and from there you can move to vLLM when you need to serve the model in production at high concurrency.

Frequently asked questions

Can a small open model use tools better than a large general-purpose one?

Yes, and it happens often. Reliability at emitting a well-formed call depends on fine-tuning, not the parameter count. An 8B model trained explicitly for tool calling, such as Qwen3 8B or Hermes 3 8B, usually gets it right more often than a much larger general-purpose model that never practised that format. That is why size is the last thing you should look at.

Do I need the internet to run an open model with tools?

Not for the model. You download the weights once and the model runs entirely on your machine with Ollama, vLLM or llama.cpp, sending nothing to a paid API. The tools are another matter: if one of them queries an external API, a search engine or the weather, that specific function will need the network when your code runs it.

How do I know which model to choose today?

Cross three sources. Check the Berkeley Function-Calling Leaderboard for pure function performance, confirm in the Ollama library or on Hugging Face that the model has tool support, and make sure your inference server ships a parser for its template. If all three line up, test the model on your own tasks before committing.

Conclusion

Choosing an open model with tool calling is not about finding the biggest one, but the one that was trained for the task and that your infrastructure can interpret. Start with one of the four solid families (Qwen3, Hermes, Llama 3.1 or Mistral), check its position on the Berkeley Function-Calling Leaderboard, make sure you serve it with the correct parser, and match it to your VRAM. With that you will have the foundation of an agent that calls tools reliably and that you control end to end.

Sources

1. Berkeley Function-Calling Leaderboard V4, Gorilla team
2. Qwen3: official announcement from the Qwen team
3. Ollama, tool support (25 July 2024)
4. vLLM, tool calling and parsers documentation

Route: Self-hosted Agentic Models and Tool Calling