How to Connect Local AI Models to Bruno AI: Ollama and LM Studio

A good chunk of API testing is repetitive. Writing the same status-code assertion for the twentieth endpoint, documenting a request you already understand, or squinting at a 401 to work out which header is wrong. That is the kind of manual work an AI assistant removes well and Bruno v4 ships one. What makes it interesting is that it does not require a cloud provider at all. You can point it at a model running on your own laptop through Ollama or LM Studio, and no request, header, or response ever leaves your machine.

This guide covers what local models are, why you would choose one, how to run them with Ollama and LM Studio, and exactly how to wire them into Bruno. If you want the background on how AI was designed into Bruno in the first place — the bring-your-own-key model and the redaction layer then read Introducing AI in Bruno: Your Key, Your Provider, Your Data Stays Put first.

What Are Local / Open Models?

When you use a hosted assistant, your prompt travels over the internet to a provider's servers, a model you cannot inspect generates a reply, and the reply comes back. You are renting access to weights someone else owns.

An open-weight model is one whose trained parameters are published for download as families such as Llama, Qwen, Mistral, Gemma, and DeepSeek. Because you can download the weights, you can run the model on your own hardware. That is what "running a local model" means: a program on your laptop loads a multi-gigabyte file into memory and generates text on your own CPU or GPU.

Ollama and LM Studio are the two most popular tools for doing this. Both handle the download, the memory management, and  crucially for Bruno both expose a small local web server that speaks the OpenAI API format. That shared format is the bridge: as far as Bruno is concerned, a model running on localhost looks the same as one running in a data centre.

Benefits of Using Local Models

The honest trade-offs

Local models are genuinely useful for this workload, but it is worth setting expectations before you spend an hour on setup:

• Quality is lower than frontier hosted models. A 7B model running on a laptop will not match Claude Opus or GPT-5 at complex reasoning. For generating a test assertion or a docstring, it is usually fine. For "explain this gnarly distributed failure", it is not.
• Speed depends entirely on your hardware. On an Apple Silicon Mac or a machine with a decent GPU, a 7B model is fast. On an older laptop running CPU-only, expect to wait.
• They eat RAM. A loaded model occupies memory for as long as it is running, alongside Bruno and everything else you have open.
• Setup is on you. No support ticket when a model misbehaves, you swap it for a different one.

A practical middle ground: use a small fast local model for inline autocomplete, and a hosted model for the chat sidebar when you need real reasoning. Bruno lets you configure both at once and pick per surface.

Picking a Model for Your Hardware

Model names carry a parameter count as 7b means seven billion parameters. Bigger is smarter and slower. Nearly every model you download is quantized, which compresses the weights to roughly half the size at a small quality cost. The default quantization in both Ollama and LM Studio (Q4) is the right choice for almost everyone.

Use this as a rough guide. The memory column means unified memory on Apple Silicon, or VRAM on a discrete GPU:

For API work specifically, a code-tuned 7B model is the best starting point for something from the Qwen Coder or DeepSeek Coder families. They were trained heavily on code, so they produce cleaner JavaScript for Bruno's test and script editors than a general chat model of the same size.

Running Local Models with Ollama

Ollama is the lighter of the two. It runs as a background service with a command-line interface, and it starts an OpenAI-compatible server automatically and there is no "start server" button to find.

Step 1:  Install Ollama

Download the installer for macOS, Windows, or Linux from ollama.com/download. On Linux you can use the install script:

curl -fsSL https://ollama.com/install.sh | sh

Confirm it is running:

ollama --version

Step 2: Pull a model that fits your machine

Pick one line based on the hardware table above:

# 8 GB machine — small and quick, best for autocomplete
ollama pull qwen2.5-coder:3b

# 16 GB machine — recommended default for API work
ollama pull qwen2.5-coder:7b

# general-purpose alternative at the same size
ollama pull llama3.1:8b

# 24 GB or more — better reasoning in the chat sidebar
ollama pull qwen2.5-coder:14b

Verify the download and note the exact name in the first column because you will need it in Bruno:

ollama list

NAME                     ID              SIZE      MODIFIED
qwen2.5-coder:7b         2b0496514337    4.7 GB    2 minutes ago

Step 3: Confirm the OpenAI-compatible endpoint

Ollama listens on port 11434 and exposes an OpenAI-compatible API under /v1. Test it before touching Bruno — if this fails, Bruno will fail too:

curl http://localhost:11434/v1/models

A JSON list of your installed models means you are ready. Keep these two values handy:

Running Local Models with LM Studio

LM Studio is the graphical alternative. If you would rather browse models with a search box, see download sizes before committing, and start a server with a toggle, this is the better fit. It also shows you live logs of every request Bruno sends, which is genuinely useful when debugging the connection.

Step 1: Install LM Studio

Download the desktop app for macOS, Windows, or Linux from lmstudio.ai and install it like any other application.

Step 2: Download a model

Step 3: Start the local server

This is the step people miss. Unlike Ollama, LM Studio does not serve an API until you switch the server on.

Verify from a terminal:

curl http://localhost:1234/v1/models

Ollama or LM Studio?

Both expose the same OpenAI-compatible API, so Bruno treats them identically. Only the port and the model ID differ.

How to Integrate a Local Model with Bruno

Make sure Ollama is installed, or the LM Studio server is toggled on, before you start. Bruno will try to reach the endpoint as soon as you save it.

Step 1: Turn on AI features

Step 2: Add your local model as an OpenAI-compatible endpoint

This is the key step. Local models do not go under the OpenAI or Anthropic rows — they go under OpenAI-Compatible Endpoints.

Step 3: Pick a default model and configure autocomplete

Set your local model as the default, then open the Autocomplete section. Trigger mode matters more with a local model than a hosted one, because every suggestion costs you real compute:

The keymap is the same regardless of provider: Tab accepts the full suggestion, ⌘ → accepts one word, Esc dismisses, and ⌘ \ triggers manually.

Step 4: Leave the security defaults alone

Bruno redacts sensitive data before building any prompt: Authorization and other credential headers, body keys like password and *_token, secret environment variables, and response bodies (which are sent as a type-only shape, so "sk-abc123" becomes "<string>"). All four rules are on by default.

With a local model you could argue the redaction is redundant, since the data is not leaving your machine anyway. Leave it on regardless. It costs you nothing in quality and the model still sees key names, structure, and types, which is all it needs to write a correct assertion and it means the day you switch that default to a hosted model, you are not suddenly leaking production tokens.

If your organisation is on the Ultimate Edition, License Administrators can centrally govern which providers and models are permitted via AI Policy is worth checking before you roll local endpoints out to a team.

Using It: Tests, Docs, Scripts, and Debugging

Once the endpoint is saved, every AI surface in Bruno works the same way it would with a hosted provider. Generated output always arrives as a proposal you review — click Apply to insert it or Back to discard. Nothing is written to your files without your click.

Getting better results from a smaller model

A 7B model rewards specific prompts far more than a frontier model does. Vague instructions produce vague output. A few habits that help:

• Name the fields. "Assert that data.user.id is a number and data.user.email is a string" beats "write tests for this".
• Send the request first. With a response in hand, the assistant sees the actual shape and picks correct property paths instead of guessing.
• Ask for one thing at a time. Small models drift when asked to generate tests, docs, and a script in one go.
• Use a code-tuned model for code surfaces. The difference between a general 7B chat model and a 7B coder model is very visible in the Tests and Script tabs.
• Reroll rather than fight. Regenerating is cheap when inference is free.

Troubleshooting

Wrap Up

The setup is short: install Ollama or LM Studio, pull a model that fits your memory, then add it in Bruno under Preferences → AI → OpenAI-Compatible Endpoints with a base URL of http://localhost:11434/v1 or http://localhost:1234/v1. Ten minutes, most of it download time.

What you get is an assistant that writes your repetitive assertions and docstrings without a subscription, without a rate limit, and without a single byte crossing your network boundary. For teams where sending API traffic to a third-party model was never going to clear review, that difference is the whole point.

And if you decide it is not for you, the toggle goes back off and Bruno is exactly the API client it was before.