Using local LLMs

Blaze CLI can use local LLMs running on your machine via Ollama. Everything stays on your computer — useful when you do not want code or data to leave the device.

Local LLMs typically lag external APIs in response quality, speed, and tool-calling accuracy. For complex workflows like Salesforce development, Blaze’s first-party route (subscription) or a frontier model via OpenRouter / OpenAI tends to be more reliable.

Prerequisites

1. Install Ollama

Grab the installer from ollama.com — macOS / Windows / Linux are supported.

After installing, open a new terminal and verify:

ollama --version

2. Pull a model

ollama pull llama3.1:8b

Pick a model that supports tool calling (function calling) so it can drive Blaze’s tools. Common picks:

• llama3.1:8b / llama3.1:70b
• qwen2.5-coder:7b / qwen2.5-coder:32b
• mistral-nemo

See the Ollama model library for the full list.

3. Run the Ollama server

In most setups, Ollama starts automatically in the background after installation. To start it manually:

ollama serve

The default listening port is 11434.

Configure Blaze

1. Type /model in Blaze

2. Pick ”+ Use your own API key…” at the bottom of the list

3. From the preset list, pick “Ollama (local LLM)”

4. Fill the form

   • Nickname: leave the default (ollama)
   • Base URL: http://localhost:11434/v1/chat/completions (default)
   • API key: leave blank (Ollama does not require auth)

5. Press Enter — Blaze fetches the models you’ve pulled locally

6. Pick a model and press Enter to finish

The local LLM route now appears in /model. You can switch back to Blaze’s first-party route at any time.

Verify it works

Once connected, send a quick message:

Hello, please introduce yourself

If you get a reply, the connection is live.

Troubleshooting

Model list is empty or fetch fails

• Run ollama list to confirm you have pulled at least one model
• Check that ollama serve (or the Ollama app) is running
• curl http://localhost:11434/api/tags should return JSON

Tools (bash / read / edit, …) don’t get called, or fail

• Confirm your model supports tool calling (look for the tools tag in the Ollama model library)
• Very small models (~3B) often handle tools poorly. Try 7B and above
• If it’s still flaky, try a different family (e.g. qwen2.5-coder)

Responses are very slow

• Local LLMs are bottlenecked by your hardware. An Apple Silicon Mac (M2 Pro or better) or a Windows machine with a discrete GPU is recommended
• If you are running a large model, try an 8B variant — it will be noticeably faster

Different port

• If you’ve changed Ollama’s port, edit the Base URL in the wizard accordingly, e.g. http://localhost:11435/v1/chat/completions

Going back to the first-party route

Pick “Blaze (subscription)” in the /model screen to switch back at any time. Your local LLM setup stays saved so you can flip between routes freely.