> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/cline/cline/llms.txt
> Use this file to discover all available pages before exploring further.
# Running Models Locally
> Run AI models on your own hardware with Ollama or LM Studio. No API costs, no data leaving your machine.
Local models let you use Cline completely offline. Your code never leaves your machine, there are no per-token charges, and you can run as many tasks as your hardware allows.
The tradeoff is hardware requirements and speed. Local models generate tokens at 5–20 tokens/second on consumer hardware, compared to 50–200+ tokens/second from cloud APIs. With the right setup, they handle real development work reliably.
## Hardware requirements
Your available RAM determines which models you can run:
| RAM | Recommended model | Quantization | Notes |
| ------- | ----------------- | --------------- | ----------------------------------- |
| 32 GB | Qwen3 Coder 30B | 4-bit (\~17 GB) | Entry-level local coding |
| 64 GB | Qwen3 Coder 30B | 8-bit (\~32 GB) | Better quality, full Cline features |
| 128 GB+ | GLM-4.5-Air | 4-bit | Cloud-competitive performance |
Most models under 30B parameters fail with Cline's tool-use format — they produce broken outputs, refuse commands, or lose context across files. Stick to **Qwen3 Coder 30B** or larger unless you've specifically verified a smaller model works.
## Recommended model: Qwen3 Coder 30B
After extensive testing, **Qwen3 Coder 30B** is the most reliable model under 70B parameters for use with Cline:
* **256K native context window** — handles entire repositories
* **Strong tool-use** — reliable command execution with Cline's format
* **Consistent outputs** — maintains context across multi-file tasks
Download sizes: 4-bit (\~17 GB), 8-bit (\~32 GB), 16-bit (\~60 GB)
## Choose your runtime
Ollama is a command-line tool for running models locally. It has lower memory overhead than LM Studio and works well for scripting and server deployments.
**Best for:** Power users, terminal-comfortable developers, server setups.
Visit [ollama.com](https://ollama.com) and download the installer for your operating system (Windows, macOS, or Linux). Run the installer and follow the prompts.
Open your terminal and pull the recommended model:
```bash theme={null}
ollama pull qwen2.5-coder:32b
```
Browse all available models at [ollama.com/search](https://ollama.com/search). Other capable options:
```bash theme={null}
ollama pull mistral-small:latest
ollama pull codellama:34b-code
```
The first download may take several minutes depending on model size and your connection.
Ollama starts automatically after installation and runs in the background. To run the model interactively to verify it works:
```bash theme={null}
ollama run qwen2.5-coder:32b
```
Keep Ollama running whenever you use it with Cline. The default server port is `11434`.
1. Click the settings gear icon (⚙️) in the Cline panel.
2. Set **API Provider** to `Ollama`.
3. Set **Base URL** to `http://localhost:11434/` (the default — no change needed unless you're running Ollama on a different host).
4. Select your downloaded model from the **Model** dropdown.
5. Save settings.
By default, Ollama uses a small context window. For Cline, set it to the model's maximum:
Create a `Modelfile` to override the context size:
```text theme={null}
FROM qwen2.5-coder:32b
PARAMETER num_ctx 262144
```
Then build a named model from it:
```bash theme={null}
ollama create qwen-coder-256k -f Modelfile
```
Select `qwen-coder-256k` in Cline's model dropdown.
In Cline settings, go to **Features → Use Compact Prompt** and toggle it on. This reduces prompt size by \~90% — essential for getting acceptable speed with local models.
**Troubleshooting Ollama:**
* *Cline can't connect* — Verify Ollama is running (`ollama list` should return results). Check that the base URL matches.
* *Slow responses* — This is normal for local models. Enable compact prompts and consider using 4-bit quantization.
* *Model errors or broken tool use* — Verify `num_ctx` is set correctly. Models with insufficient context cause truncated outputs.
LM Studio is a desktop application with a graphical interface for downloading and running local models. It's the easiest way to get started.
**Best for:** Desktop users who prefer a GUI, quick experimentation.
Visit [lmstudio.ai](https://lmstudio.ai) and download the installer for your operating system. LM Studio requires AVX2 CPU support (most computers from 2013 onward).
1. Open LM Studio.
2. Navigate to the **Discover** tab.
3. Search for `Qwen3-Coder-30B-A3B-Instruct`.
4. Select the appropriate quantization for your RAM:
* **32 GB RAM** — 4-bit GGUF (\~17 GB download)
* **64 GB RAM** — 8-bit GGUF (\~32 GB download)
* **Mac (Apple Silicon)** — MLX format for better performance
5. Click **Download** and wait for it to complete.
1. Navigate to the **Developer** tab.
2. Load your downloaded model.
3. Toggle the server switch to **Running**.
The server starts at `http://localhost:1234`. Keep LM Studio open while using Cline.
After loading your model in the Developer tab, configure these critical settings:
| Setting | Value | Why |
| ------------------------- | ----------------- | -------------------------------------- |
| **Context Length** | `262144` | Set to model maximum for best results |
| **KV Cache Quantization** | Off (unchecked) | Critical — enables consistent tool use |
| **Flash Attention** | On (if available) | Improves inference speed |
1. Click the settings gear icon (⚙️) in the Cline panel.
2. Set **API Provider** to `LM Studio`.
3. The **Base URL** defaults to `http://localhost:1234` — leave this as-is.
4. Select your loaded model from the **Model** dropdown.
5. Save settings.
In Cline settings, go to **Features → Use Compact Prompt** and toggle it on. This reduces prompt size by \~90%, which is essential for local inference speed.
**Troubleshooting LM Studio:**
* *Cline can't connect* — Verify the server is running in the Developer tab and a model is loaded.
* *Inconsistent tool use or errors* — Make sure **KV Cache Quantization is off**. This is the most common cause of tool-use failures with LM Studio.
* *Out of memory* — Switch to a lower quantization (4-bit instead of 8-bit) or close other applications to free RAM.
## The OpenAI-compatible endpoint pattern
Both Ollama and LM Studio expose a local HTTP server that follows the OpenAI API format. This means you can also configure them as a custom **OpenAI-compatible** provider in Cline if the dedicated provider option doesn't appear:
| Runtime | Base URL | Model name format |
| --------- | --------------------------- | -------------------------- |
| Ollama | `http://localhost:11434/v1` | `qwen2.5-coder:32b` |
| LM Studio | `http://localhost:1234/v1` | Name shown in LM Studio UI |
To use this approach, select **OpenAI Compatible** as the provider in Cline, enter the base URL above, and enter any non-empty string as the API key (local servers don't authenticate).
## Performance expectations
| Metric | Typical range |
| -------------------------- | ----------------------------------- |
| Initial model load | 10–30 seconds |
| Token generation | 5–20 tokens/sec |
| Context processing (large) | Slower — scales with context size |
| Memory usage | Close to quantization download size |
### Tips for better performance
* Use 4-bit quantization to maximize speed; use 8-bit if you have 64 GB+ RAM and want better quality.
* Enable compact prompts in Cline — this is the single biggest improvement you can make.
* Store models on an NVMe SSD, not a spinning hard drive.
* Close other applications to free RAM for the model.
* Enable Flash Attention if your hardware supports it.
## When to use local vs. cloud models
| Local models are better when... | Cloud models are better when... |
| ----------------------------------------- | -------------------------------------- |
| Code must stay on your machine | Codebase exceeds 256K tokens |
| You work offline | You need consistent team performance |
| You want zero API cost | You need the latest model capabilities |
| You're experimenting with many iterations | Speed is critical |
## Next steps
Compare all models and find the right one for your workflow.
Understand context limits and how to stay within them with large codebases.