Run a private AI model on your own server with Ollama

Install Ollama

Ollama downloads and runs open models with a single command. It installs as a background service.

curl -fsSL https://ollama.com/install.sh | sh
ollama --version

Download and try a model

Start with a small model to test your hardware. This command downloads the model the first time and opens a chat. Type /bye to leave.

ollama run llama3.2

Check each model's licence before business use, and pick one that fits your RAM. Larger models are usually better, but need much more memory.

Call it from your own tools

Ollama provides a local API on port 11434. Anything on the server, including scripts and automations, can use it, and your text stays on your machine.

curl http://localhost:11434/api/generate -d '{
  "model": "llama3.2",
  "prompt": "Summarise in one sentence: The customer needs a quote for five PCs by Friday.",
  "stream": false
}'

Connect it to n8n

In n8n, connect an Ollama Chat Model to any AI node. If both run on the same server and n8n is in Docker, n8n cannot reach the host through localhost. Add the lines below to the n8n service in docker-compose.yml, then use http://host.docker.internal:11434 as the base URL.

    extra_hosts:
      - "host.docker.internal:host-gateway"

Ollama listens only on the local machine by default. To reach it from n8n in Docker you may need to set OLLAMA_HOST=0.0.0.0 in its service, and if you do, use the firewall so only your own server can connect to port 11434.

Decide when local AI is the right choice

Local models keep your data in-house and have no per-message fee, but you provide the hardware and maintenance, and the largest cloud models are often more capable. Many businesses use a local model for sensitive documents and a cloud model for everything else.