5 min read

Ollama lets you run large language models (LLMs) like Llama, Mistral, and Gemma on your own hardware. No API keys, no per-token costs, no data leaving your server. You need a VPS with enough RAM and CPU to run the model you want.

Hardware Requirements

LLMs are memory-hungry. The model needs to fit entirely in RAM (or VRAM if using a GPU). Approximate requirements for CPU-only inference with the default 4-bit models:

ModelParametersRAM RequiredSpeed (CPU)Use Case
Llama 3.2 3B3B4 GBFastLightweight tasks, classification
Gemma 3 4B4B6 GBFastSummarization, simple chat
Llama 3.1 8B8B8 GBUsableGeneral chat, writing, analysis
Qwen3 8B8B8 GBUsableGeneral purpose, coding help
Llama 3.3 70B70B48 GBVery slowHigh quality, impractical on CPU

Recommended GoZen VPS specs:

  • Small models (2B-3B): 4 vCPU / 6 GB RAM is sufficient
  • Medium models (7B-8B): 8+ vCPU / 16 GB RAM recommended
  • Large models (70B+): Not practical on CPU-only VPS. Need a GPU server

The AMD EPYC VPS (Business Cloud) plans give you dedicated AMD EPYC cores, which keeps token speed steady for 7B-8B models. See AMD EPYC VPS for the plan details.

Installing Ollama

SSH into your VPS:

  curl -fsSL https://ollama.com/install.sh | sh

Verify:

  ollama --version

Ollama runs as a systemd service automatically:

  sudo systemctl status ollama

Running Your First Model 

  # Download and run Llama 3.1 8B
ollama run llama3.1

# You'll see a chat prompt:
# >>> Send a message (/? for help)

Type a prompt and press Enter. The first run downloads the model (4-5 GB for 8B). Subsequent runs start instantly.

  # Mistral 7B - fast, good quality
ollama run mistral

# Llama 3.2 3B - very lightweight
ollama run llama3.2:3b

# Qwen2.5 Coder - tuned for programming
ollama run qwen2.5-coder:7b

# Gemma 3 4B - small general-purpose model
ollama run gemma3:4b

List downloaded models:

  ollama list

Remove a model:

  ollama rm llama3.1

Using the API

Ollama exposes a REST API on port 11434. You can call it from any application:

  # Generate a completion
curl http://localhost:11434/api/generate -d '{
  "model": "llama3.1",
  "prompt": "Explain DNS in two sentences.",
  "stream": false
}'

# Chat format
curl http://localhost:11434/api/chat -d '{
  "model": "llama3.1",
  "messages": [
    {"role": "user", "content": "What is DKIM?"}
  ],
  "stream": false
}'

From Python 

  import requests

response = requests.post("http://localhost:11434/api/generate", json={
    "model": "llama3.1",
    "prompt": "Write a SQL query to find duplicate emails",
    "stream": False
})

print(response.json()["response"])

From JavaScript/Node.js 

  const response = await fetch("http://localhost:11434/api/generate", {
  method: "POST",
  body: JSON.stringify({
    model: "llama3.1",
    prompt: "Explain caching in one paragraph",
    stream: false,
  }),
});

const data = await response.json();
console.log(data.response);

Adding a Web Interface

Open WebUI gives you a ChatGPT-like interface for Ollama:

  docker run -d \
  --name open-webui \
  --network host \
  -e HOST=127.0.0.1 \
  -e PORT=3000 \
  -v open-webui:/app/backend/data \
  -e OLLAMA_BASE_URL=http://127.0.0.1:11434 \
  --restart always \
  ghcr.io/open-webui/open-webui:main

With HOST=127.0.0.1, Open WebUI listens on 127.0.0.1:3000 only, so it is not reachable from the internet. Put it behind HTTPS with a reverse proxy pointing at http://127.0.0.1:3000, or test it first through an SSH tunnel from your computer:

  ssh -L 3000:127.0.0.1:3000 your-user@your-server-ip
# then open http://localhost:3000 in your browser

Create your account as soon as the page loads. The first account that signs up becomes the Open WebUI admin.

Exposing Ollama Securely

By default, Ollama only listens on localhost. If you need to access it from other machines:

  1. Edit the service file:
  sudo systemctl edit ollama

Add:

  [Service]
Environment="OLLAMA_HOST=0.0.0.0"
  1. Restart:
  sudo systemctl restart ollama
  
  # Only allow access from specific IPs
sudo ufw allow from YOUR.TRUSTED.IP to any port 11434

Performance Tuning

Adjusting Thread Count

By default, Ollama uses the available CPU cores. On a shared-vCPU VPS you may want to limit it with the num_thread model parameter:

  # Inside an interactive session
ollama run llama3.1
>>> /set parameter num_thread 4

For API calls, pass it in options:

  curl http://localhost:11434/api/generate -d '{
  "model": "llama3.1",
  "prompt": "Hello",
  "options": {"num_thread": 4}
}'

To make it permanent for a model, add PARAMETER num_thread 4 to a Modelfile and build it with ollama create.

Using Quantized Models

Quantized models use less RAM and run faster at a small quality cost:

  # Pull the Q4 quantized version (smaller, faster)
ollama run llama3.1:8b-instruct-q4_0

Monitoring Resource Usage 

  # Watch CPU and RAM in real-time
htop

# Check Ollama-specific usage
ollama ps

Troubleshooting

ProblemFix
“Out of memory” during model loadModel is too large for your RAM. Try a smaller model or quantized version
Very slow inferenceCPU-only is inherently slow for large models. Use a smaller model (2B-3B) for acceptable speed
API connection refusedCheck the service with sudo systemctl status ollama and journalctl -u ollama -n 50. Restart with sudo systemctl restart ollama
Port 11434 not reachable remotelyOllama defaults to localhost only. Set OLLAMA_HOST=0.0.0.0 if you need remote access
Model download failsCheck disk space. 7B models need 4-5 GB. df -h to verify

What to Do Next

Last updated 21 Oct 2026, 00:00 +0300. history

Was this page helpful?