Skip to main content
Running FERAL offline means every inference stays on your machine. No API keys to rotate, no data leaves your network, and you can unplug the Ethernet cable after setup.

​
Prerequisites

​
Tested Combinations

These combos are regularly tested by the team. Mix and match to fit your hardware budget.
RAM figures are approximate peak usage. When running LLM + STT + TTS simultaneously, add the individual figures together.

​
Setup

​
Troubleshooting

​
”Connection refused” from Ollama

Make sure ollama serve is running. By default it listens on http://localhost:11434. Check with:

​
Slow first response

The first inference after pulling a model is slow because Ollama loads weights into memory. Subsequent calls are fast. You can pre-warm with:

​
Out of memory

If your machine runs out of RAM:
  1. Switch to a smaller model (qwen2:7b uses ~1 GB less than llama3:8b).
  2. Close other heavy applications.
  3. On Linux, increase swap: sudo fallocate -l 8G /swapfile && sudo mkswap /swapfile && sudo swapon /swapfile.

​
STT not detecting speech

Ensure your microphone is accessible and the correct device is selected:
Set the device index if needed:

​
Performance Tips

Choose models based on your hardware:
  • 8 GB RAM (no GPU): qwen2:7b + faster-whisper base + Piper — expect 2–4 s response times.
  • 16 GB RAM (no GPU): llama3:8b + faster-whisper small + Piper — expect 1–3 s response times.
  • Apple Silicon (M1+): Ollama uses Metal acceleration automatically. llama3:8b runs at ~30 tokens/s on M2.
  • NVIDIA GPU (8 GB+ VRAM): Ollama detects CUDA automatically. Expect 40–80 tokens/s depending on model and GPU.
General tips:
  • Keep Ollama running between sessions to avoid cold-start latency.
  • Use faster-whisper base unless you need higher accuracy — small is 2× slower for a modest accuracy gain.
  • Piper TTS is CPU-only and extremely fast; it won’t bottleneck your setup.
  • If you run FERAL on a headless server, disable TTS with FERAL_TTS_PROVIDER=none to save resources.