Skip to main content
Running FERAL offline means every inference stays on your machine. No API keys to rotate, no data leaves your network, and you can unplug the Ethernet cable after setup.

​
Prerequisites

​
Combinations

Mix and match to fit your hardware budget.
RAM figures are approximate peak usage. When running LLM + STT + TTS simultaneously, add the individual figures together.
The voice engines above are not all equally verified. macOS say is the only local TTS with a real synthesis test in this repository; Piper’s synthesis path has never produced audio here because the macOS arm64 wheels abort on a hardcoded CI espeak path; and neither local STT engine has ever run a real transcription in this repository (only their refusal paths are tested). The per-engine detail is in Local Voice. Read it before you commit to a stack.
The audio side of a local voice stack is documented in full at Local Voice: engine tiers, what downloads when and how big it is, the measured endpointing improvement, and why local mode fails loudly rather than falling back to a cloud provider.
“Fully local voice” means the audio, not the conversation. A local voice stack transcribes on your machine and then sends the text to whatever LLM you configured. Pair it with Ollama (below) to keep the whole turn local.

​
Setup

​
Troubleshooting

​
”Connection refused” from Ollama

Make sure ollama serve is running. By default it listens on http://localhost:11434. Check with:

​
Slow first response

The first inference after pulling a model is slow because Ollama loads weights into memory. Subsequent calls are fast. You can pre-warm with:

​
Out of memory

If your machine runs out of RAM:
  1. Switch to a smaller model (qwen2:7b uses ~1 GB less than llama3:8b).
  2. Close other heavy applications.
  3. On Linux, increase swap: sudo fallocate -l 8G /swapfile && sudo mkswap /swapfile && sudo swapon /swapfile.

​
STT not detecting speech

Ensure your microphone is accessible and the correct device is selected:
Set the device index if needed:

​
Performance Tips

Choose models based on your hardware:
  • 8 GB RAM (no GPU): qwen2:7b + faster-whisper base + Piper. The LLM dominates the turn; the endpointing and TTS costs are documented in Local Voice.
  • 16 GB RAM (no GPU): llama3:8b + faster-whisper small + Piper.
  • Apple Silicon (M1+): Ollama uses Metal acceleration automatically. llama3:8b runs at ~30 tokens/s on M2.
  • NVIDIA GPU (8 GB+ VRAM): Ollama detects CUDA automatically. Expect 40–80 tokens/s depending on model and GPU.
General tips:
  • Keep Ollama running between sessions to avoid cold-start latency.
  • Use the base model unless you need higher accuracy; small is meaningfully slower for a modest accuracy gain.
  • On macOS, prefer whispercpp over faster-whisper. CTranslate2 has no Metal backend, so faster-whisper runs on CPU there while every log line claims local acceleration.
  • Piper TTS is CPU-only. Its throughput has not been measured in this repository, because its synthesis path has never run here; see Local Voice.