Llamafiles are open-source AI models packaged as a single file, that runs on your laptop, offline.

Built on top of llama.cpp, this enables offline, cross-platform inference with minimal setup—no Python, CUDA, or complex dependencies required. A Llamafile bundles:

  • The model weights (quantized for efficiency).
  • The inference engine (llama.cpp).
  • A built-in server for serving the model via a REST API.

Using Apertus Models with Llamafile

Pre-built Llamafiles for the Apertus 8B model, and soon for smaller versions, are available here:

Example (Linux/macOS):

# Download a Llamafile
curl -L https://huggingface.co/mozilla-ai/llamafile_0.10/resolve/main/Apertus-8B-Instruct-2509.llamafile -o apertus.llamafile
chmod +x apertus.llamafile

Execute the Llamafile directly:

./apertus.llamafile

This starts a local server (default: http://localhost:8080). Interact via:

  • CLI: curl http://localhost:8080/completions -d '{"prompt": "Your text here"}'
  • API: Send POST requests to the /completions endpoint (OpenAI-compatible format).

Custom Builds (Advanced)

To create a Llamafile from an Apertus model in GGUF format:

# Install llamafile (requires Docker or Podman)
curl -L https://github.com/Mozilla-Ocho/llamafile/releases/download/0.8.13/llamafile -o llamafile
chmod +x llamafile

# Convert a GGUF model to Llamafile
./llamafile --model /path/to/apertus-8b.gguf --out apertus.llamafile

See our ollama page for some recommended GGUF builds.

Resources