YGG-OLLA hosts an Olla LLM proxy/load balancer built on top of yggdrasil-base which allows hosting an Olla container on the Yggdrasil mesh network.

Olla is a high-performance proxy and load balancer for LLM infrastructure with intelligent routing, automatic failover, and unified model discovery across local and remote inference backends.

SETUP

1. Clone the repo and initialize submodules:

   git submodule update --init --recursive

2. Build the container:

   ./build.sh

3. Install quadlet files and create storage dirs:

   ./install.sh

4. Place your config at:

   ~/.storage/containers/ygg-olla/olla/config/config.yaml

5. Start the service:

   systemctl --user enable --now ygg-olla

CONFIGURATION

See https://thushan.github.io/olla/ for Olla configuration options.

AI DISCLAIMER

This project is partially or fully developed with the assistance of AI tools. Review all generated code, configurations, and documentation before deploying to production.