Add relaxed-MTP llama-server as a new connection, surface generation speed

Wires the custom relaxed-acceptance llama-server (see ../mtp-relaxed-decoding) into the stack as a new llama-mtp service, registered as an additional OpenAI-compatible connection alongside Ollama. ollama-auth now proxies to llama-mtp instead of the now-empty Ollama, so LAN clients (e.g. Home Assistant's voice pipeline) keep working against the same URL/token with no reconfiguration. Also merges llama.cpp's `timings` extension (dropped by the generic OpenAI response schema) into message.usage and surfaces it as a labeled tok/s badge, so relaxed-MTP responses get the same visible generation-speed info Ollama responses already had. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-04 11:33:12 -04:00
parent 7eff527d16
commit 6f583deff9
6 changed files with 113 additions and 13 deletions
@@ -13,18 +13,24 @@ server {
            return 401;
        }

-        # Docker's embedded DNS (127.0.0.11) can reassign the "ollama"
-        # hostname to a new container IP on restart. proxy_pass to a
-        # literal upstream caches that IP for the container's lifetime;
-        # routing through a variable forces nginx to re-resolve via this
-        # resolver on each request instead, so a restarted ollama doesn't
-        # need a matching ollama-auth restart to be reachable again.
+        # Ollama itself now serves no models -- this proxy targets
+        # llama-mtp (the relaxed-MTP server) instead, so LAN clients that
+        # already trust this URL/token (e.g. Home Assistant's voice
+        # pipeline) don't need reconfiguring. Docker's embedded DNS
+        # (127.0.0.11) can reassign a container hostname to a new IP on
+        # restart; proxy_pass to a literal upstream caches that IP for the
+        # container's lifetime, so route through a variable to force
+        # nginx to re-resolve via this resolver on each request instead.
        resolver 127.0.0.11 valid=10s;
-        set $ollama_upstream ollama;
-        proxy_pass http://$ollama_upstream:11434;
+        set $mtp_upstream llama-mtp;
+        proxy_pass http://$mtp_upstream:8030;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        proxy_set_header Host $host;
+        # Client presents OLLAMA_AUTH_TOKEN (validated above); llama-server
+        # behind this proxy expects its own MTP_AUTH_TOKEN instead, so swap
+        # it here rather than requiring every client to be reconfigured.
+        proxy_set_header Authorization "Bearer ${MTP_AUTH_TOKEN}";
        proxy_buffering off;
        proxy_read_timeout 600s;
        proxy_send_timeout 600s;