Add relaxed-MTP llama-server as a new connection, surface generation speed

Wires the custom relaxed-acceptance llama-server (see ../mtp-relaxed-decoding) into the stack as a new llama-mtp service, registered as an additional OpenAI-compatible connection alongside Ollama. ollama-auth now proxies to llama-mtp instead of the now-empty Ollama, so LAN clients (e.g. Home Assistant's voice pipeline) keep working against the same URL/token with no reconfiguration. Also merges llama.cpp's `timings` extension (dropped by the generic OpenAI response schema) into message.usage and surfaces it as a labeled tok/s badge, so relaxed-MTP responses get the same visible generation-speed info Ollama responses already had. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-04 11:33:12 -04:00
parent 7eff527d16
commit 6f583deff9
6 changed files with 113 additions and 13 deletions
@@ -66,7 +66,12 @@ services:
    environment:
      - 'OLLAMA_BASE_URL=http://ollama:11434'
      - 'WEBUI_SECRET_KEY='
-      - 'DEFAULT_MODELS=gemma-4-e2b-mtp'
+      # Only takes effect on a fresh DB -- once ui.default_models exists in
+      # the config table, that value wins (see open-webui's PersistentConfig
+      # seed_defaults behavior). Currently gemma-4-e2b-mtp was removed from
+      # Ollama in favor of the relaxed-MTP llama-mtp service; the live value
+      # is 'gemma-4-e2b,/models/gemma-4-E2B-it-Q4_K_M.gguf'.
+      - 'DEFAULT_MODELS=gemma-4-e2b,/models/gemma-4-E2B-it-Q4_K_M.gguf'
      - 'ENABLE_OAUTH_SIGNUP=true'
      # Links Authentik login to the existing local account with the same
      # email instead of rejecting it as a collision -- safe here since this