* fix: prevent worker death during document upload by using run_coroutine_threadsafe
Replace asyncio.run() with asyncio.run_coroutine_threadsafe() in
save_docs_to_vector_db() to prevent uvicorn worker health check failures.
The issue: asyncio.run() creates a new event loop and blocks the thread
completely, preventing the worker from responding to health checks during
long-running embedding operations (>5 seconds default timeout).
The fix: Schedule the async embedding work on the main event loop using
run_coroutine_threadsafe(). This keeps the main loop responsive to health
check pings while the sync caller waits for the result.
Changes:
- main.py: Store main event loop reference in app.state.main_loop at startup
- retrieval.py: Use run_coroutine_threadsafe() instead of asyncio.run()
https://claude.ai/code/session_01UQSYvSTkXb57sFb7M85Kcw
* add env var
---------
Co-authored-by: Claude <noreply@anthropic.com>
fix: reduce TTFT by caching model lookups in chat completion
Skip expensive get_all_models() calls when models are already cached
in app.state. This significantly reduces Time To First Token (TTFT)
for chat completions and embeddings requests.
Previously, every request called get_all_models() which fetches model
lists from all configured backends. Now we check the cache first and
only call get_all_models() on cache miss.
Affected endpoints:
- openai: generate_chat_completion, embeddings
- ollama: embed, embeddings
Fixes#20069
Co-authored-by: Michael <42099345+mickeytheseal@users.noreply.github.com>
Each access to request.app.state.config.<KEY> triggers a synchronous
Redis GET. In get_all_models_responses() and get_merged_models(), the
config keys OPENAI_API_BASE_URLS, OPENAI_API_KEYS, and
OPENAI_API_CONFIGS were read on every loop iteration — resulting
in some cases in 200-300 Redis round-trips for OPENAI_API_BASE_URLS alone.
Read each config value once into a local variable at the start of the
function and reuse it throughout.
Fix `AttributeError` in `model_response_handler` when processing channel messages with `null` data field. The function iterates over thread messages to build conversation history, but some messages may have `data=None` causing a crash when accessing `thread_message.data.get()`. Added null check using `(thread_message.data or {}).get("files", [])` to safely handle messages without data.
- Add chat_message table for message-level analytics with usage JSON field
- Add migration to backfill from existing chats
- Add /analytics endpoints: summary, models, users, daily
- Support hourly/daily granularity for time-series data
- Fill missing days/hours in date range
Timothy Jaeryang Baek