perf: halve Redis round trips on model resolution and socket pools (#27225)

When WEBSOCKET_MANAGER=redis, app.state.MODELS and the socket session/
usage pools are Redis-backed dicts, so every membership test and
getitem is a network round trip:

- generate_chat_completion checked `model_id not in models` (HEXISTS)
  and then read `models[model_id]` (HGET) on every chat completion.
  A single .get() now serves both, with the same not-found error.
- The direct-connection branch spread the pool with `{**MODELS, ...}`,
  which iterates keys() then fetches each value — HKEYS plus one HGET
  per model. dict(MODELS.items()) issues a single HGETALL instead.
- get_user_ids_from_room called SESSION_POOL.get(sid) twice per
  session (once to filter, once for the value); the usage handler
  checked membership then fetched the same key. Both now do one
  lookup.

In non-Redis mode these are plain dicts and behavior is identical.


Claude-Session: https://claude.ai/code/session_01MHg5zs1VBjvRWQ54qHpfYD

Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
Classic298
2026-07-23 21:32:01 -04:00
committed by GitHub
co-authored by Claude
parent 429f2df50c
commit 656a848043
2 changed files with 15 additions and 12 deletions
+8 -8
View File
@@ -286,14 +286,14 @@ def get_session_ids_from_room(room):
def get_user_ids_from_room(room):
active_session_ids = get_session_ids_from_room(room)
# Single pool lookup per session (each .get is a Redis round trip
# when the session pool is Redis-backed).
active_user_ids = list(
set(
[
SESSION_POOL.get(session_id)['id']
for session_id in active_session_ids
if SESSION_POOL.get(session_id) is not None
]
)
{
entry['id']
for entry in (SESSION_POOL.get(session_id) for session_id in active_session_ids)
if entry is not None
}
)
return active_user_ids
@@ -357,7 +357,7 @@ async def usage(sid, data):
# Store the new usage data and task
USAGE_POOL[model_id] = {
**(USAGE_POOL[model_id] if model_id in USAGE_POOL else {}),
**(USAGE_POOL.get(model_id) or {}),
sid: {'updated_at': current_time},
}