perf: halve Redis round trips on model resolution and socket pools (#27225)
When WEBSOCKET_MANAGER=redis, app.state.MODELS and the socket session/
usage pools are Redis-backed dicts, so every membership test and
getitem is a network round trip:
- generate_chat_completion checked `model_id not in models` (HEXISTS)
and then read `models[model_id]` (HGET) on every chat completion.
A single .get() now serves both, with the same not-found error.
- The direct-connection branch spread the pool with `{**MODELS, ...}`,
which iterates keys() then fetches each value — HKEYS plus one HGET
per model. dict(MODELS.items()) issues a single HGETALL instead.
- get_user_ids_from_room called SESSION_POOL.get(sid) twice per
session (once to filter, once for the value); the usage handler
checked membership then fetched the same key. Both now do one
lookup.
In non-Redis mode these are plain dicts and behavior is identical.
Claude-Session: https://claude.ai/code/session_01MHg5zs1VBjvRWQ54qHpfYD
Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -286,14 +286,14 @@ def get_session_ids_from_room(room):
|
||||
def get_user_ids_from_room(room):
|
||||
active_session_ids = get_session_ids_from_room(room)
|
||||
|
||||
# Single pool lookup per session (each .get is a Redis round trip
|
||||
# when the session pool is Redis-backed).
|
||||
active_user_ids = list(
|
||||
set(
|
||||
[
|
||||
SESSION_POOL.get(session_id)['id']
|
||||
for session_id in active_session_ids
|
||||
if SESSION_POOL.get(session_id) is not None
|
||||
]
|
||||
)
|
||||
{
|
||||
entry['id']
|
||||
for entry in (SESSION_POOL.get(session_id) for session_id in active_session_ids)
|
||||
if entry is not None
|
||||
}
|
||||
)
|
||||
return active_user_ids
|
||||
|
||||
@@ -357,7 +357,7 @@ async def usage(sid, data):
|
||||
|
||||
# Store the new usage data and task
|
||||
USAGE_POOL[model_id] = {
|
||||
**(USAGE_POOL[model_id] if model_id in USAGE_POOL else {}),
|
||||
**(USAGE_POOL.get(model_id) or {}),
|
||||
sid: {'updated_at': current_time},
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user