perf: linear content accumulation in the streaming response handler (#27231)

The streaming handler rebuilt the full accumulated response with
`content = f'{content}{value}'` on every content delta — a complete
string copy per chunk, making accumulation O(n^2) over the response
length. Use in-place `content += value` for the (universal) str case,
which CPython extends in place, keeping accumulation O(n); the
f-string fallback is preserved for non-str values so coercion
behavior is unchanged.

In the ENABLE_REALTIME_CHAT_SAVE branch, full_output() — which
concatenates the entire accumulated output — was called twice per
chunk (once for the DB upsert, once for the emitted delta). Compute
it once and reuse.


Claude-Session: https://claude.ai/code/session_01MHg5zs1VBjvRWQ54qHpfYD

Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
Classic298
2026-07-23 12:24:29 -04:00
committed by GitHub
co-authored by Claude
parent 8becf9443e
commit 62491debfa
+9 -3
View File
@@ -4490,7 +4490,12 @@ async def streaming_chat_response_handler(response, ctx):
user,
)
content = f'{content}{value}'
if isinstance(value, str):
# In-place append — avoids copying the full
# accumulated response on every chunk.
content += value
else:
content = f'{content}{value}'
# Check if we're inside a tag-based block
# (reasoning, code_interpreter, or solution).
@@ -4597,16 +4602,17 @@ async def streaming_chat_response_handler(response, ctx):
if ENABLE_REALTIME_CHAT_SAVE and not metadata.get('chat_id', '').startswith(
'channel:'
):
current_output = full_output()
# Save message in the database
await Chats.upsert_message_to_chat_by_id_and_message_id(
metadata['chat_id'],
metadata['message_id'],
{
'output': full_output(),
'output': current_output,
},
)
data = {
'output': full_output(),
'output': current_output,
}
delta_type = 'content'
else: