perf: write a chat's messages in one transaction instead of one per message (#28806)

Saving a chat rewrote its message rows one at a time. Each message took its own session out of the pool and committed on its own, and the save endpoint hands over the entire merged history rather than only what changed, so a two hundred message chat cost two hundred sessions and two hundred commits on every save.

The messages now go through a single select and a single commit. The field mapping for the insert and the update branch moved into two small helpers, so the batch and the single-message path cannot drift apart.

Measured on a two hundred message chat with one message edited: 201 queries and 200 transactions before, 2 queries and 1 transaction after, ~149 ms against ~6 ms. Re-saving an unchanged history now costs one select and no writes at all.

One behaviour change worth stating: a message the database cannot store used to be skipped on its own, and now costs the rest of that same save. This table is a rebuildable fast path, so the reader falls back to the history on the chat row and re-triggers the backfill, and the next save reconciles everything still present. A per-message retry was tried and dropped, because a commit that lands but still raises would re-apply the usage merge and double the recorded token counts.
This commit is contained in:
Classic298 authored and GitHub committed 2026-08-19 12:46:49 -05:00
1 parent bcb50fe7b0
commit 81fe43f210
2 files changed
+104 -77

No files matched your search

+9 -12
View File
@@ -959,18 +959,15 @@ class ChatTable:
"""Write messages to the ``chat_message`` table so future lookups
use the fast path. Errors are logged but never raised.
"""
for message_id, message in messages.items():
if not isinstance(message, dict) or not message.get('role'):
continue
try:
await ChatMessages.upsert_message(
message_id=message_id,
chat_id=chat_id,
user_id=user_id,
data=message,
)
except Exception as e:
log.warning('Backfill failed for message %s in chat %s: %s', message_id, chat_id, e)
writable = {
message_id: message
for message_id, message in messages.items()
if isinstance(message, dict) and message.get('role')
}
try:
await ChatMessages.upsert_messages(chat_id, user_id, writable)
except Exception as e:
log.warning('Backfill failed for chat %s: %s', chat_id, e)
async def reconcile_messages_by_chat_id(self, chat_id: str, user_id: str, messages: dict[str, dict]) -> None:
"""Sync ``chat_message`` rows with the committed JSON blob.