Commit Graph
489 Commits
Author SHA1 Message Date
Classic298 310ae91302 perf: make tools defer_content real and batch tool and knowledge access filters (#27387)
Tools.get_tools(defer_content=True) contained the literal dead statement "stmt = stmt": the deferral was a no-op, so every tools listing loaded the full Python source of every tool (five caller sites pass defer_content=True expecting the optimization: the tools list endpoints and the user and group permission overviews). The listing now selects every column except content, and ToolModel.content becomes optional to represent deferred rows; router projections are content-less response models, so nothing downstream reads the source on these paths.

On top of that, get_tools_by_user_id issued one grant query per non-owned tool and Knowledges.get_knowledge_bases_by_user_id did the same per knowledge base (the latter also sits inside per-file access checks). Both now resolve grants for all non-owned rows in a single get_accessible_resource_ids call, the same batch helper the model listing already uses.

Benchmark (real SQLite DB):

| metric | before | after |
| --- | --- | --- |
| tools listing, 33 tools x ~200 KB source | 5.13 ms | 3.18 ms |
| grant queries per accessible-tools call, N non-owned tools | N | 1 |
| grant queries per accessible-KBs call, N non-owned KBs | N | 1 |

The listing row scales with source size; on Postgres the deferral additionally avoids shipping every tool's source over the wire per listing, and each removed grant query was a real round trip.

Functionally verified: deferred listings match full listings field for field with content None, grants included and router projections working; access filtering returns exactly owned plus granted tools and knowledge bases and nothing for strangers; full (non-deferred) reads still carry the source.
2026-07-23 17:51:07 -05:00
Classic298 5b518cbe43 perf: list knowledge base file metadata without loading extracted file contents (#27386)
Knowledges.get_file_metadatas_by_id fetched full File rows, whose data column carries the entire extracted text of each document, validated each into a FileModel and then threw everything except id, hash, meta and the timestamps away. The function backs every knowledge base detail view and runs again after every file add or remove (eight call sites in the knowledge router), so rendering a filename list for a 50-file knowledge base parsed tens of megabytes of JSON per request.

The listing now selects exactly the five columns the response needs, joined through KnowledgeFile, mirroring the column-only helper that already existed in the files model for id-based lookups.

Benchmark (real SQLite DB, 50 files with ~200 KB extracted text each):

| metric | before | after |
| --- | --- | --- |
| KB file metadata listing | 14.2 ms | 1.20 ms |

The gap widens linearly with file size and count since extracted contents no longer get read, parsed or validated at all.

Functionally verified: output matches the old implementation field for field on all 50 files and an unknown knowledge base still returns an empty list.
2026-07-23 17:50:55 -05:00
Classic298 d67bc4ffcd perf: batch the file access check queries (#27383)
has_access_to_file runs for every non-owner file GET, per RAG file check and per shared-chat or model-attached file. Its final step called Models.get_models_by_user_id, which issued one grant query per non-owned workspace model, so a single file check on an instance with M workspace models cost M grant queries plus a group query, with the deny path always paying full price. Its collection_name step listed every knowledge base the user can access (itself one grant query per knowledge base) just to scan the list for one id. And get_accessible_folder_files repeated the whole pipeline per folder entry, refetching the caller's group memberships every time.

Three changes, all using parameters and helpers that already exist:
- Models.get_models_by_user_id resolves grants for all non-owned models in one get_accessible_resource_ids call and accepts prefetched user_group_ids.
- The collection_name check fetches the one referenced knowledge base and performs a single owner-or-grant check with the already-resolved group ids, preserving the write-requires-owner guard exactly (including its short-circuit before any grant query).
- get_accessible_folder_files resolves group ids once and threads them through every per-entry check.

Benchmark:

| metric | before | after |
| --- | --- | --- |
| filter loop CPU, 300 workspace models (queries stubbed) | 47 us | 19 us |
| grant queries per file-access check, M workspace models | M | 1 |
| group membership queries per folder listing, F files | F | 1 |

The stubbed CPU row understates the win: each removed query in the other two rows was a real database round trip.

Functionally verified with stubbed accessors: owned plus granted models are returned with owned ids excluded from the batch query; model-attached file access resolves through the batched path; the collection_name path does one KB fetch and one grant check with no full listing; a missing KB falls through; write access via a KB still requires the KB owner to own the file and short-circuits before the grant query; folder listings fetch groups exactly once.
2026-07-23 17:50:08 -05:00
Timothy Jaeryang Baek c8f2e09fdc refac 2026-07-23 14:29:27 -04:00
Timothy Jaeryang Baek 25faa19941 refac 2026-07-23 14:27:55 -04:00
Timothy Jaeryang Baek ca11bd90a7 chore: format 2026-07-23 13:41:16 -04:00
Timothy Jaeryang Baek f9107edeeb refac 2026-07-23 12:48:14 -04:00
Timothy Jaeryang Baek bb12b1a18b refac 2026-07-23 04:16:14 -04:00
Timothy Jaeryang Baek cc9a44569e refac 2026-07-23 04:01:30 -04:00
Timothy Jaeryang Baek 48625e657f refac 2026-07-23 03:54:19 -04:00
Timothy Jaeryang Baek cf887b68ea refac 2026-07-23 02:54:56 -04:00
Timothy Jaeryang Baek 409fb39717 refac 2026-07-21 13:53:30 -04:00
Timothy Jaeryang Baek 49e57f4e7e chore: format 2026-07-20 22:11:42 -04:00
Timothy Jaeryang Baek af9a315ac3 refac 2026-07-20 01:33:47 -04:00
Timothy Jaeryang Baek 1a32d92d08 refac 2026-07-17 06:50:16 -04:00
Timothy Jaeryang Baek e5e2cd7876 refac 2026-07-17 04:48:21 -04:00
Timothy Jaeryang Baek 56f2cb5302 refac 2026-07-17 04:11:11 -04:00
Timothy Jaeryang Baek 743b9fd3ce refac 2026-07-16 21:57:43 -04:00
Timothy Jaeryang Baek 4d27bfff92 refac 2026-07-16 01:42:24 -04:00
Timothy Jaeryang Baek 185bca8552 refac 2026-07-15 22:34:52 -04:00
Timothy Jaeryang Baek 423cafd4e7 refac 2026-07-15 21:43:47 -04:00
Timothy Jaeryang Baek 7a9928ef17 refac 2026-07-14 23:08:41 -04:00
Timothy Jaeryang Baek a9617ca218 refac 2026-07-14 01:15:29 -04:00
Timothy Jaeryang Baek f1ded9409a refac 2026-07-14 01:13:40 -04:00
Timothy Jaeryang Baek 7088d245bb refac 2026-07-14 00:10:28 -04:00
Timothy Jaeryang Baek 73c0be8389 refac 2026-07-01 03:01:46 -05:00
Timothy Jaeryang BaekandSyed Osama Ali Shah ab22fe64bd refac
Co-Authored-By: Syed Osama Ali Shah <86572800+osamaali313@users.noreply.github.com>
2026-07-01 02:20:50 -05:00
Timothy Jaeryang Baek 403392b41b chore: format 2026-06-29 13:35:39 -05:00
Timothy Jaeryang Baek 22a44e67a8 refac 2026-06-29 13:15:29 -05:00
Timothy Jaeryang Baek 517cd8d102 refac 2026-06-29 13:03:14 -05:00
Timothy Jaeryang Baek b4073f6378 refac 2026-06-29 12:29:10 -05:00
Timothy Jaeryang Baek a146e17bdc refac 2026-06-29 11:33:32 -05:00
Timothy Jaeryang Baek 81e245548d refac 2026-06-29 11:00:53 -05:00
Timothy Jaeryang BaekandClassic298 eb53281c9a refac
Co-Authored-By: Classic298 <27028174+Classic298@users.noreply.github.com>
2026-06-29 06:08:31 -05:00
Timothy Jaeryang Baek b0487dd6dd refac 2026-06-29 05:49:54 -05:00
Timothy Jaeryang Baek 8927c9bb3d refac 2026-06-29 05:14:34 -05:00
Timothy Jaeryang Baek 67c9de8efd refac 2026-06-29 04:42:59 -05:00
Timothy Jaeryang Baek ab84bbf08c refac 2026-06-29 04:19:33 -05:00
Classic298andClaude Opus 4.8 a66477b710 fix: bind channel thread parent/reply to the URL channel (#25766)
GET /api/v1/channels/{id}/messages/{message_id}/thread authorized only the URL channel, but get_messages_by_parent_id() appended the thread parent (loaded by id) without checking it belonged to that channel, so a caller could read a message from a channel they cannot access by passing its id as the thread root. Require the parent to be in the requested channel before returning it, and reject a posted parent_id/reply_to_id that does not belong to the channel.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 03:46:50 -05:00
Timothy Jaeryang Baek 33cd199e6d refac 2026-06-29 03:16:59 -05:00
Timothy Jaeryang Baek 2bdd2ab94e refac 2026-06-29 02:43:14 -05:00
G30 e69ce6e1c6 perf(channels): batch N+1 queries for reactions and thread replies (#25831)
Replace per-message database queries with batch IN-clause queries in
channel message handlers. This eliminates the N+1 query pattern that
caused ~102 queries per channel page load (50 messages × 2 queries each).

Changes:
- Add get_reactions_by_message_ids() to MessageTable: single query
  fetches all reactions for multiple messages using IN clause with
  User JOIN, returns dict[message_id, list[Reactions]]
- Add get_thread_reply_counts_by_message_ids() to MessageTable: single
  GROUP BY aggregate query returns (count, max_created_at) per parent,
  replacing full object loads just to call len()
- Refactor get_channel_messages(): 102 → 4 queries per page
- Refactor get_pinned_channel_messages(): 22 → 3 queries per page
- Refactor get_channel_thread_messages(): 53 → 4 queries per page
- Refactor send_notification(): N+1 membership check → batch set lookup
2026-06-29 02:05:16 -05:00
Timothy Jaeryang Baek 46c1d6591b refac 2026-06-29 01:41:43 -05:00
Timothy Jaeryang Baek 3730a9eaac refac 2026-06-29 01:38:41 -05:00
alvarellos 368b4a5b22 solve-valves-icon-disappear-issue (#26256) 2026-06-29 00:56:44 -05:00
Timothy Jaeryang Baek 7e13fd7ad1 refac 2026-06-29 00:26:35 -05:00
Timothy Jaeryang Baek 95391221df refac 2026-06-23 23:35:44 +02:00
Timothy Jaeryang Baek 15c7e37438 refac 2026-06-23 23:13:32 +02:00
Timothy Jaeryang Baek 3f0c0e0a0d refac 2026-06-19 00:16:06 +02:00
Timothy Jaeryang Baek 8958b64b5a refac 2026-06-18 11:02:14 +02:00