Enabling ENABLE_MILVUS_MULTITENANCY_MODE with the default MILVUS_URI (embedded Milvus Lite at DATA_DIR/vector_db/milvus.db) fails on the first embedding write: _create_shared_collection calls collection.create_index(RESOURCE_ID_FIELD) with no index params. A Milvus server auto-selects a scalar index type in that case, but Milvus Lite rejects the call with "create_index missing required 'index_type' parameter", so shared collection creation raises and every embedding write 500s (memory add, file upload, knowledge writes).
Keep the parameterless call as the first attempt so behavior on Milvus servers is unchanged, fall back to an explicit INVERTED scalar index, and if that also fails log a warning and continue. The scalar index only accelerates resource_id filters; inserts and filtered queries work without it, so a missing index must not break collection creation.
Verified against embedded Milvus Lite: shared collections now create (with the warning), and memory add, file upload and memory query succeed end to end. Against a Milvus server the first attempt is identical to the current code, so nothing changes where it works today.
has_collection did `collection_name in self.client.list_collections()`,
but chromadb's list_collections() returns Collection objects (1.x), not
name strings — so the membership test is always False, even when the
collection exists. Compare against the collection names instead (with a
hasattr guard tolerating versions that yield plain names).
Found via the dependency-contract test suite (unit/deps/test_chromadb.py).
The Qdrant backends implemented delete(ids=...) as a payload filter on
metadata.id, but points are stored with the item id as the Qdrant point id
(see _create_points), and not every point carries an id in its payload.
Memory points store only {created_at} in metadata (KB metadata embeddings
likewise), so deleting a single memory matched nothing and left an orphaned
vector that kept being injected into RAG context.
Delete by point id instead: PointIdsList for the standard backend, and a
tenant-scoped HasIdCondition for multitenancy (point ids are unique, so tenant
isolation is preserved). Filter-based deletion is unchanged.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat: add support for Valkey vector database
Signed-off-by: Riley Des <riley.desserre@improving.com>
* feat: add CLIENT SETNAME to Valkey vector store connections
Set client_name on GlideClientConfiguration for both the main client
and batch client so connections are identifiable in CLIENT LIST,
monitoring dashboards, and CloudWatch metrics.
Signed-off-by: Riley Des <riley.desserre@improving.com>
---------
Signed-off-by: Riley Des <riley.desserre@improving.com>
Open WebUI's collection ACL accepted any unknown name as a
legacy/ephemeral collection. In Milvus multi-tenancy mode that name
becomes the `resource_id` and is interpolated unescaped into a SQL-like
Milvus expression — `resource_id == '<name>'` — so a name like
x' or resource_id != '' or resource_id == 'x
turns the filter into a tautology and returns every tenant's chunks
from the shared collection.
All collection names Open WebUI generates are UUIDs, SHA-256 hex
digests, or fixed-prefix variants of those — they all fit
[A-Za-z0-9_-]. Add a strict format check in
filter_accessible_collections (utils.py) that drops any name outside
that set before any ACL or vector-store lookup, applied even on the
admin bypass path. _validate_collection_access then surfaces the dropped
name as a 403.
As defense in depth, MilvusClient now validates resource_id at every
expression-construction site and escapes single quotes / backslashes in
any other string interpolated into a filter (delete ids, metadata
filter values). Non-string filter values are typed-checked instead of
str()-formatted.
Co-authored-by: Claude <noreply@anthropic.com>
Replace bare except clauses with except Exception to follow Python best practices and avoid catching unexpected system exceptions like KeyboardInterrupt and SystemExit.
In opensearch-py >= 3.0.0, IndicesClient.refresh() no longer accepts the
index name as a positional argument. This causes a TypeError when
uploading documents to knowledge bases with OpenSearch backend.
Changes positional arguments to keyword arguments (index=...) in all
three refresh() calls in the OpenSearch vector DB client.
Fixes#20649
* fix: Use get_index() instead of list_indexes() in has_collection() to handle pagination
Fixes#19233
Replace list_indexes() pagination scan with direct get_index() lookup
in has_collection() method. The previous implementation only checked
the first ~1,000 indexes due to unhandled pagination, causing RAG
queries to fail for indexes beyond the first page.
Benefits:
- Handles buckets with any number of indexes (no pagination needed)
- ~8x faster (0.19s vs 1.53s in testing)
- Proper exception handling for ResourceNotFoundException
- Scales to millions of indexes
* Update s3vector.py
Unneeded exception handling removed to match original OWUI code
* Adding hnsw index type for pgvector, allowing vector dimensions larger than 2000
* remove some variable assignments
* Make USE_HALFVEC variable configurable
* Simplify USE_HALFVEC handling
* Raise runtime error if the index requires rebuilt
---------
Co-authored-by: Moritz <moritz.mueller2@tu-dresden.de>
This commit introduces support for the DISKANN index type in the Milvus vector database integration.
Changes include:
- Added `MILVUS_DISKANN_MAX_DEGREE` and `MILVUS_DISKANN_SEARCH_LIST_SIZE` configuration variables.
- Updated the Milvus client to recognize and configure the DISKANN index type during collection creation.
The pymilvus library expects -1 for unlimited queries, but the code was passing None, which caused a TypeError. This commit changes the default value of the limit parameter in the query method from None to -1. It also updates the call site in the get method to pass -1 instead of None and updates the type hint and a comment to reflect this change.
Timothy Jaeryang Baek