refac: use MilvusClient instead of deprecated ORM-style PyMilvus APIs (#27521)
* refac: use MilvusClient instead of deprecated ORM-style PyMilvus APIs PyMilvus 2.6 emits a PyMilvusDeprecationWarning for every ORM-style call (`connections.connect`, `utility.*`, `Collection` and its methods) and will remove those APIs in PyMilvus 3.1. Both Milvus backends still used them, so a running instance floods its logs with deprecation warnings during indexing and retrieval, and would break outright once PyMilvus 3.1 lands. Both vector clients now go through `MilvusClient`: - `milvus_multitenancy.py`: collection creation, index creation, has_collection, insert, search, query iteration, delete and reset. - `milvus.py`: the remaining ORM calls in `query()` (`connections.connect`, `Collection(...).load()`, `Collection.query_iterator`), plus the now-unused `FieldSchema` import. Behaviour is unchanged: same schema, same index parameters and the same two-step scalar-index fallback, same filter expressions, same result shapes. Verified against embedded Milvus (milvus-lite, pymilvus 2.6.14) with a functional harness over both clients: insert, get, query by string/int/bool metadata filters, vector search, tenant isolation, oversized-text truncation, delete by id and by filter, delete_collection and reset all return identical results before and after, while the deprecation warnings drop from 57 to 0 for the multi-tenancy client and from 16 to 0 for the standard one. One Milvus Lite nuance worth recording: `MilvusClient` sends index build parameters (`M`, `efConstruction`, `nlist`) as flat keys rather than as a nested `params` blob. A Milvus server accepts both forms, Milvus Lite only reads the nested one, so those tuning values are ignored on Lite. `MilvusClient` offers no way to send the nested form, and `milvus.py` already built its index parameters this way, so both backends are now consistent. Fixes #26978 * refac: correct the Milvus scalar-index comment The comment claimed that embedded Milvus Lite requires an explicit scalar index type. It does not: Milvus Lite rejects `create_index` on a VARCHAR field outright ("create_index only supports vector fields"), for every index type and with or without a metric type, so neither the parameterless call nor the explicit INVERTED fallback can succeed there. Filtered queries on `resource_id` still work on Lite, just unindexed. Only the accurate half is kept, which is the reason the parameterless call is deliberate rather than an omission.
This commit is contained in:
1 parent
f2ff310b2a
commit
a15e44a5ff
2 files changed
+67
-81
No files matched your search
@@ -25,7 +25,7 @@ from open_webui.retrieval.vector.main import (
|
||||
VectorItem,
|
||||
)
|
||||
from open_webui.retrieval.vector.utils import process_metadata
|
||||
from pymilvus import Collection, DataType, FieldSchema, connections
|
||||
from pymilvus import DataType
|
||||
from pymilvus import MilvusClient as Client
|
||||
from pymilvus.exceptions import MilvusException
|
||||
|
||||
@@ -202,8 +202,6 @@ class MilvusClient(VectorDBBase):
|
||||
return self._result_to_search_result(result)
|
||||
|
||||
def query(self, collection_name: str, filter: dict, limit: int = -1):
|
||||
connections.connect(uri=MILVUS_URI, token=MILVUS_TOKEN, db_name=MILVUS_DB)
|
||||
|
||||
collection_name = collection_name.replace('-', '_')
|
||||
if not self.has_collection(collection_name):
|
||||
log.warning(f'Query attempted on non-existent collection: {self.collection_prefix}_{collection_name}')
|
||||
@@ -218,16 +216,16 @@ class MilvusClient(VectorDBBase):
|
||||
|
||||
filter_string = ' && '.join(filter_expressions)
|
||||
|
||||
collection = Collection(f'{self.collection_prefix}_{collection_name}')
|
||||
collection.load()
|
||||
self.client.load_collection(collection_name=f'{self.collection_prefix}_{collection_name}')
|
||||
|
||||
try:
|
||||
log.info(
|
||||
f"Querying collection {self.collection_prefix}_{collection_name} with filter: '{filter_string}', limit: {limit}"
|
||||
)
|
||||
|
||||
iterator = collection.query_iterator(
|
||||
expr=filter_string,
|
||||
iterator = self.client.query_iterator(
|
||||
collection_name=f'{self.collection_prefix}_{collection_name}',
|
||||
filter=filter_string,
|
||||
output_fields=[
|
||||
'id',
|
||||
'data',
|
||||
|
||||
Reference in new issue
Block a user