FEAT: support milvus to full text search #11430
Open
+680
−680
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Modified to support new sparse_vector field types and configurations required for full-text search. - Implemented full-text search capabilities in . - Updated to include new dependencies(pymilvus==2.5.0 ]python-dotenv=1.0.1) - Enhanced integration tests in to cover the new full-text search functionality. This commit improves the search functionality within the Milvus data source, ensuring efficient and accurate retrieval of relevant data. Signed-off-by: YoungLH <974840768@qq.com>
…lt docker-compose.yaml Signed-off-by: YoungLH <974840768@qq.com>
Signed-off-by: YoungLH <974840768@qq.com>
dosubot
bot
added
size:M
|
May i Know will this affect the old data? @kgpp34 |
…of existing data Signed-off-by: YoungLH <974840768@qq.com>
dosubot
bot
added
size:L
Signed-off-by: YoungLH <974840768@qq.com>
I fix my code and it will guarantee the change will not influence the CRUD on the old data |
|
Please fix the lint errors |
i have fixed the poetry lock link error |
|
pls resolve conflicts thanks @kgpp34 |
…_search # Conflicts: # api/core/rag/datasource/vdb/milvus/milvus_vector.py # api/poetry.lock
Signed-off-by: YoungLH <974840768@qq.com>
I have resolved these conflicts, thanks |
# for free
to join this conversation on GitHub.
Already have an account?
# to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This pull request focuses on enabling full-text search capabilities within the Milvus data source, improving the overall search functionality and ensuring that the integration is robust and reliable.
Resolves #11370
Motivation and Context
The motivation behind this change is to improve the search capabilities of the Dify project by integrating full-text search support in the Milvus data source. This enhancement allows for more efficient and accurate retrieval of relevant data, which is crucial for improving the overall user experience.
Dependencies
pymilvus(version update to 2.5.0)python-dotenv(version update to 1.0.1)milvus(version 2.5.0-beta)Detailed Changes
Modified:
core/rag/datasource/vdb/field.pyModified:
core/rag/datasource/vdb/milvus/milvus_vector.pyModified:
pyproject.tomlModified:
tests/integration_tests/vdb/milvus/test_milvus.pyModified:
docker/docker-compose.yamlChecklist
Important
Please review the checklist below before submitting your pull request.
dev/reformat(backend) andcd web && npx lint-staged(frontend) to appease the lint gods