vector store not working properly with orderby clause

[affan00733]

Search before asking

• I have searched the EVA DB issues and found no similar bug report.

EVA DB Component

Other

Bug

• when performing the similarity based query on vector that is creating a vector store from the table features by FAISS and QDRANT, it seems to be corrupted as when locally reading the entire vector store of 2 pdfs it is returning 517

Environment

No response

Minimal Reproducible Example

import faiss index = faiss.read_index("/Users/afaanansari/Desktop/gtech/eva/tutorials/evadb_data/index/FAISS_faiss_indexs.index") index.ntotal

Additional

No response

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

[github-actions]

👋 Hello @affan00733, thanks for your interest in EVA DB 🙏 Please visit our 🔮 Tutorials to get started, where you can find quickstart guides for simple tasks like Image Classification all the way to more interesting tasks like Emotion Analysis.

If this is a 🐞 Bug Report, please provide a minimum reproducible example to help us debug it.

If this is a ❓ Question, please provide as much information as possible, including dataset examples and query results.

[jiashenC]

Can you provide detailed steps to reproduce this error? Not sure if this is related to order by desc, but #898 tests desc works fine.

[jiashenC]

Error can be reproduced through tutorials/privategpt notebook.

• Answers are not ordered correctly.
• Limit clause not working.

[jiashenC]

The root cause is different paragraphs in from the same PDF are assigned to the same _row_id. Is this PR #868 intended to address this issue? @gaurav274

[gaurav274]

Nice catch. Yes, we need to improve the support of creating indexes directly on unstructured tables.

[xzdandy]

I think the issue different paragraphs in from the same PDF are assigned to the same _row_id. this has been already addressed.