New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

我觉得使用 SentenceTransformer 作为句子向量化不好 #28

Open

shguan2018 opened this issue Sep 29, 2021 · 1 comment

shguan2018 commented Sep 29, 2021

我觉得使用 SentenceTransformer 作为句子向量化不好,dimension 高达 512,句子向量化存储到 annoy 里导致空间特别大

Collaborator

xiaoyichao commented Oct 22, 2021

我觉得使用 SentenceTransformer 作为句子向量化不好,dimension 高达 512,句子向量化存储到 annoy 里导致空间特别大

如果觉得大。可以不使用向量召回，只使用Es召回，BERT类的应用都有高消耗的情况，硬件和效果之间总得做个平衡。一般的数据集向量化之后都是工业上可接受的范围。还有一个思路就是将向量在Es中召回，Es7支持向量召回。

# for free to join this conversation on GitHub. Already have an account? # to comment