Onnx quantized backend for Clip-ViT-B-16

[PraNavKumAr01]

@tomaarsen
Just wanted to know if clip (text + image) embedding models will have an onnx quantized model? i tried finding it everywhere but had no luck. If it is there can you please point me to it? And if not, is it possible to create a model_qint8_avx512_vnni.onnx for it.
Can we expect it to be there with future updates or would i have to run some experiments and convert it on my own?

[tomaarsen]

Hello!

I'm afraid that CLIP models don't have ONNX support in Sentence Transformers right now. In short, CLIP models are loaded with the CLIPModel module, whereas the ONNX support is implemented in the Transformer module. This latter module is used for text-based embedding models.

• Tom Aarsen