New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

Explore audio compatible models #148

Open

deven96 opened this issue Nov 28, 2024 · 1 comment

Assignees

Labels

enhancement experimentation

Owner

deven96 commented Nov 28, 2024

Potential for adding models that can process audio files into embeddings

deven96 added enhancement experimentation labels

deven96 assigned HAKSOAT

Collaborator

HAKSOAT commented Jan 9, 2025 •

edited

Loading

We probably should be looking in the direction of

CLAP: https://huggingface.co/laion/clap-htsat-unfused
Wav2Vec2 model (and its variants): https://huggingface.co/facebook/wav2vec2-base-960h.

I can also see the Hubert model: https://huggingface.co/docs/transformers/en/model_doc/hubert and Encodec: https://huggingface.co/facebook/encodec_24khz, but they do not seem to have a tokenizer and requires training.

# for free to join this conversation on GitHub. Already have an account? # to comment