Finer selection

[LysandreJik]

WIP PR, do not merge.

This PR offers a more powerful way to filter models by tags. This breaks the limitations imposed by selecting a model by a list of tags.

It introduces five enumerations: Language, Task, Dataset, License and Library. These four enumerations allow filtering of models (and eventually datasets when #194 is merged). Alongside these five enumerations are four operators which can be used to make a more powerful filter: & (AND) , | (OR) and ~ (NOT).

See below for some examples of common use-cases:

Example of single categories using languages

from huggingface_hub.hf_api import HfApi, Language

# List all English models
english_models = api.list_models_by_tag(Language.en)
len(english_models)  # 3061

# List all English or French models
en_or_fr_models = api.list_models_by_tag(Language.en | Language.fr)
len(en_or_fr_models)  # 3318

# List all English or French models
en_and_fr_models = api.list_models_by_tag(Language.en | Language.fr)
len(en_and_fr_models)  # 27

# List all English models that are not French models
en_and_fr_models = api.list_models_by_tag(Language.en & ~Language.fr)
len(en_and_fr_models)  # 3034

More advanced typical use-cases.

I'm looking for a model that is both English and French, specialized in translation

from huggingface_hub.hf_api import HfApi, Language, Task

# List all English models
translation = api.list_models_by_tag(Task.translation & (Language.en & Language.fr))
len(translation)  # 19

I'm looking for a model that is both English and French, specialized in translation. However, I don't want a model trained on multilingual, as those usually contain a lot of different languages. French & English only!

from huggingface_hub.hf_api import HfApi, Language, Task

# List all English models
translation = api.list_models_by_tag(Task.translation & (Language.en & Language.fr & ~Language.mt))
len(translation)  # 17

[adrinjalali]

You could probably use Flag instead? (https://docs.python.org/3/library/enum.html#flag)

[osanseviero]

This is probably obsolete with the new advanced search introduced by @muellerzr

[muellerzr]

Correct. @LysandreJik are we good to close this you think?

[LysandreJik]

Indeed, obsolete and should be closed.