display available cached versions in TGI server error message of Neuron backend

[jimburtoft]

Pulling from huggingface/optimum-neuron#776

If a model is cached with a different configuration, I want to display alternative options to the user.

If someone copies from the deploy code on Hugging Face and changes something (e.g. sequence length), it is not obvious why it isn't working from this code. (especially if they don't understand compiling because they are referencing the original model)

Based on a true story!

added some carriage returns to make it more readable

get_hub_cached_entries does generate an error if it is fed a model that doesn't have a model_type. For example: (randomly selected) model_id = "hexgrad/Kokoro-82M"

Traceback (most recent call last):
File "", line 1, in
File "/opt/aws_neuronx_venv_pytorch_2_1/lib/python3.10/site-packages/optimum/neuron/utils/hub_cache_utils.py", line 431, in get_hub_cached_entries
model_type = target_entry.config["model_type"]
KeyError: 'model_type'

However, we already call that function inside of is_cached at the top of this block, so I don't know if we are filtering for certain types of models before we get to this point or not. If not, the existing code would generate that error before it ever gets here.

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• [x ] Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

[dacorvo]

This method is already called by is_cached: I'd rather avoid having two consecutive calls to the hub.
The is_cached method is never called anywhere else, so maybe you can change its signature to something like has_compatible_entry(neuron_config, entries). That way you can first fetch the entries, check if one is compatible, and otherwise just loop over incompatible entries just like you do.

[dacorvo]

It looks quite redundant with the first line of the error message. Do we really need to say something more specific here ?