Refine 404 errors #878

LysandreJik · 2022-05-18T20:56:25Z

This PR adds finer error management to the Hugging Face Hub error messages.

Here's the difference between current main and this PR on the following errors:

An example for hf_api: model_info

Using the model_info method:

In [1]: from huggingface_hub import model_info

In [2]: model_info('random_model')

Before:

---------------------------------------------------------------------------
HTTPError                                 Traceback (most recent call last)
Input In [2], in <cell line: 1>()
----> 1 model_info('random_model')

[...]

File ~/Workspaces/Python/huggingface_hub/.env/lib/python3.10/site-packages/requests/models.py:960, in Response.raise_for_status(self)
    957     http_error_msg = u'%s Server Error: %s for url: %s' % (self.status_code, reason, self.url)
    959 if http_error_msg:
--> 960     raise HTTPError(http_error_msg, response=self)

HTTPError: 404 Client Error: Not Found for url: https://huggingface.co/api/models/random_model

After:

---------------------------------------------------------------------------
RepositoryNotFoundError                   Traceback (most recent call last)
Input In [2], in <cell line: 1>()
----> 1 model_info('random_model')

[...]

File ~/Workspaces/Python/huggingface_hub/src/huggingface_hub/utils/_errors.py:29, in _raise_for_status(request)
     27 error_code = request.headers["X-Error-Code"]
     28 if error_code == "RepoNotFound":
---> 29     raise RepositoryNotFoundError(
     30         f"404 Client Error: Repository Not Found for url: {request.url}"
     31     )
     32 elif error_code == "EntryNotFound":
     33     raise EntryNotFoundError(
     34         f"404 Client Error: Entry Not Found for url: {request.url}"
     35     )

RepositoryNotFoundError: 404 Client Error: Repository Not Found for url: https://huggingface.co/api/models/random_model

An example for cached_download

cached_download is an interesting test-case as it can raise the three different errors. Right now, on main, using either of the three following errors results in the same error message:

In [1]: cached_download(hf_hub_url('<bad_repo_id>', 'config.json'))
In [1]: cached_download(hf_hub_url('lysandre/sharded-repo, '<bad_file>'))
In [1]: cached_download(hf_hub_url('lysandre/sharded-repo', 'config.json', revision='<bad_revision>'))

In [1]: cached_download('https://huggingface.co/random_org/random_repo', 'file.json')

[...]

File ~/Workspaces/Python/huggingface_hub/.env/lib/python3.10/site-packages/requests/models.py:960, in Response.raise_for_status(self)
    957     http_error_msg = u'%s Server Error: %s for url: %s' % (self.status_code, reason, self.url)
    959 if http_error_msg:
--> 960     raise HTTPError(http_error_msg, response=self)

HTTPError: 404 Client Error: Not Found for url: <URL>

This PR will now provide tailored error messages:

Wrong repo ID

In [1]: cached_download(hf_hub_url('<bad_repo_id>', 'config.json'))


[...]

File ~/Workspaces/Python/huggingface_hub/src/huggingface_hub/utils/_errors.py:29, in _raise_for_status(request)
     27 error_code = request.headers["X-Error-Code"]
     28 if error_code == "RepoNotFound":
---> 29     raise RepositoryNotFoundError(
     30         f"404 Client Error: Repository Not Found for url: {request.url}"
     31     )
     32 elif error_code == "EntryNotFound":
     33     raise EntryNotFoundError(
     34         f"404 Client Error: Entry Not Found for url: {request.url}"
     35     )

RepositoryNotFoundError: 404 Client Error: Repository Not Found for url: https://huggingface.co/%3Cbad_repo%3E/resolve/main/config.json

Wrong file

In [1]: cached_download(hf_hub_url('lysandre/sharded-repo, '<bad_file>'))


[...]

File ~/Workspaces/Python/huggingface_hub/src/huggingface_hub/utils/_errors.py:33, in _raise_for_status(request)
     29     raise RepositoryNotFoundError(
     30         f"404 Client Error: Repository Not Found for url: {request.url}"
     31     )
     32 elif error_code == "EntryNotFound":
---> 33     raise EntryNotFoundError(
     34         f"404 Client Error: Entry Not Found for url: {request.url}"
     35     )
     36 elif error_code == "RevisionNotFound":
     37     raise RevisionNotFoundError(
     38         f"404 Client Error: Revision Not Found for url: {request.url}"
     39     )

EntryNotFoundError: 404 Client Error: Entry Not Found for url: https://huggingface.co/lysandre/sharded-repo/resolve/main/%3Cbad%3E

Wrong revision

In [1]: cached_download(hf_hub_url('lysandre/sharded-repo', 'config.json', revision='<bad_revision>'))

In [2]: cached_download(hf_hub_url('lysandre/sharded-repo', 'config.json', revision='bad'))

[...]

File ~/Workspaces/Python/huggingface_hub/src/huggingface_hub/utils/_errors.py:37, in _raise_for_status(request)
     33         raise EntryNotFoundError(
     34             f"404 Client Error: Entry Not Found for url: {request.url}"
     35         )
     36     elif error_code == "RevisionNotFound":
---> 37         raise RevisionNotFoundError(
     38             f"404 Client Error: Revision Not Found for url: {request.url}"
     39         )
     41 request.raise_for_status()

RevisionNotFoundError: 404 Client Error: Revision Not Found for url: https://huggingface.co/lysandre/sharded-repo/resolve/bad/config.json

HuggingFaceDocBuilderDev · 2022-05-18T21:08:01Z

The documentation is not available anymore as the PR was closed or merged.

LysandreJik · 2022-05-18T21:39:07Z

This PR can be independent of the cache-file-layout branch, so I'll switch the target to main and rebase cache-file-layout on this branch to benefit from the fix.

julien-c · 2022-05-19T07:52:16Z

src/huggingface_hub/utils/_errors.py

+        elif error_code == "EntryNotFound":
+            raise EntryNotFoundError(
+                f"404 Client Error: Entry Not Found for url: {request.url}"
+            )
+        elif error_code == "RevisionNotFound":
+            raise RevisionNotFoundError(
+                f"404 Client Error: Revision Not Found for url: {request.url}"
+            )


nit but I would change the order of those two as EntryNotFound implies that a revision was found

i.e. the errors are more and more "closer to being successful"

adrinjalali

Otherwise LGTM.

src/huggingface_hub/utils/_errors.py

adrinjalali · 2022-05-19T09:48:21Z

src/huggingface_hub/utils/_errors.py

+    Raised when trying to access a hf.co URL with an invalid repository name, or
+    with a private repo name the user does not have access to.
+    """


The first line should be a short single line explanation, then a longer description if necessary, an an example of when the exception would be raised would also be nice.

Good call! Updated in latest commit

osanseviero · 2022-05-20T07:58:30Z

src/huggingface_hub/utils/_errors.py

+    """
+
+
+class EntryNotFoundError(HTTPError):


Should this maybe be FileNotFoundError?

Will let @julien-c comment on the choice of "entry", but I'd rather stay as close as possible to the error thrown by the backend

no strong opinion! up to you.

Maybe we can document that this mean that the repo AND the revision exist/are accessible, BUT not the file/entry? or is it clear enough already?

LysandreJik · 2022-06-10T09:54:32Z

Will adapt to the new 401 error.

LysandreJik · 2022-06-10T12:30:38Z

@SBrandeis, @adrinjalali, could you verify and confirm this looks good to you?

adrinjalali

Otherwise LGTM!

adrinjalali · 2022-06-10T12:36:21Z

src/huggingface_hub/hf_api.py

@@ -1180,6 +1181,18 @@ def model_info(

        Returns:
            [`huggingface_hub.hf_api.ModelInfo`]: The model repository information.
+
+        <Tip>


should this be a tip or in the Raises section? (same for the following changes)

So far we have been using <Tip> across the codebase as Raises wasn't working with doc-builder. It's my fault, it seems I was delaying huggingface/doc-builder#141.

If it's fine with you, I'll merge it like this and open an issue to modify all of those <Tip> to Raises: once the PR above is merged.

SBrandeis · 2022-06-10T12:41:10Z

tests/test_file_download.py

+        url = hf_hub_url("bert-base", filename="pytorch_model.bin")
+        with self.assertRaisesRegex(
+            RepositoryNotFoundError, "401 Client Error: Repository Not Found"
+        ):
            _ = cached_download(url, legacy_cache_layout=True)


Note the behavior changes when authenticated - in that case the error message will be:
404 Client Error: Repository Not Found

SBrandeis · 2022-06-10T12:45:55Z

LGTM 👍

Let's maybe add a comment stating that http_get / cached_download should only be used to request stuff related to a repo? (because otherwise raising RepoNotFOundError does not make a lot of sense)

Or maybe add an additional check in _raise_for_status to that end?

Feel free to disregard if you think it's not relevant :)

julien-c · 2022-06-11T11:10:37Z

src/huggingface_hub/utils/_errors.py

+    if "X-Error-Code" in request.headers:
+        error_code = request.headers["X-Error-Code"]
+        if error_code == "RepoNotFound":
+            raise RepositoryNotFoundError(
+                f"404 Client Error: Repository Not Found for url: {request.url}. "
+                "If the repo is private, make sure you are authenticated."
+            )
+        elif error_code == "RevisionNotFound":
+            raise RevisionNotFoundError(
+                f"404 Client Error: Revision Not Found for url: {request.url}"
+            )
+        elif error_code == "EntryNotFound":
+            raise EntryNotFoundError(
+                f"404 Client Error: Entry Not Found for url: {request.url}"
+            )
+
+    if request.status_code == 401:
+        # The repo was not found and the user is not Authenticated
+        raise RepositoryNotFoundError(
+            f"401 Client Error: Repository Not Found for url: {request.url}. "
+            "If the repo is private, make sure you are authenticated."
+        )


Logic of _raise_for_status LGTM (also cc @Pierrci @allendorf @coyotte508 for a double check)

BTW we should maybe have "extended" i.e. wrapped the existing HTTPError instead of re-instantiating a new one.

This change broke this code in Speechbrain:
https://github.com/speechbrain/speechbrain/blob/d6bfe138a90dff3490f7196acbd9c62939289d46/speechbrain/pretrained/fetching.py#L115-L121

cc @osanseviero @mravanelli

Also the 404 shouldn't be hardcoded but should ideally be replaced by {request.status_code} to reflect the actual server output

LysandreJik · 2022-06-13T07:16:40Z

I added some comments @SBrandeis. Should be good to merge.

* Add errors * Style * Review * hf_hub_download support * Errors for hf_hub_download * Typo * Handle 401 error * Tests for 401 error * Typo * Review comments

@XciD

* added autoeval fields to repocard types * modified test, linted * Fix typo (#902) * Refine 404 errors (#878) * Add errors * Style * Review * hf_hub_download support * Errors for hf_hub_download * Typo * Handle 401 error * Tests for 401 error * Typo * Review comments * added documentation for the repocard change in repocard, updated metadata_eval_result, added documentation * 🏗 Use `hub-ci` for tests (#898) * 🏗 Use `hub-ci` for tests cc @XciD * 🩹 Also update URL for staging mode * ✅ 401 is raised when the user is not authenticated * 🗑 Deprecare `identical_ok` * Longer deprecation period * ✅ Fix the last failing test * Warning match docstring * FIX Avoid creating repository when it exists on remote (#900) * fix for spaces * fix for spaces * removed creating repository and added warning * revert my changes * added tests * removed debugger 😐 * fixed repository removal * Added tests and error * import pytest * fixed tests * fixed tests * style * removed repo removal * make style * fixed test with Lysandres patch * added fix * Remove deprecations (#910) * Remove deprecations * Typo * Update src/huggingface_hub/README.md Co-authored-by: Julien Chaumond <julien@huggingface.co> Co-authored-by: Julien Chaumond <julien@huggingface.co> * Add request ID to all requests (#909) * Add request ID to all requests * Typo * Typo * Review comments * Update src/huggingface_hub/utils/_errors.py Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com> Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com> * Invert deprecation for create_repo (#912) * Invert deprecation for create_repo * Set to 0.10 * Constant was accidentally removed during deprecation transition (#913) Co-authored-by: Lyle Nel <lyle@owlin.com> * Update src/huggingface_hub/repocard_types.py Co-authored-by: lewtun <lewis.c.tunstall@gmail.com> * Update repocard_types.py * Update repocard_types.py * reorded metadata_eval_request_docs, added metrics_config and metrics_verified to tests Co-authored-by: lsb <leebutterman@gmail.com> Co-authored-by: Lysandre Debut <lysandre.debut@reseau.eseo.fr> Co-authored-by: Simon Brandeis <33657802+SBrandeis@users.noreply.github.com> Co-authored-by: Merve Noyan <merveenoyan@gmail.com> Co-authored-by: Julien Chaumond <julien@huggingface.co> Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com> Co-authored-by: Lyle Nel <lyle-nel@users.noreply.github.com> Co-authored-by: Lyle Nel <lyle@owlin.com> Co-authored-by: lewtun <lewis.c.tunstall@gmail.com>

LysandreJik force-pushed the custom-errors branch from bc45c4e to 9058578 Compare May 18, 2022 21:41

LysandreJik changed the base branch from cache-file-layout to main May 18, 2022 21:41

LysandreJik force-pushed the custom-errors branch from 992aa13 to 2ef1f90 Compare May 18, 2022 21:56

LysandreJik mentioned this pull request May 18, 2022

New git-aware cache file layout #801

Merged

LysandreJik requested review from adrinjalali and osanseviero May 18, 2022 22:11

julien-c reviewed May 19, 2022

View reviewed changes

adrinjalali reviewed May 19, 2022

View reviewed changes

LysandreJik mentioned this pull request May 19, 2022

Bad error message when downloading private model without being logged in. huggingface/transformers#15980

Closed

osanseviero reviewed May 20, 2022

View reviewed changes

LysandreJik added 4 commits June 9, 2022 12:05

Add errors

823972e

Style

2fc028b

Review

31ea59b

hf_hub_download support

8e65f05

LysandreJik force-pushed the custom-errors branch from 0ff6f39 to 8e65f05 Compare June 9, 2022 10:11

LysandreJik added 2 commits June 9, 2022 12:13

Errors for hf_hub_download

6e746f1

Typo

dfc313a

LysandreJik added this to the v0.8 milestone Jun 10, 2022

LysandreJik mentioned this pull request Jun 10, 2022

Release v0.8 #899

Closed

LysandreJik added 3 commits June 10, 2022 12:18

Handle 401 error

e7ba7dc

Tests for 401 error

29fcc65

Typo

b5d8470

adrinjalali reviewed Jun 10, 2022

View reviewed changes

SBrandeis reviewed Jun 10, 2022

View reviewed changes

SBrandeis mentioned this pull request Jun 10, 2022

🏗 Use hub-ci for tests #898

Merged

julien-c reviewed Jun 11, 2022

View reviewed changes

Review comments

36c1d71

SBrandeis self-requested a review June 13, 2022 07:25

SBrandeis approved these changes Jun 13, 2022

View reviewed changes

julien-c approved these changes Jun 13, 2022

View reviewed changes

julien-c merged commit d61d6ec into huggingface:main Jun 13, 2022

Pierrci mentioned this pull request Jun 14, 2022

Hub API calls: print X-Request-Id value in case of server error #906

Closed

TristanThrush pushed a commit that referenced this pull request Jun 19, 2022

Refine 404 errors (#878)

ddb266c

* Add errors * Style * Review * hf_hub_download support * Errors for hf_hub_download * Typo * Handle 401 error * Tests for 401 error * Typo * Review comments

osanseviero mentioned this pull request Jun 22, 2022

Fix custom handling of refined HTTPError #924

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refine 404 errors #878

Refine 404 errors #878

LysandreJik commented May 18, 2022

HuggingFaceDocBuilderDev commented May 18, 2022 •

edited

Loading

LysandreJik commented May 18, 2022 •

edited

Loading

julien-c May 19, 2022

adrinjalali left a comment

adrinjalali May 19, 2022

LysandreJik Jun 9, 2022

osanseviero May 20, 2022

LysandreJik Jun 9, 2022

julien-c Jun 9, 2022

LysandreJik commented Jun 10, 2022

LysandreJik commented Jun 10, 2022

adrinjalali left a comment

adrinjalali Jun 10, 2022

LysandreJik Jun 13, 2022

SBrandeis Jun 10, 2022

SBrandeis commented Jun 10, 2022

julien-c Jun 11, 2022 •

edited

Loading

julien-c Jun 21, 2022

julien-c Jun 21, 2022

julien-c Jun 24, 2022

LysandreJik commented Jun 13, 2022

Refine 404 errors #878

Refine 404 errors #878

Conversation

LysandreJik commented May 18, 2022

HuggingFaceDocBuilderDev commented May 18, 2022 • edited Loading

LysandreJik commented May 18, 2022 • edited Loading

Choose a reason for hiding this comment

adrinjalali left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

LysandreJik commented Jun 10, 2022

LysandreJik commented Jun 10, 2022

adrinjalali left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SBrandeis commented Jun 10, 2022

julien-c Jun 11, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

LysandreJik commented Jun 13, 2022

HuggingFaceDocBuilderDev commented May 18, 2022 •

edited

Loading

LysandreJik commented May 18, 2022 •

edited

Loading

julien-c Jun 11, 2022 •

edited

Loading