Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

opam does not wait for the Software Heritage fallback archives to be "cooked" #5721

Closed
kit-ty-kate opened this issue Nov 8, 2023 · 3 comments · Fixed by #6036
Closed

opam does not wait for the Software Heritage fallback archives to be "cooked" #5721

kit-ty-kate opened this issue Nov 8, 2023 · 3 comments · Fixed by #6036
Assignees
@rjbou
Labels
KIND: BUG
Milestone
2.4.0~alpha1

Comments

@kit-ty-kate
Copy link
Member

When using the software heritage fallback feature, it is understood that regenerating (as noted in #5720) the archive from a given SWHID (called "cooking" by Software Heritage) might take some time. The code currently denotes this by a 60 seconds timer:

opam/src/repository/opamDownload.ml

Line 299 in c4961ba

Unix.sleep 10;

However in practice, trying it locally, leads only a short wait then the following failure:

OpamSolution.Fetch_fail("\"/opt/homebrew/bin/wget -X POST --content-disposition -t 3 -O - -U opam/2.2.0~alpha2 -- https://archive.softwareheritage.org/api/1/vault/directory/6f61d745641e701d5cad73a9e918d5e888d5ab64/\" exited with code 8 \"2023-11-08 16:00:08 ERROR 404: Not Found.\"")

Calling the same command again when the archive has finished cooking, succeeds.
@kit-ty-kate
Copy link
Member Author

The software heritage fallback was disabled by default in #5899 so moving this issue off the 2.2 milestone

@kit-ty-kate
Copy link
Member Author

After further investigation it looks like wget does not work to fetch the expected data, but curl does:

$ curl https://archive.softwareheritage.org/api/1/vault/directory/6f61d745641e701d5cad73a9e918d5e888d5ab64/
{"fetch_url": ...}

$ wget -O - https://archive.softwareheritage.org/api/1/vault/directory/6f61d745641e701d5cad73a9e918d5e888d5ab64/
<!DOCTYPE html>...

@rjbou rjbou self-assigned this May 27, 2024
@rjbou
Copy link
Collaborator

rjbou commented Jun 13, 2024

Some more output

Skipping 121 bytes of body: [{"exception":"NotFoundExc","reason":"Cooking of swh:1:dir:45b87e1ca127914b936884dc73c9f788df5b7abd was never requested."}] done.
2024-06-13 18:04:26 ERROR 404: Not Found.

@rjbou rjbou linked a pull request Jun 19, 2024 that will close this issue
Software Heritage fallback fixes #6036
Merged
@rjbou rjbou mentioned this issue Jun 19, 2024
Merged
@rjbou rjbou added this to the 2.3 milestone Jun 27, 2024
# for free to join this conversation on GitHub. Already have an account? # to comment
Assignees

@rjbou rjbou

Labels
KIND: BUG
Projects
None yet
Milestone
2.4.0~alpha1
Development

Successfully merging a pull request may close this issue.

Software Heritage fallback fixes rjbou/opam
2 participants
@kit-ty-kate @rjbou