Error: Failed to persist attestation: Invalid Argument - values do not match

[falcorocks]

hello!

The attestation action is broken currently for GHE users, it worked yesterday afternoon Central Europe time.
The attestation action works correctly with public sigstore, which makes me think maybe something is not right with Github instance of Fulcio?

The full error I get is https://github.com/falcorocksacme/crispy-octo-lamp/actions/runs/10630041914/job/29468140909

Error: Error: Failed to persist attestation: Invalid Argument - values do not match: https://github.com/falcorocksacme/crispy-octo-lamp != https://falcorockacme.ghe.com/falcorocksacme/crispy-octo-lamp - https://docs.github.com/rest/repos/repos#create-an-attestation

It appears that for some reason the action now expects the signer repository to be hosted at https://falcorockacme.ghe.com, which is not my case currently, I'at github.com/falcorocksacme.

Note: I had to create an organisation with an internal repository, run the action and then make the repository public to force the usage of the Github managed sigstore instance.

[algo7]

I was about to open an issue regarding the exact same problem. My repository has to stay private due to company policy so unfortunately I can't use the workaround mentioned above. Everything was working yesterday, has anything been changed on GitHub's side?

[falcorocks]

BTW @algo7 I just found out that there's this ENV variable available to force the usage of the Github Sigstore instance for public projects

attest-build-provenance/.github/workflows/ci.yml

Line 65 in 13f0f0d

INPUT_PRIVATE-SIGNING: 'true'

Would be nice if it was documented in the README 👼

[algo7]

BTW @algo7 I just found out that there's this ENV variable available to force the usage of the Github Sigstore instance for public projects

attest-build-provenance/.github/workflows/ci.yml

Line 65 in 13f0f0d

INPUT_PRIVATE-SIGNING: 'true'

Would be nice if it was documented in the README 👼

Ye, the docs is not clear but I suspect that there are something else that might have changed on GitHub's side that the team maintaining this action is not aware of. Btw, setting INPUT_PRIVATE-SIGNING: 'true' only works for public repo right?

[falcorocks]

BTW @algo7 I just found out that there's this ENV variable available to force the usage of the Github Sigstore instance for public projects

attest-build-provenance/.github/workflows/ci.yml

Line 65 in 13f0f0d

INPUT_PRIVATE-SIGNING: 'true'

Would be nice if it was documented in the README 👼

Ye, the docs is not clear but I suspect that there are something else that might have changed on GitHub's side that the team maintaining this action is not aware of. Btw, setting INPUT_PRIVATE-SIGNING: 'true' only works for public repo right?

Indeed this appears like it was caused by a change elsewhere. I suspect at their Fulcio instance or with the OIDC token... As for the INPUT_PRIVATE-SIGNING: 'true' I have not tested myself. Aside from the bug, a few considerations on availability:

1. the Github Sigstore deployment powering this feature should have a public status at https://www.githubstatus.com/ like the public good sigstore does
2. https://github.com/actions/attest-build-provenance/blob/13f0f0dbc538ab425876cb63ebfe4b7f3d080cbd/.github/workflows/ci.yml should be probably run continuosly/regularly to quickly discover this type of problems
3. if rely on Github to sign artifacts, and artifact attestation goes down we are blocked. I wish the action provided a way to sign with a user managed key in the worst case scenario. One can implement this by himself, however it would be neat if the action provided this logic through inputs

[bdehamer]

I'm investigating this now. I suspect that this issue was introduced as part of the fix for tenancy-related change which was recently introduced (in v1.4.1). Until we can get a fix deployed, you should be able to work-around this by pinning your action to v1.4.0.

[amber-beasley-liatrio]

similar behavior happening with verify

cli/cli#9543

[amber-beasley-liatrio]

I attempted to pin action to v1.4.0 but, I am still getting the same error

it might be helpful to know that v1 was working yesterday, and today it is not

uses: actions/attest-build-provenance@v1.4.0

!= https://myorg-partnerdemo.ghe.com/myorg/myrepo

instead of the domain being GitHub

[algo7]

I attempted to pin action to v1.4.0 but, I am still getting the same error

uses: actions/attest-build-provenance@v1.4.0

!= https://myorg-partnerdemo.ghe.com/myorg/myrepo

instead of the domain being GitHub

Same here, manually pinning the action's version to v1.4.0 does not seem to resolve the issue.

[bdehamer]

Issue has been identified in a recent deployment of the GitHub internal Fulcio instance. In the process of rolling back now.

[bdehamer]

you should be able to work-around this by pinning your action to v1.4.0

This is incorrect. I thought the issue might have been related to some recent changes in the action itself but that is NOT the case.

The root cause is this change deployed in v1.6.3 of Fulcio -- which impacts the GH internal instance and the Sigstore Public Good instance (this version was never deployed to Sigstore PGI).

The GH Fulcio instance has been rolled-back.

[LaszloKoller]

I can report that a GitHub workflow utilizing the actions/attest-build-provenance@v1 action is now finishing successfully, whereas that same workflow failed during the attestation action (step) just a few hours ago. (I simply re-ran the previously failed workflow attempt.)

[algo7]

I can confirm that it's working with v1.4.2 as well.

[bdehamer]

Btw, setting INPUT_PRIVATE-SIGNING: 'true' only works for public repo right?

This is correct. This is primarily intended to allow us to test against our GH Fulcio instance from public repositories (which would otherwise default to using the Sigstore Public Good instance).

If users think that this would be a generally useful feature, we could certainly expose this more formally . . . till now we haven't heard requests for this feature so it's remained undocumented.

[falcorocks]

@bdehamer I too can confirm that now the action works, thanks!