Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Libgit2 ED25519 check & clone respect context #445

Closed

Conversation

darkowlzz
Copy link
Contributor

This adds a test to detect any regression in libgit2's ED25519 key
support. go-git supports ED25519 but not the current version of
libgit2 used in flux. The updates to libgit2 in v1.2.0 adds support
for ED25519. This test would help ensure the right version of libgit2
is used by enforcing test failure.
Refer #399

The test expects the failure for now. But the condition must be flipped
once #437 lands in reconcilers-dev branch.

It also updates libgit2 checkout implementations to respect the context
passed to it during cloning. This is needed to shorten the wait time for
tests that depend on failure behavior and should fail early.
TestCheckout_ED25519 test would take 30s without a context with
timeout. This helps shorten the time for such tests.
This also makes the GitRepository.Spec.Timeout property effective
when using libgit2.

Some more details in the commits.

⚠️ target branch of the PR is reconcilers-dev.

select {
case err := <-errCh:
if err != nil {
return nil, fmt.Errorf("unable to clone '%s', error: %w", url, gitutil.LibGit2Error(err))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some of the checkouts didn't use LibGit2Error() with the received error. I think that was missed. Now all of the checkout implementations will have this.

Copy link
Member

@squaremo squaremo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work Sunny. I can do another review once it's ready to merge.

pkg/git/libgit2/checkout.go Outdated Show resolved Hide resolved
pkg/git/libgit2/checkout.go Outdated Show resolved Hide resolved
g := NewWithT(t)

timeout := 5 * time.Second
url := "ssh://git@github.com/projectcontour/contour"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can the test git server not be used here, rather than relying on the internet? I've used it in the image-automation-controller tests.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 for getting rid of the references to "real" Git repositories, and replacing them with mocks.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issues with mocks is that we can't tell if our code works on large repos with a consistent history. If we remove it from here please add it to e2e tests, using nodejs would be best as it has many GB in .git.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you mean by "consistent history" that's lacking in a local server?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Though yes, integration tests are the place to test "real world" scenarios. But this particular test is for correctness, so using a local server is not just fine, it's preferred, since it helps isolate the property under test.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I should've said "large history", as in hundreds of tags and thousands of commits.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That should be handled as a separate test case, most likely with a specific (Go) tag attached to it (e.g. integration), so that it only runs in CI.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've refactored the test to use a test git server.

This adds a test to detect any regression in libgit2's ED25519 key
support. go-git supports ED25519 but not the current version of
libgit2 used in flux. The updates to libgit2 in v1.2.0 adds support
for ED25519. This test would help ensure the right version of libgit2
is used.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
@darkowlzz darkowlzz force-pushed the libgit2-ed25519-check branch from cc7789e to 4f29459 Compare October 5, 2021 18:39
Update libgit2-git2go checkout with a clone wrapper to respect the
context passed to it. go-git accepts context passed to it during cloning
but the git2go does not accept the context passed to it. This makes the
GitRepository.Spec.Timeout ineffective when using libgit2.

This change updates all the different libgit2 checkouts to use the new
clone function.

This is also needed to shorten the wait time for tests that depend on
failure behavior and should fail early. TestCheckout_ED25519 test would
take 30s without a context with timeout. This helps shorten the time
for such tests.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
@darkowlzz darkowlzz force-pushed the libgit2-ed25519-check branch from 4f29459 to 5130752 Compare October 6, 2021 10:11
@darkowlzz
Copy link
Contributor Author

darkowlzz commented Oct 6, 2021

The failure in CI is due to a different libgit2 version. The new build changes aren't in reconcilers-dev branch yet.

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants