Skip to content
This repository has been archived by the owner on Oct 10, 2023. It is now read-only.

TKR Source Controller: Install TKR package contents directly #2524

Merged
merged 2 commits into from
Jun 2, 2022

Conversation

imikushin
Copy link
Contributor

@imikushin imikushin commented May 31, 2022

What this PR does / why we need it

Install TKR package contents directly

TKR package contents get automatically installed without the use of Carvel PackageInstall resource (which doesn't support shared resources across packages). This is necessary because a TKR package may contain resources that are reused by other TKR packages, e.g. different TKRs shipping the same Kubernetes version may share some OSImages or addon packages (addon packages may even be reused across K8s versions).

Inject Registry as an interface into components that need it

This improves testability and modularity of affected components, and reuse of Registry as a component. Now, both Fetcher and TKR Package Reconciler re-use the same Registry instance provided externally.

Only fetch compatible TKR BOMs and TKR packages

We're also now using compatibility information to download only compatible TKRs into the management cluster.

PatchSet improvements

We're making PatchSet more generic and efficient.

Before this change, PatchSet would not work with ConfigMaps. We're also removing a dependency on util/patch from Cluster API.

PatchSet is also now safe for concurrent use.

Which issue(s) this PR fixes

N/A

Describe testing done for PR

  • Added/updated unit tests.
  • Manual testing with an AWS management cluster.

Release note

Fixed TKR Source Controller (pkg/v2/tkr/controller/tkr-source): it now installs 
TKR package contents.

Changed TKR Source Controller (pkg/v2/tkr/controller/tkr-source): only compatible TKR BOMs and TKR packages are being fetched now.

Improved util/patchset.PatchSet: it is now safe for concurrent use and more efficient.

PR Checklist

  • Squash the commits into one or a small number of logical commits
  • Use good commit messages
  • Ensure PR contains terms all contributors can understand and links all contributors can access

Additional information

Special notes for your reviewer

@imikushin imikushin requested review from a team and prkalle as code owners May 31, 2022 20:09
@imikushin imikushin added ok-to-merge PRs should be labelled with this before merging and removed cla-not-required labels May 31, 2022
@codecov
Copy link

codecov bot commented May 31, 2022

Codecov Report

Merging #2524 (f4ea46d) into main (db9d5fb) will decrease coverage by 0.00%.
The diff coverage is 58.06%.

❗ Current head f4ea46d differs from pull request most recent head f7e6dab. Consider uploading reports for the commit f7e6dab to get more accurate results

@@            Coverage Diff             @@
##             main    #2524      +/-   ##
==========================================
- Coverage   42.32%   42.32%   -0.01%     
==========================================
  Files         398      398              
  Lines       39518    39692     +174     
==========================================
+ Hits        16726    16798      +72     
- Misses      21219    21308      +89     
- Partials     1573     1586      +13     
Impacted Files Coverage Δ
...v1/tkr/controllers/source/tkr_source_controller.go 64.35% <ø> (ø)
...g/v2/tkr/controller/tkr-source/pkgcr/reconciler.go 50.54% <49.67%> (-15.46%) ⬇️
pkg/v1/util/patchset/patchset.go 96.42% <100.00%> (-3.58%) ⬇️
pkg/v1/webhooks/certificates.go 7.20% <0.00%> (-1.21%) ⬇️
addons/controllers/clusterbootstrap_controller.go 68.21% <0.00%> (-0.99%) ⬇️
pkg/v1/webhooks/webhooks.go 0.00% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update db9d5fb...f7e6dab. Read the comment docs.

We're making PatchSet more generic and efficient.

Before this change, PatchSet would not work with ConfigMaps. We're also
removing a dependency on util/patch from Cluster API.

PatchSet is also now safe for concurrent use.

Signed-off-by: Ivan Mikushin <imikushin@vmware.com>
@imikushin imikushin force-pushed the tkr-sc-no-pkgi branch 2 times, most recently from abc34ce to 665627e Compare June 1, 2022 00:18
@imikushin imikushin added kind/enhancement Categorizes issue or PR as related to an enhancement priority/critical-urgent kind/refactor kind/cleanup Categorizes issue or PR as related to cleaning up code, process area/tkr go Pull requests that update Go code cla-not-required labels Jun 1, 2022
TKR package contents get automatically installed without the use of
Carvel PackageInstall resource (which doesn't support shared resources
across packages).

This is necessary because a TKR package may contain resources that are
reused by other TKR packages, e.g. different TKRs shipping the same
Kubernetes version may share some OSImages or addon packages
(addon packages may even be reused across K8s versions).

Inject Registry as an interface

This improves testability and modularity of affected components, and
reuse of Registry as a component. Now, both Fetcher and TKR Package
Reconciler re-use the same Registry instance provided externally.

Only fetch compatible TKR BOMs and TKR packages

We're also now using compatibility information to download only
compatible TKR packages and TKR BOMs into the management cluster.

Signed-off-by: Ivan Mikushin <imikushin@vmware.com>
Copy link
Contributor

@prkalle prkalle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks @imikushin

if err := f.createTKRPackages(ctx, tag); err != nil {
errs = append(errs, errors.Wrapf(err, "failed to create TKR Package for image %s", fmt.Sprintf("%s:%s", f.Config.TKRRepoImagePath, tag)))
}
}
if len(errs) != 0 {
return errs
}

f.Log.Info("Done fetching TKR Packages", "image", f.Config.TKRRepoImagePath)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: This log in case of errors would be misleading?

Copy link
Contributor

@vuil vuil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, thanks

@imikushin imikushin merged commit 4494221 into vmware-tanzu:main Jun 2, 2022
imikushin pushed a commit to imikushin/tanzu-framework that referenced this pull request Jun 7, 2022
Without this fix, `bom-metadata` ConfigMap in the `tkr-system` namespace
would differ in `binaryData.compatibility` value from its contents
prior to vmware-tanzu#2524.

Adding a unit test that would fail illustrating the difference:
```
  Expected
      <string>: "Manage..."
  to equal       |
      <string>: "manage..."
```

Signed-off-by: Ivan Mikushin <imikushin@vmware.com>
imikushin added a commit that referenced this pull request Jun 7, 2022
Without this fix, `bom-metadata` ConfigMap in the `tkr-system` namespace
would differ in `binaryData.compatibility` value from its contents
prior to #2524.

Adding a unit test that would fail illustrating the difference:
```
  Expected
      <string>: "Manage..."
  to equal       |
      <string>: "manage..."
```

Signed-off-by: Ivan Mikushin <imikushin@vmware.com>
ankeesler pushed a commit to ankeesler/tanzu-framework that referenced this pull request Jun 14, 2022
…zu#2559)

Without this fix, `bom-metadata` ConfigMap in the `tkr-system` namespace
would differ in `binaryData.compatibility` value from its contents
prior to vmware-tanzu#2524.

Adding a unit test that would fail illustrating the difference:
```
  Expected
      <string>: "Manage..."
  to equal       |
      <string>: "manage..."
```

Signed-off-by: Ivan Mikushin <imikushin@vmware.com>
# for free to subscribe to this conversation on GitHub. Already have an account? #.
Labels
area/tkr cla-not-required go Pull requests that update Go code kind/cleanup Categorizes issue or PR as related to cleaning up code, process kind/enhancement Categorizes issue or PR as related to an enhancement kind/refactor ok-to-merge PRs should be labelled with this before merging priority/critical-urgent
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants