Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

Support scanning license files in golang packages over the network #1630

Merged
merged 22 commits into from
Apr 14, 2023
Merged

Support scanning license files in golang packages over the network #1630

merged 22 commits into from
Apr 14, 2023

Conversation

deitch
Copy link
Contributor

@deitch deitch commented Feb 28, 2023

Fixes #1056

As discussed with @kzantow

This checks in $GOPATH/pkg/ for a package@version, and, if it finds it, reads the licenses from there. If not found, it falls back to reading the package from the internet via the official go proxy https://proxy.golang.org

In addition, it has two new CLI flags: go-fetch bool and --go-proxy string. The fetch is supposed to enable fetching over the Internet, default to false; the proxy is supposed to override the default.

The CLI flags are not wired up, as I could not quite figure out how to pass them all the way through to the go cataloger. I managed to get them onto the app *config.Application, but there is a bunch of steps down. Some pointers would be helpful.

@deitch deitch force-pushed the golang-licenses branch 4 times, most recently from a601f15 to da16e28 Compare March 2, 2023 20:52
@deitch
Copy link
Contributor Author

deitch commented Mar 2, 2023

I do not understand why the CLI tests fail. 🤷‍♂️

@deitch deitch mentioned this pull request Mar 3, 2023
Merged
@deitch
Copy link
Contributor Author

deitch commented Mar 3, 2023

At the request of @kzantow , I am splitting this one up into 2 PRs.

The first #1645 , only checks local GOPATH/mod for packages to find licenses. After that one is in, we will open a second PR, adding a CLI option to reach out to the Internet if a package is not found locally.

@deitch deitch force-pushed the golang-licenses branch 3 times, most recently from 2a56d83 to 7df5d43 Compare March 10, 2023 07:52
@deitch deitch changed the title support for scanning license files in golang packages support for scanning license files in golang packages on the Internet Mar 23, 2023
@deitch
Copy link
Contributor Author

deitch commented Mar 23, 2023

I rebased this on main after #1645 merged in. This needs the option added to enable finding modules on the Internet, but should be useful as a basis.

@deitch
support for scanning license files in golang packages
4deef4e 
Signed-off-by: Avi Deitcher <avi@deitcher.net>
@deitch
Copy link
Contributor Author

deitch commented Mar 23, 2023

Actually, I mostly managed to get it in place. Still needs some help.

@kzantow
chore: update remote license handling
ee6514d 
Signed-off-by: Keith Zantow <kzantow@gmail.com>
@kzantow kzantow changed the title support for scanning license files in golang packages on the Internet Support scanning license files in golang packages over the network Mar 23, 2023
kzantow
kzantow reviewed Mar 23, 2023
View reviewed changes
README.md Show resolved Hide resolved
wagoodman
wagoodman reviewed Mar 24, 2023
View reviewed changes
README.md Outdated Show resolved Hide resolved
@kzantow @deitch
chore: undo go mod change
339fb1d 
Signed-off-by: Keith Zantow <kzantow@gmail.com>
Signed-off-by: Avi Deitcher <avi@deitcher.net>
wagoodman
wagoodman reviewed Apr 13, 2023
View reviewed changes
syft/event/event.go Outdated Show resolved Hide resolved
@deitch
update GolangCataloger opts
acc4283 
Signed-off-by: Avi Deitcher <avi@deitcher.net>
kzantow added 6 commits April 14, 2023 10:15
@kzantow
chore: fix proxy zip extraction and address PR feedback
c1b0e55 
Signed-off-by: Keith Zantow <kzantow@gmail.com>
@kzantow
Merge remote-tracking branch 'upstream/main' into golang-licenses
8b83da7
@kzantow
chore: update to main and PR feedback
f172380 
Signed-off-by: Keith Zantow <kzantow@gmail.com>
@kzantow
chore: PR feedback
98ea39a 
Signed-off-by: Keith Zantow <kzantow@gmail.com>
@kzantow
chore: more updates based on main changes
56390c6 
Signed-off-by: Keith Zantow <kzantow@gmail.com>
@kzantow
chore: dead code
5243916 
Signed-off-by: Keith Zantow <kzantow@gmail.com>
wagoodman
wagoodman approved these changes Apr 14, 2023
View reviewed changes
Copy link
Contributor

@wagoodman wagoodman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

approving in advance assuming the NopResolver is implemented. Nice work @deitch @kzantow !

kzantow and others added 2 commits April 14, 2023 12:48
@kzantow
chore: add empty resolver
20e09fc 
Signed-off-by: Keith Zantow <kzantow@gmail.com>
@wagoodman
Merge remote-tracking branch 'origin/main' into golang-licenses
2a17e89 
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
@kzantow kzantow merged commit b692595 into anchore:main Apr 14, 2023
@deitch deitch deleted the golang-licenses branch April 15, 2023 18:18
@deitch
Copy link
Contributor Author

deitch commented Apr 15, 2023

🥳

spiffcs added a commit that referenced this pull request Apr 17, 2023
@spiffcs
Merge branch 'main' into 1577-license-revamp
cd9a81a 
* main: (35 commits)
  Fix kernel cataloger test fixtures (#1742)
  feat: Support scanning license files in golang packages over the network (#1630)
  Add package-to-file location evidence relationships (#1698)
  Add Linux Kernel cataloger (#1694)
  Add annotations for evidence on package locations (#1723)
  add format make target (#1733)
  Update tests to not fail on Mac M1's. (#1730)
  chore(deps): update bootstrap tools to latest versions (#1728)
  Add support for nar files. (#1727)
  add highlevel details about catalogers (#1726)
  chore(deps): bump golang.org/x/net from 0.8.0 to 0.9.0 (#1722)
  chore(deps): update stereoscope to e95d60a265e384df29b7a139f5c5402d6ad72e06 (#1721)
  feat: gradle lockfile support (#1719)
  chore(deps): bump github.com/docker/docker (#1715)
  chore(deps): bump golang.org/x/mod from 0.9.0 to 0.10.0 (#1713)
  chore(deps): bump golang.org/x/term from 0.6.0 to 0.7.0 (#1714)
  chore(deps): bump github.com/spf13/cobra from 1.6.1 to 1.7.0 (#1716)
  chore(deps): bump peter-evans/create-pull-request from 4 to 5 (#1712)
  chore: update tools-golang to v0.5.0 (#1717)
  Add Nix cataloger (#1696)
  ...

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
This was referenced Apr 18, 2023
Closed
Closed
Closed
Closed
Closed
Closed
Merged
Closed
Closed
GijsCalis pushed a commit to GijsCalis/syft that referenced this pull request Feb 19, 2024
@deitch @kzantow @wagoodman
feat: Support scanning license files in golang packages over the netw…
a81f237 
…ork (anchore#1630)

Signed-off-by: Avi Deitcher <avi@deitcher.net>
Signed-off-by: Keith Zantow <kzantow@gmail.com>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Co-authored-by: Keith Zantow <kzantow@gmail.com>
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
# for free to join this conversation on GitHub. Already have an account? # to comment
Reviewers

@kzantow kzantow kzantow left review comments

@wagoodman wagoodman wagoodman approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Unable to identify license on Golang packages imported by URL
3 participants
@deitch @kzantow @wagoodman