Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

CycloneDX convertion into Syft improperly handles SPDX licenses #3172

Closed
NyanKiyoshi opened this issue Aug 29, 2024 · 2 comments
Closed

CycloneDX convertion into Syft improperly handles SPDX licenses #3172

NyanKiyoshi opened this issue Aug 29, 2024 · 2 comments
Labels
bug Something isn't working good-first-issue Good for newcomers

Comments

@NyanKiyoshi
Copy link
Contributor

When converting a CycloneDX JSON BOM into the Syft format, SPDX licenses are dropped.

The CycloneDX specifications define that either (one of) the following values should be set under components[].licenses:

  • CycloneDX format: [{"license": {"id": "...", "name": "..."}}, ...]
  • Or (exclusive or), SPDX format, thus only [{"expression": "..."}, ...]

But Syft expects to have the following value which is invalid in the specs: {"expression": "...", "license": {...}}.

This is caused by the lines 57 to 59 at:

syft/syft/format/internal/cyclonedxutil/helpers/licenses.go

Lines 50 to 73 in 11d77b4

func decodeLicenses(c *cyclonedx.Component) []pkg.License {
licenses := make([]pkg.License, 0)
if c == nil || c.Licenses == nil {
return licenses
}
for _, l := range *c.Licenses {
if l.License == nil {
continue
}
// these fields are mutually exclusive in the spec
switch {
case l.License.ID != "":
licenses = append(licenses, pkg.NewLicenseFromURLs(l.License.ID, l.License.URL))
case l.License.Name != "":
licenses = append(licenses, pkg.NewLicenseFromURLs(l.License.Name, l.License.URL))
case l.Expression != "":
licenses = append(licenses, pkg.NewLicenseFromURLs(l.Expression, l.License.URL))
default:
}
}
return licenses
}

What you expected to happen:

When an SPDX license is provided inside a CycloneDX component, Syft should not drop it.

Potential Solution

We could drop the if l.License == nil { continue } and change the case conditions to handle the nil case properly:

func decodeLicenses(c *cyclonedx.Component) []pkg.License {
	// [...]

	for _, l := range *c.Licenses {
		switch {
		case l.License != nil && l.License.ID != "":
			licenses = append(licenses, pkg.NewLicenseFromURLs(l.License.ID, l.License.URL))
		case l.License != nil && l.License.Name != "":
			licenses = append(licenses, pkg.NewLicenseFromURLs(l.License.Name, l.License.URL))
		case l.Expression != "":
			licenses = append(licenses, pkg.NewLicense(l.Expression))
		default:
		}
	}

	return licenses
}

Diffs:

--- a/licenses.go
+++ b/licenses.b.go
@@ -54,17 +54,13 @@ func decodeLicenses(c *cyclonedx.Component) []pkg.License {
        }
 
        for _, l := range *c.Licenses {
-               if l.License == nil {
-                       continue
-               }
-               // these fields are mutually exclusive in the spec
                switch {
-               case l.License.ID != "":
+               case l.License != nil && l.License.ID != "":
                        licenses = append(licenses, pkg.NewLicenseFromURLs(l.License.ID, l.License.URL))
-               case l.License.Name != "":
+               case l.License != nil && l.License.Name != "":
                        licenses = append(licenses, pkg.NewLicenseFromURLs(l.License.Name, l.License.URL))
                case l.Expression != "":
-                       licenses = append(licenses, pkg.NewLicenseFromURLs(l.Expression, l.License.URL))
+                       licenses = append(licenses, pkg.NewLicense(l.Expression))
                default:
                }
        }

Steps to reproduce the issue:

Prerequisites

Create two files into a directory:

  1. valid.json (what Syft doesn't handle properly) [download] 
    {
  "bomFormat": "CycloneDX",
  "specVersion": "1.5",
  "serialNumber": "urn:uuid:bc59a2de-780c-40ae-9316-4ad1287f1bb2",
  "version": 1,
  "metadata": {
    "timestamp": "2024-08-29T12:36:56Z",
    "tools": {
      "components": [
        {
          "group": "@cyclonedx",
          "name": "cdxgen",
          "version": "10.9.5",
          "purl": "pkg:npm/%40cyclonedx/cdxgen@10.9.5",
          "type": "application",
          "bom-ref": "pkg:npm/@cyclonedx/cdxgen@10.9.5",
          "author": "OWASP Foundation",
          "publisher": "OWASP Foundation"
        }
      ]
    },
    "authors": [
      {
        "name": "OWASP Foundation"
      }
    ],
    "lifecycles": [
      {
        "phase": "build"
      }
    ],
    "component": {
      "group": "",
      "name": "app",
      "version": "latest",
      "type": "application",
      "bom-ref": "pkg:gem/app@latest",
      "purl": "pkg:gem/app@latest"
    },
    "properties": [
      {
        "name": "cdx:bom:componentTypes",
        "value": "pypi"
      }
    ]
  },
  "components": [
    {
      "author": "\"Jeffrey A. Clark\" <aclark@aclark.net>",
      "group": "",
      "name": "pillow",
      "version": "10.4.0",
      "description": "Python Imaging Library (Fork)",
      "hashes": [
        {
          "alg": "SHA-256",
          "content": "4d9667937cfa347525b319ae34375c37b9ee6b525440f3ef48542fcf66f2731e"
        }
      ],
      "licenses": [
        {
          "expression": "Historical Permission Notice and Disclaimer (HPND)"
        }
      ],
      "purl": "pkg:pypi/pillow@10.4.0",
      "type": "library",
      "bom-ref": "pkg:pypi/pillow@10.4.0",
      "evidence": {
        "identity": {
          "field": "purl",
          "confidence": 1,
          "methods": [
            {
              "technique": "instrumentation",
              "confidence": 1,
              "value": "/tmp/cdxgen-venv-St5Cu7"
            }
          ]
        }
      },
      "properties": [
        {
          "name": "SrcFile",
          "value": "/app/requirements.txt"
        }
      ]
    }
  ],
  "services": [],
  "dependencies": [
    {
      "ref": "pkg:pypi/app@latest",
      "dependsOn": []
    },
    {
      "ref": "pkg:pypi/pillow@10.4.0",
      "dependsOn": []
    }
  ]
}
  2. invalid.json (what Syft expects) [download] 
    {
  "bomFormat": "CycloneDX",
  "specVersion": "1.5",
  "serialNumber": "urn:uuid:bc59a2de-780c-40ae-9316-4ad1287f1bb2",
  "version": 1,
  "metadata": {
    "timestamp": "2024-08-29T12:36:56Z",
    "tools": {
      "components": [
        {
          "group": "@cyclonedx",
          "name": "cdxgen",
          "version": "10.9.5",
          "purl": "pkg:npm/%40cyclonedx/cdxgen@10.9.5",
          "type": "application",
          "bom-ref": "pkg:npm/@cyclonedx/cdxgen@10.9.5",
          "author": "OWASP Foundation",
          "publisher": "OWASP Foundation"
        }
      ]
    },
    "authors": [
      {
        "name": "OWASP Foundation"
      }
    ],
    "lifecycles": [
      {
        "phase": "build"
      }
    ],
    "component": {
      "group": "",
      "name": "app",
      "version": "latest",
      "type": "application",
      "bom-ref": "pkg:gem/app@latest",
      "purl": "pkg:gem/app@latest"
    },
    "properties": [
      {
        "name": "cdx:bom:componentTypes",
        "value": "pypi"
      }
    ]
  },
  "components": [
    {
      "author": "\"Jeffrey A. Clark\" <aclark@aclark.net>",
      "group": "",
      "name": "pillow",
      "version": "10.4.0",
      "description": "Python Imaging Library (Fork)",
      "hashes": [
        {
          "alg": "SHA-256",
          "content": "4d9667937cfa347525b319ae34375c37b9ee6b525440f3ef48542fcf66f2731e"
        }
      ],
      "licenses": [
        {
          "expression": "Historical Permission Notice and Disclaimer (HPND)",
          "license": {
            "id": "HPND",
            "url": "https://opensource.org/license/historical-php"
          }
        }
      ],
      "purl": "pkg:pypi/pillow@10.4.0",
      "type": "library",
      "bom-ref": "pkg:pypi/pillow@10.4.0",
      "evidence": {
        "identity": {
          "field": "purl",
          "confidence": 1,
          "methods": [
            {
              "technique": "instrumentation",
              "confidence": 1,
              "value": "/tmp/cdxgen-venv-St5Cu7"
            }
          ]
        }
      },
      "properties": [
        {
          "name": "SrcFile",
          "value": "/app/requirements.txt"
        }
      ]
    }
  ],
  "services": [],
  "dependencies": [
    {
      "ref": "pkg:pypi/app@latest",
      "dependsOn": []
    },
    {
      "ref": "pkg:pypi/pillow@10.4.0",
      "dependsOn": []
    }
  ]
}

Differences between valid.json and invalid.json:

# diff -u valid.json invalid.json 
--- valid.json  2024-08-29 14:46:59
+++ invalid.json        2024-08-29 14:45:56
@@ -59,7 +59,11 @@
       ],
       "licenses": [
         {
-          "expression": "Historical Permission Notice and Disclaimer (HPND)"
+          "expression": "Historical Permission Notice and Disclaimer (HPND)",
+          "license": {
+            "id": "HPND",
+            "url": "https://opensource.org/license/historical-php"
+          }
         }
       ],
       "purl": "pkg:pypi/pillow@10.4.0",

JSON schema validation showing that it is indeed invalid (using https://github.com/CycloneDX/sbom-utility/releases/tag/v0.16.0):

$ sbom-utility validate --input-file valid.json --quiet # OK
$ sbom-utility validate --input-file invalid.json --quiet # Not OK
1. {
        "type": "number_one_of",
        "field": "components.0.licenses",
        "context": "(root).components.0.licenses",
        "description": "Must validate one and only one schema (oneOf)",
        "value": [
            {
                "expression": "Historical Permission Notice and Disclaimer (HPND)",
                "license": {
                    "id": "HPND",
                    "url": "https://opensource.org/license/historical-php"
                }
            }
        ]
    }
2. {
        "type": "additional_property_not_allowed",
        "field": "components.0.licenses.0",
        "context": "(root).components.0.licenses.0",
        "description": "Additional property expression is not allowed",
        "value": "Historical Permission Notice and Disclaimer (HPND)"
    }

Steps

  1. Convert CycloneDX JSON BOM into Syft JSON format:

    docker run --rm -v "$(pwd)":/app/ -ti ghcr.io/anchore/syft:v1.11.1 convert /app/valid.json --quiet -o syft-json > syft-format.json

  2. No licenses should be present in the Syft JSON format (the bug):

    $ jq . syft-format.json
{
  "artifacts": [
    [...]
    {
      "id": "b2e94cea47d09252",
      "name": "pillow",
      "version": "10.4.0",
      "type": "python",
      "foundBy": "",
      "locations": null,
      "licenses": [],  /// <---- The issue
      "language": "python",
      "cpes": [],
      "purl": "pkg:pypi/pillow@10.4.0"
    }
  ],
  [...]
  "schema": {
    "version": "16.0.15",
    "url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-16.0.15.json"
  }
}

  3. Convert the invalid CycloneDX JSON BOM into the Syft format:

    docker run --rm -v "$(pwd)":/app/ -ti ghcr.io/anchore/syft:v1.11.1 convert /app/invalid.json --quiet -o syft-json > syft-format.json

  4. The license should now be present despite being invalid:

    $ jq . syft-format.json
{
  "artifacts": [
    [...]
    {
      "id": "6eb502d35e5c15e7",
      "name": "pillow",
      "version": "10.4.0",
      "type": "python",
      "foundBy": "",
      "locations": null,
      "licenses": [
        {
          "value": "HPND",  <---- It's now here
          "spdxExpression": "HPND",
          "type": "declared",
          "urls": [
            "https://opensource.org/license/historical-php"
          ],
          "locations": []
        }
      ],
      "language": "python",
      "cpes": [],
      "purl": "pkg:pypi/pillow@10.4.0"
    }
  ],
  "artifactRelationships": [],
  [...]
  "schema": {
    "version": "16.0.15",
    "url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-16.0.15.json"
  }
}

By mitigating the issue (see the 'Potential Solution' section), we get the expected result:

$ syft convert valid.json --quiet -o syft-json > syft-format.json
$ jq . syft-format.json
{
  "artifacts": [
    [...]
    {
      "id": "ceefc6aa54ba5e1c",
      "name": "pillow",
      "version": "10.4.0",
      "type": "python",
      "foundBy": "",
      "locations": null,
      "licenses": [
        {
          "value": "Historical Permission Notice and Disclaimer (HPND)", // <---- This is correct now
          "spdxExpression": "",
          "type": "declared",
          "urls": [],
          "locations": []
        }
      ],
      "language": "python",
      "cpes": [],
      "purl": "pkg:pypi/pillow@10.4.0"
    }
  ],
   [...]
  "schema": {
    "version": "16.0.15",
    "url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-16.0.15.json"
  }
}

Anything else we need to know?

The issue was found in https://github.com/anchore/grant, it causes the grant to be flag some dependencies as having no license (version: 4362dc2.

  • Unmitigated:

    $ grant check /app/bom2.json -o table --show-packages
* /app/bom2.json
  * No License Violations Found for Rule default-deny-all
  * packages found with no licenses
    * app
    * pillow <--- Bug

  • Mitigated ('Potential Solution' section), it works properly:

    $ grant check /app/bom2.json -o table --show-packages
* /app/bom2.json
  * license matches for rule: default-deny-all; matched with pattern *
    * Historical Permission Notice and Disclaimer (HPND)  <--- Works!
      * pillow
  * packages found with no licenses
    * app
check failed

Environment:

  • Output of syft version: 
    Application: syft
Version:    1.11.1
BuildDate:  2024-08-20T16:25:20Z
GitCommit:  95b4a88256bddebb91831250f28f602f8c36552a
GitDescription: v1.11.1
Platform:   linux/amd64
GoVersion:  go1.22.6
Compiler:   gc
  • OS: Mac OS 14.6.1 (Sonoma)
@NyanKiyoshi NyanKiyoshi added the bug Something isn't working label Aug 29, 2024
@kzantow kzantow moved this to Ready in OSS Aug 29, 2024
@kzantow
Copy link
Contributor

kzantow commented Aug 29, 2024

Thanks for the detailed report, @NyanKiyoshi ! I've added this to the ready issues, but we would definitely welcome a PR, too.

@kzantow kzantow added the good-first-issue Good for newcomers label Aug 29, 2024
NyanKiyoshi added a commit to NyanKiyoshi/syft that referenced this issue Aug 29, 2024
@NyanKiyoshi
fix: properly decode SPDX license expressions in CycloneDX format
fb85c70 
This fixes the issue reported at anchore#3172,
where Syft would drop SPDX licenses due a logic error in the decoder.

CycloneDX specifications require `components[].licenses[].license` to be nil
when `components[].licenses[].expression` (SPDX) is non nil.

Signed-off-by: Mikail Kocak <mikail-gh@pm.me>
@NyanKiyoshi NyanKiyoshi mentioned this issue Aug 29, 2024
Merged
@NyanKiyoshi
Copy link
Contributor Author

Resolved via #3175

@github-project-automation github-project-automation bot moved this from Ready to Done in OSS Aug 29, 2024
@BrewTestBot BrewTestBot mentioned this issue Sep 11, 2024
Merged
# for free to join this conversation on GitHub. Already have an account? # to comment
Assignees
No one assigned
Labels
bug Something isn't working good-first-issue Good for newcomers
Projects
OSS
Archived in project
Milestone
No milestone
Development

No branches or pull requests
2 participants
@kzantow @NyanKiyoshi