Package Count doesn't match list of packages

[amouat]

What happened:

Syft returns the number of packages in an image, but this doesn't match the number of package names listed. This happens on multiple images (e.g. redis:alpine reports 19 packages, but only 18 are named) but is particularly pronounced on some Chainguard Images e.g. cgr.dev/chainguard/redis reports 34 packages but only 16 are named).

What you expected to happen:

The total number of packages reported to match the number of named packages. Or some explanation for the difference.

Steps to reproduce the issue:

 syft cgr.dev/chainguard/redis
 ✔ Loaded image                                cgr.dev/chainguard/redis:latest
 ✔ Parsed image                    sha256:9287ce7a86a6908a3ee338c036e15e0d2366
 ✔ Cataloged packages              [34 packages]
NAME                    VERSION      TYPE
bash                    5.2.15-r3    apk
busybox                 1.36.1-r2    apk
ca-certificates-bundle  20230506-r0  apk
glibc                   2.38-r5      apk
glibc-locale-posix      2.38-r5      apk
ld-linux                2.38-r5      apk
libcrypt1               2.38-r5      apk
libcrypto3              3.1.4-r1     apk
libssl3                 3.1.4-r1     apk
ncurses                 6.4-r2       apk
ncurses-terminfo-base   6.4-r2       apk
openssl-config          3.1.4-r1     apk
posix-libc-utils        2.38-r5      apk
redis-7.2               7.2.3-r0     apk
redis-cli-7.2           7.2.3-r0     apk
wolfi-baselayout        20230201-r6  apk

Anything else we need to know?:

Not sure if this a bug or a misunderstanding on my part.

Environment:

• Output of syft version:

Application: syft
Version:    0.95.0
BuildDate:  2023-11-07T16:43:20Z
GitCommit:  9b98785aab9346999a0b5e9f5e4b4e63a1b1916c
GitDescription: v0.95.0
Platform:   darwin/arm64
GoVersion:  go1.21.3
Compiler:   gc

• OS (e.g: cat /etc/os-release or similar):

MacOS

$ uname -a
Darwin Adrians-MacBook-Pro-2.local 23.0.0 Darwin Kernel Version 23.0.0: Fri Sep 15 14:41:43 PDT 2023; root:xnu-10002.1.13~1/RELEASE_ARM64_T6000 arm64

[spiffcs]

Thanks for the issue @amouat

It seems the tool has incorrect output on the length of the json as well - let me investigate and see what's happening in the UI that might be giving incorrect numbers here.

From first glance there seems to be a couple of seemingly duplicate entries being surfaced in the json output that are not as apparent in the table form:

Example: the json is surfacing two packages for bash:

  {
   "id": "f8308ae7aed9d466",
   "name": "bash",
   "version": "5.2.15-r3",
   "type": "apk",
   "foundBy": "sbom-cataloger",
   "locations": [
    {
     "path": "/var/lib/db/sbom/bash-5.2.15-r3.spdx.json",
     "layerID": "sha256:6e5a3e444977e2084ca8372179a561b3c58d362c6cca0df0cf39b64861f0060e",
     "annotations": {
      "evidence": "primary"

  {
   "id": "c7607e63df149c5b",
   "name": "bash",
   "version": "5.2.15-r3",
   "type": "apk",
   "foundBy": "apk-db-cataloger",
   "locations": [
    {
     "path": "/lib/apk/db/installed",
     "layerID": "sha256:6e5a3e444977e2084ca8372179a561b3c58d362c6cca0df0cf39b64861f0060e",
     "annotations": {
      "evidence": "primary"
     }
    }
   ],

syft -o json cgr.dev/chainguard/redis | jq '.artifacts | length'
 ✔ Loaded image                                                                cgr.dev/chainguard/redis:latest
 ✔ Parsed image                        sha256:7c1b50681cdc9190d819b8039544134b8cace20a6c91b622488b11a9382612b8
 ✔ Cataloged packages              [34 packages]
32

We might be doing some deduplication after that number is generated resulting in the final list in the json output being 32 vs the original 34

The packages discovered by the sbom-cataloger are not showing in the table format. @anchore/tools we should use thread to discuss what the correct behavior should be in this case. Do we use the distributed SBOM located under /var/lib/db/sbom for the bash entry in favor of the analysis syft does? How do we reconcile these packages? Should a merge of the packages happen that makes a single package with multiple found-by or is the duplication here representative of two forms of discovery?

[spiffcs]

Just for confirmation I ran syft with the following config (note the sbom-cataloger is ommited):

catalogers:
  - alpm-db-cataloger
  - apkdb-cataloger
  - binary-cataloger
  - cargo-auditable-binary-cataloger
  - cocoapods-cataloger
  - conan-cataloger
  - dartlang-lock-cataloger
  - dotnet-deps-cataloger
  - dpkg-db-cataloger
  - elixir-mix-lock-cataloger
  - erlang-rebar-lock-cataloger
  - go-module-file-cataloger
  - go-module-binary-cataloger
  - graalvm-native-image-cataloger
  - haskell-cataloger
  - java-cataloger
  - java-gradle-lockfile-cataloger
  - java-pom-cataloger
  - javascript-lock-cataloger
  - javascript-package-cataloger
  - linux-kernel-cataloger
  - nix-store-cataloger
  - php-composer-installed-cataloger
  - php-composer-lock-cataloger
  - portage-cataloger
  - python-package-cataloger
  - python-installed-package-cataloger
  - rpm-db-cataloger
  - rpm-archive-cataloger
  - ruby-gemfile-cataloger
  - ruby-installed-gemspec-cataloger
  - rust-cargo-lock-cataloger
  - spm-cataloger

 syft cgr.dev/chainguard/redis
 ✔ Loaded image                                                                cgr.dev/chainguard/redis:latest
 ✔ Parsed image                        sha256:7c1b50681cdc9190d819b8039544134b8cace20a6c91b622488b11a9382612b8
 ✔ Cataloged packages              [18 packages]
NAME                    VERSION      TYPE
bash                    5.2.15       binary
bash                    5.2.15-r3    apk
busybox                 1.36.1       binary
busybox                 1.36.1-r2    apk
ca-certificates-bundle  20230506-r0  apk
glibc                   2.38-r5      apk
glibc-locale-posix      2.38-r5      apk
ld-linux                2.38-r5      apk
libcrypt1               2.38-r5      apk
libcrypto3              3.1.4-r1     apk
libssl3                 3.1.4-r1     apk
ncurses                 6.4-r2       apk
ncurses-terminfo-base   6.4-r2       apk
openssl-config          3.1.4-r1     apk
posix-libc-utils        2.38-r5      apk
redis-7.2               7.2.3-r0     apk
redis-cli-7.2           7.2.3-r0     apk
wolfi-baselayout        20230201-r6  apk

Here we see the expected result so the bug lies in how syft is reconciling a discovered sbom

[amouat]

Thanks for looking at this! Note that there might still be another issue: redis:alpine seems out by one as well (I wonder if that's because of the redis binary that isn't an apk package).

[tgerla]

Hey @amouat, we made some changes to the table output summaries in Syft 1.0.0 that I believe will solve these mismatched counts. Can you give your test case another try and let us know? Thanks! (#2672 and #2679)

[amouat]

Thanks @tgerla . It works for cgr.dev/chainguard/redis now, but I then tried cgr.dev/chainguard/nginx and I get:

❯ syft cgr.dev/chainguard/nginx
 ✔ Loaded image                                                                              cgr.dev/chainguard/nginx:latest
 ✔ Parsed image                                      sha256:9b3f68c45ef8bebfc2604bb4077c0285f4b2016828ecd68b7ae513a9a4434290
 ✔ Cataloged contents                                       82cb11fedeb692168a9a36423a906bccd60c45fe34b10c7bef45a2a623886349
   ├── ✔ Packages                        [17 packages]
   ├── ✔ File digests                    [172 files]
   ├── ✔ File metadata                   [172 locations]
   └── ✔ Executables                     [34 executables]
NAME                           VERSION      TYPE
ca-certificates-bundle         20240315-r0  apk
glibc                          2.38-r14     apk
glibc-locale-posix             2.38-r14     apk
ld-linux                       2.38-r14     apk
libcrypt1                      2.38-r14     apk
libcrypto3                     3.2.1-r0     apk
libgcc                         13.2.0-r5    apk
libssl3                        3.2.1-r0     apk
libstdc++                      13.2.0-r5    apk
nginx-mainline                 1.25.4-r1    apk
nginx-mainline-config          1.25.4-r1    apk
nginx-mainline-package-config  1.25.4-r1    apk
openssl-config                 3.2.1-r0     apk
pcre                           8.45-r1      apk
wolfi-baselayout               20230201-r7  apk
zlib                           1.3.1-r0     apk

Which says there are 17 packages but only 16 are listed.

Syft version:

❯ syft version
Application: syft
Version:    1.2.0
BuildDate:  2024-04-12T18:31:58Z
GitCommit:  dde5d349b1eef740c285255e6a9e3a8f5c9938e1
GitDescription: v1.2.0
Platform:   darwin/arm64
GoVersion:  go1.21.9
Compiler:   gc

[tgerla]

Hi @amouat, sorry about that, I believe now the issue is actually fixed in Grype 0.77.2: anchore/grype#1837 -- I will close this issue but please let me know if you run into any other weird counts. Thanks!

[tgerla]

Oops -- just realized this is in Syft, not Grype. Keeping open!