Syft tries to create the cache directory at a location that has no permission #2984

Joerki · 2024-06-22T04:54:51Z

What happened:
Hi @kzantow,

the new cache implementation is partially not working. Without changing my previous configuration (#2798). If, in a Docker container syft tries to create the cache subdirectory at an unusual place (root dir, $HOME is missing), this operation failes and syft crashes later on.

When I add -e HOME=/tmp to the docker command line it does not crash.

[0000] WARN unable to get filesystem cache at /.cache/syft: unable to create directory at '/.cache/syft//.cache/syft': mkdir /.cache: permission denied

What you expected to happen:
Syft should check writing permissions of the directory where cache shall be created and reports the problem with explicit directory when this operation fails.

Steps to reproduce the issue:
Set HOME to a directory without write permissions.
Scan an Docker image with Go binaries (like mongo)
Enable settings to fetch packages to identify licenses (see settings below).

I created a Docker image with syft binary to use it in differend environments. In the case that HOME is "/", syft crashes.
If I change HOME to "/" in my usual, local environment (WSL), syft is reporting the warning, but does not crash.

Anything else we need to know?:

[0000]  WARN unable to get filesystem cache at /.cache/syft: unable to create directory at '/.cache/syft//.cache/syft': mkdir /.cache: permission denied
[0000]  WARN unable to get filesystem cache at /.cache/syft: unable to create directory at '/.cache/syft//.cache/syft': mkdir /.cache: permission denied
[0000]  INFO syft version: 1.7.0
[0000] DEBUG config:
  log:
      quiet: false
      level: debug
      file: ""
  dev:
      profile: none
  config: .syft.yaml
  output:
      - syft-table
  format:
      pretty: null
      template:
          path: ""
          legacy: false
      json:
          legacy: false
          pretty: true
      spdx-json:
          pretty: false
      cyclonedx-json:
          pretty: false
      cyclonedx-xml:
          pretty: false
  check-for-app-update: false
  default-catalogers: []
  select-catalogers: []
  package:
      search-unindexed-archives: false
      search-indexed-archives: true
      exclude-binary-overlap-by-ownership: true
  file:
      metadata:
          selection: owned-by-package
          digests:
              - sha1
              - sha256
      content:
          skip-files-above-size: 256000
          globs: []
      executable:
          globs: []
  scope: all-layers
  parallelism: 1
  relationships:
      package-file-ownership: true
      package-file-ownership-overlap: true
  golang:
      search-local-mod-cache-licenses: true
      local-mod-cache-dir: /go/pkg/mod
      search-remote-licenses: true
      proxy: https://proxy.golang.org,direct
      no-proxy: ""
      main-module-version:
          from-ld-flags: true
          from-contents: true
          from-build-settings: true
  java:
      use-network: false
      maven-url: ""
      max-parent-recursive-depth: 0
  javascript:
      search-remote-licenses: false
      npm-base-url: ""
  linux-kernel:
      catalog-modules: true
  python:
      guess-unpinned-requirements: false
  registry:
      insecure-skip-tls-verify: false
      insecure-use-http: false
      auth: []
      ca-cert: ""
  from: []
  platform: ""
  source:
      name: ""
      version: ""
      base-path: ""
      file:
          digests:
              - SHA-256
      image:
          default-pull-source: ""
  exclude: []
  cache:
      dir: /.cache/syft
      ttl: 7d

failed to run tasks: 1 error occurred:
        * failed to run task: runtime error: invalid memory address or nil pointer dereference at:
goroutine 14 [running]:
runtime/debug.Stack()
        /opt/hostedtoolcache/go/1.22.4/x64/src/runtime/debug/stack.go:24 +0x5e
github.com/anchore/syft/internal/task.runTaskSafely.func1()
        /home/runner/work/syft/syft/internal/task/executor.go:67 +0x3d
panic({0x174b720?, 0x2d5a080?})
        /opt/hostedtoolcache/go/1.22.4/x64/src/runtime/panic.go:770 +0x132
github.com/anchore/syft/syft/pkg/cataloger/golang.(*goLicenseResolver).findLicensesInFS.func1({0x1f5fc08, 0x1}, {0x0?, 0x0?}, {0xc0001036c0?, 0x38?})
        /home/runner/work/syft/syft/syft/pkg/cataloger/golang/licenses.go:142 +0x97
io/fs.WalkDir({0x1f6e9a0, 0xc005c40c20}, {0x1f5fc08, 0x1}, 0xc0024dc948)
        /opt/hostedtoolcache/go/1.22.4/x64/src/io/fs/walk.go:120 +0xcc
github.com/anchore/syft/syft/pkg/cataloger/golang.(*goLicenseResolver).findLicensesInFS(0x1f6e740?, {0xc00430ad00?, 0xc003992ba0?}, {0x1f6e9a0?, 0xc005c40c20?})
        /home/runner/work/syft/syft/syft/pkg/cataloger/golang/licenses.go:141 +0x9c
github.com/anchore/syft/syft/pkg/cataloger/golang.(*goLicenseResolver).getLicensesFromLocal(0xc0023e94a0, {0xc003c6b85c?, 0xc000126720?}, {0xc003c6b876?, 0x30?})
        /home/runner/work/syft/syft/syft/pkg/cataloger/golang/licenses.go:123 +0xe5
github.com/anchore/syft/syft/pkg/cataloger/golang.(*goLicenseResolver).getLicenses(0xc0023e94a0, {0x1f88300, 0xc000126720}, {0xc003c6b85c, 0x19}, {0xc003c6b876, 0x6})
        /home/runner/work/syft/syft/syft/pkg/cataloger/golang/licenses.go:93 +0x13b
github.com/anchore/syft/syft/pkg/cataloger/golang.(*goBinaryCataloger).newGoBinaryPackage(_, {_, _}, _, {_, _}, {_, _}, {0xc002fdd566, 0x6}, ...)
        /home/runner/work/syft/syft/syft/pkg/cataloger/golang/package.go:18 +0xa6
github.com/anchore/syft/syft/pkg/cataloger/golang.(*goBinaryCataloger).buildGoPkgInfo(0xc0023e94a0, {0x1f88300, 0xc000126720}, {{{{0xc001419128, 0x11}, {0xc000b214a0, 0x47}}, {0xc0059c0cf0, 0x11}, {0x1102, ...}}, ...}, ...)
        /home/runner/work/syft/syft/syft/pkg/cataloger/golang/parse_go_binary.go:117 +0x409
github.com/anchore/syft/syft/pkg/cataloger/golang.(*goBinaryCataloger).parseGoBinary(0xc0023e94a0, {0x3e8?, 0x3e8?}, {0x1f88300, 0xc000126720}, 0x17?, {{{{{...}, {...}}, {0xc0059c0cf0, 0x11}, ...}, ...}, ...})
        /home/runner/work/syft/syft/syft/pkg/cataloger/golang/parse_go_binary.go:75 +0x2a5
github.com/anchore/syft/syft/pkg/cataloger/generic.invokeParser({0x1f7f060, 0xc0003eeaf0}, {0x1f88300, 0xc000126720}, {{{{0xc001419128, 0x11}, {0xc000b214a0, 0x47}}, {0xc0059c0cf0, 0x11}, ...}, ...}, ...)
        /home/runner/work/syft/syft/syft/pkg/cataloger/generic/cataloger.go:197 +0x43e
github.com/anchore/syft/syft/pkg/cataloger/generic.(*Cataloger).Catalog(0xc000124f00, {0x1f7f060, 0xc0003eeaf0}, {0x1f88300, 0xc000126720})
        /home/runner/work/syft/syft/syft/pkg/cataloger/generic/cataloger.go:167 +0x4f8
github.com/anchore/syft/internal/task.NewPackageTask.func1({0x1f7f060, 0xc0003eeaf0}, {0x1f88300, 0xc000126720}, {0x1f7f990, 0xc0024281e0})
        /home/runner/work/syft/syft/internal/task/package_task_factory.go:103 +0x267
github.com/anchore/syft/internal/task.task.Execute(...)
        /home/runner/work/syft/syft/internal/task/task.go:64
github.com/anchore/syft/internal/task.runTaskSafely({0x1f7f060?, 0xc0003eeaf0?}, {0x1f77ea8?, 0xc000126080?}, {0x1f88300?, 0xc000126720?}, {0x1f7f990?, 0xc0024281e0?})
        /home/runner/work/syft/syft/internal/task/executor.go:71 +0xa7
github.com/anchore/syft/internal/task.(*Executor).Execute.func1()
        /home/runner/work/syft/syft/internal/task/executor.go:49 +0x131
created by github.com/anchore/syft/internal/task.(*Executor).Execute in goroutine 37
        /home/runner/work/syft/syft/internal/task/executor.go:40 +0xa5

Environment:

Output of syft version:
It is newest Syft version 1.7.0
OS (e.g: cat /etc/os-release or similar):
Linux, on Debian WSL also in self-created Docker container based on Debian image.

The text was updated successfully, but these errors were encountered:

Joerki · 2024-06-23T05:33:11Z

It is also crashing on windows

[0000]  INFO syft version: 1.7.0
[0000] DEBUG config:
  log:
      quiet: false
      level: debug
      file: ""
  dev:
      profile: none
  config: .syft.yaml
  output:
      - syft-json=mongo.json
  format:
      pretty: null
      template:
          path: ""
          legacy: false
      json:
          legacy: false
          pretty: true
      spdx-json:
          pretty: false
      cyclonedx-json:
          pretty: false
      cyclonedx-xml:
          pretty: false
  check-for-app-update: false
  default-catalogers: []
  select-catalogers: []
  package:
      search-unindexed-archives: false
      search-indexed-archives: true
      exclude-binary-overlap-by-ownership: true
  file:
      metadata:
          selection: owned-by-package
          digests:
              - sha1
              - sha256
      content:
          skip-files-above-size: 256000
          globs: []
      executable:
          globs: []
  scope: all-layers
  parallelism: 1
  relationships:
      package-file-ownership: true
      package-file-ownership-overlap: true
  golang:
      search-local-mod-cache-licenses: true
      local-mod-cache-dir: C:\Users\w010701\go\pkg\mod
      search-remote-licenses: true
      proxy: https://proxy.golang.org,direct
      no-proxy: ""
      main-module-version:
          from-ld-flags: true
          from-contents: true
          from-build-settings: true
  java:
      use-network: false
      maven-url: ""
      max-parent-recursive-depth: 0
  javascript:
      search-remote-licenses: false
      npm-base-url: ""
  linux-kernel:
      catalog-modules: true
  python:
      guess-unpinned-requirements: false
  registry:
      insecure-skip-tls-verify: false
      insecure-use-http: false
      auth: []
      ca-cert: ""
  from: []
  platform: ""
  source:
      name: ""
      version: ""
      base-path: ""
      file:
          digests:
              - SHA-256
      image:
          default-pull-source: ""
  exclude: []
  cache:
      dir: C:\Users\abcdef\AppData\Local\cache\syft
      ttl: 7d

Crashing on Debian WSL with this configuration:

joerg@DE12914:~$ syft scan amlac02.azurecr.io/rel/mongo:1.1.1 -o syft-json=mongo.json -vv
[0000]  INFO syft version: 1.7.0
[0000] DEBUG config:
  log:
      quiet: false
      level: debug
      file: ""
  dev:
      profile: none
  config: .syft.yaml
  output:
      - syft-json=mongo.json
  format:
      pretty: null
      template:
          path: ""
          legacy: false
      json:
          legacy: false
          pretty: true
      spdx-json:
          pretty: false
      cyclonedx-json:
          pretty: false
      cyclonedx-xml:
          pretty: false
  check-for-app-update: false
  default-catalogers: []
  select-catalogers: []
  package:
      search-unindexed-archives: false
      search-indexed-archives: true
      exclude-binary-overlap-by-ownership: true
  file:
      metadata:
          selection: owned-by-package
          digests:
              - sha1
              - sha256
      content:
          skip-files-above-size: 256000
          globs: []
      executable:
          globs: []
  scope: all-layers
  parallelism: 1
  relationships:
      package-file-ownership: true
      package-file-ownership-overlap: true
  golang:
      search-local-mod-cache-licenses: true
      local-mod-cache-dir: /home/joerg/go/pkg/mod
      search-remote-licenses: true
      proxy: https://proxy.golang.org,direct
      no-proxy: ""
      main-module-version:
          from-ld-flags: true
          from-contents: true
          from-build-settings: true
  java:
      use-network: false
      maven-url: ""
      max-parent-recursive-depth: 0
  javascript:
      search-remote-licenses: false
      npm-base-url: ""
  linux-kernel:
      catalog-modules: true
  python:
      guess-unpinned-requirements: false
  registry:
      insecure-skip-tls-verify: false
      insecure-use-http: false
      auth: []
      ca-cert: ""
  from: []
  platform: ""
  source:
      name: ""
      version: ""
      base-path: ""
      file:
          digests:
              - SHA-256
      image:
          default-pull-source: ""
  exclude: []
  cache:
      dir: /home/joerg/.cache/syft
      ttl: 7d

Joerki · 2024-06-23T07:49:19Z

Hi @kzantow,

thanks for your changes. I'll try to build and check.

I also worked on pure Linux machine (before I saw your commit) to check and doing change of settings, and found out:

When I have search-local-mod-cache-licenses setting to true, I get a crash.

golang:
search-local-mod-cache-licenses: true

In case it's false, program works as usual.

kzantow · 2024-06-24T15:58:59Z

Hi @Joerki -- I think I've addressed the issues you've raised here, where invalid directories would result in panics rather than being handled properly. Apologies for that, this seems like a case that probably should have been accounted for from the get-go. Please let us know if the latest release continues to give you issues!

Joerki added the bug Something isn't working label Jun 22, 2024

anchoretoolsops added this to OSS Jun 22, 2024

Joerki changed the title ~~Syft tries to create the cache directory at an unusual place and crashes~~ If Syft tries to create the cache directory at a location with no permission it crashes Jun 23, 2024

Joerki changed the title ~~If Syft tries to create the cache directory at a location with no permission it crashes~~ Syft tries to create the cache directory at a location that has no permission Jun 23, 2024

kzantow mentioned this issue Jun 23, 2024

fix: handle errors reading go licenses #2985

Merged

kzantow closed this as completed in #2985 Jun 24, 2024

github-project-automation bot moved this to Done in OSS Jun 24, 2024

BrewTestBot mentioned this issue Jun 24, 2024

syft 1.8.0 Homebrew/homebrew-core#175493

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Syft tries to create the cache directory at a location that has no permission #2984

Syft tries to create the cache directory at a location that has no permission #2984

Joerki commented Jun 22, 2024 •

edited

Loading

Joerki commented Jun 23, 2024 •

edited

Loading

Joerki commented Jun 23, 2024

kzantow commented Jun 24, 2024 •

edited

Loading

Syft tries to create the cache directory at a location that has no permission #2984

Syft tries to create the cache directory at a location that has no permission #2984

Comments

Joerki commented Jun 22, 2024 • edited Loading

Joerki commented Jun 23, 2024 • edited Loading

Joerki commented Jun 23, 2024

kzantow commented Jun 24, 2024 • edited Loading

Joerki commented Jun 22, 2024 •

edited

Loading

Joerki commented Jun 23, 2024 •

edited

Loading

kzantow commented Jun 24, 2024 •

edited

Loading