Skip to content

macOS podman build fails with Dockerfile not in context #13561

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Closed
ghost opened this issue Mar 18, 2022 · 4 comments
Closed

macOS podman build fails with Dockerfile not in context #13561

ghost opened this issue Mar 18, 2022 · 4 comments
Labels
kind/bug Categorizes issue or PR as related to a bug. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. remote Problem is in podman-remote

Comments

@ghost
Copy link

ghost commented Mar 18, 2022

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

On OSX (darwin) running podman build with the -f pointing to a file that's outside the context directory tree, the build fails with no such file or directory error. This issue also affects building from stdin (-f-) for the same reason (since stdin is written to a tmp file)

Steps to reproduce the issue:

mkdir podman-file-out-of-context
cd podman-file-out-of-context
printf 'FROM ubi8/ubi-minimal:latest\n\nRUN echo "test"\n' > Dockerfile
mkdir context_dir
podman build -f Dockerfile context_dir
# OR
cat Dockerfile | podman build -f- context_dir

Describe the results you received:

Error: stat /var/tmp/libpod_builder041882528/build/Users/cameron/dev/podman-file-out-of-context/Dockerfile: no such file or directory

Describe the results you expected:

The image to build

Additional information you deem important (e.g. issue happens only occasionally):

Output of podman version:

podman version 4.0.2

Output of podman info --debug:

host:
  arch: amd64
  buildahVersion: 1.24.1
  cgroupControllers:
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.1.0-2.fc35.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.0, commit: '
  cpus: 2
  distribution:
    distribution: fedora
    variant: coreos
    version: "35"
  eventLogger: journald
  hostname: localhost.localdomain
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 1000000
    uidmap:
    - container_id: 0
      host_id: 501
      size: 1
    - container_id: 1
      host_id: 100000
      size: 1000000
  kernel: 5.15.18-200.fc35.x86_64
  linkmode: dynamic
  logDriver: journald
  memFree: 562688000
  memTotal: 4104019968
  networkBackend: netavark
  ociRuntime:
    name: crun
    package: crun-1.4.2-1.fc35.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.4.2
      commit: f6fbc8f840df1a414f31a60953ae514fa497c748
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  remoteSocket:
    exists: true
    path: /run/user/501/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: true
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.1.12-2.fc35.x86_64
    version: |-
      slirp4netns version 1.1.12
      commit: 7a104a101aa3278a2152351a082a6df71f57c9a3
      libslirp: 4.6.1
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.3
  swapFree: 0
  swapTotal: 0
  uptime: 10h 46m 58.21s (Approximately 0.42 days)
plugins:
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  volume:
  - local
registries:
  search:
  - docker.io
store:
  configFile: /var/home/core/.config/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /var/home/core/.local/share/containers/storage
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 5
  runRoot: /run/user/501/containers
  volumePath: /var/home/core/.local/share/containers/storage/volumes
version:
  APIVersion: 4.0.2
  Built: 1646319416
  BuiltTime: Thu Mar  3 08:56:56 2022
  GitCommit: ""
  GoVersion: go1.16.14
  OsArch: linux/amd64
  Version: 4.0.2


Package info (e.g. output of rpm -q podman or apt list podman):

N/A

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/main/troubleshooting.md)

Yes

Additional environment details (AWS, VirtualBox, physical, etc.):

@openshift-ci openshift-ci bot added the kind/bug Categorizes issue or PR as related to a bug. label Mar 18, 2022
@github-actions github-actions bot added the remote Problem is in podman-remote label Mar 18, 2022
@ghost
Copy link
Author

ghost commented Mar 18, 2022

I did a bit of digging myself and I believe the issue is related to the filepath.Walk through the sources when building the tar file due to the skip on root dir:

return nil // skip root dir

I tried out this code and it seems to make the build work now:

diff --git a/pkg/bindings/images/build.go b/pkg/bindings/images/build.go
index c508cb767..cfba1efe0 100644
--- a/pkg/bindings/images/build.go
+++ b/pkg/bindings/images/build.go
@@ -557,6 +557,57 @@ func nTar(excludes []string, sources ...string) (io.ReadCloser, error) {
 				return
 			}
 
+			// define add file link process to be shared by walk process and single file processor
+			addFileLink := func(info os.FileInfo, filename string) error {
+				di, isHardLink := checkHardLink(info)
+				hdr, err := tar.FileInfoHeader(info, "")
+				if err != nil {
+					return err
+				}
+				hdr.Uid, hdr.Gid = 0, 0
+				orig, ok := seen[di]
+				if ok {
+					hdr.Typeflag = tar.TypeLink
+					hdr.Linkname = orig
+					hdr.Size = 0
+					hdr.Name = filename
+					return tw.WriteHeader(hdr)
+				}
+				f, err := os.Open(filename)
+				if err != nil {
+					return err
+				}
+
+				hdr.Name = filename
+				if err := tw.WriteHeader(hdr); err != nil {
+					f.Close()
+					return err
+				}
+
+				_, err = io.Copy(tw, f)
+				f.Close()
+				if err == nil && isHardLink {
+					seen[di] = filename
+				}
+				return err
+			}
+
+			// if source is a regular file, skip to process it
+			if info, err := os.Stat(s); err == nil && info.Mode().IsRegular() {
+				err = func() error {
+					excluded, err := pm.Matches(s) // nolint:staticcheck
+					if err != nil {
+						return err
+					}
+					if excluded {
+						return nil
+					}
+					return addFileLink(info, s)
+				}()
+				merr = multierror.Append(merr, err)
+				continue
+			}
+
 			err = filepath.Walk(s, func(path string, info os.FileInfo, err error) error {
 				if err != nil {
 					return err
@@ -577,41 +628,10 @@ func nTar(excludes []string, sources ...string) (io.ReadCloser, error) {
 				}
 
 				if info.Mode().IsRegular() { // add file item
-					di, isHardLink := checkHardLink(info)
-					if err != nil {
-						return err
-					}
-
-					hdr, err := tar.FileInfoHeader(info, "")
+					err := addFileLink(info, name)
 					if err != nil {
 						return err
 					}
-					hdr.Uid, hdr.Gid = 0, 0
-					orig, ok := seen[di]
-					if ok {
-						hdr.Typeflag = tar.TypeLink
-						hdr.Linkname = orig
-						hdr.Size = 0
-						hdr.Name = name
-						return tw.WriteHeader(hdr)
-					}
-					f, err := os.Open(path)
-					if err != nil {
-						return err
-					}
-
-					hdr.Name = name
-					if err := tw.WriteHeader(hdr); err != nil {
-						f.Close()
-						return err
-					}
-
-					_, err = io.Copy(tw, f)
-					f.Close()
-					if err == nil && isHardLink {
-						seen[di] = name
-					}
-					return err
 				} else if info.Mode().IsDir() { // add folders
 					hdr, lerr := tar.FileInfoHeader(info, name)
 					if lerr != nil {

Not sure if there would be other implications with this change though with other systems.

@ghost
Copy link
Author

ghost commented Mar 18, 2022

before changes:

make podman-remote-darwin && echo "FROM ubi8/ubi-minimal:latest" | bin/darwin/podman build -f- ~/dev/podman-file-out-of-context/context_dir
CGO_ENABLED=0 \
                GOOS=darwin \
                GOARCH=amd64 \
                go build \
                -mod=vendor  \
                -o bin/darwin/podman-mac-helper \
                ./cmd/podman-mac-helper
/Library/Developer/CommandLineTools/usr/bin/make \
                CGO_ENABLED=0 \
                GOOS=darwin \
                GOARCH=amd64 \
                bin/darwin/podman
CGO_ENABLED=0 GOOS=darwin GOARCH=amd64 go build \
                -mod=vendor  \
                -ldflags '-X github.com/containers/podman/v4/libpod/define.gitCommit=8ebb94f2ced7fb8fa60ae0b53522cc6d9d6d0240 -X github.com/containers/podman/v4/libpod/define.buildInfo=1647642121 -X github.com/containers/podman/v4/libpod/config._installPrefix=/usr/local -X github.com/containers/podman/v4/libpod/config._etcDir=/usr/local/etc -X github.com/containers/common/pkg/config.additionalHelperBinariesDir= ' \
                -tags "remote exclude_graphdriver_btrfs btrfs_noversion exclude_graphdriver_devicemapper containers_image_openpgp" \
                -o bin/darwin/podman ./cmd/podman
Error: stat /var/tmp/libpod_builder531946892/build/private/var/folders/6d/npjyy5zs5x98y8fbpnjb_16m0000gn/T/build2533223464: no such file or directory

after changes:

make podman-remote-darwin && echo "FROM ubi8/ubi-minimal:latest" | bin/darwin/podman build -f- ~/dev/podman-file-out-of-context/context_dir
CGO_ENABLED=0 \
                GOOS=darwin \
                GOARCH=amd64 \
                go build \
                -mod=vendor  \
                -o bin/darwin/podman-mac-helper \
                ./cmd/podman-mac-helper
/Library/Developer/CommandLineTools/usr/bin/make \
                CGO_ENABLED=0 \
                GOOS=darwin \
                GOARCH=amd64 \
                bin/darwin/podman
CGO_ENABLED=0 GOOS=darwin GOARCH=amd64 go build \
                -mod=vendor  \
                -ldflags '-X github.com/containers/podman/v4/libpod/define.gitCommit=8ebb94f2ced7fb8fa60ae0b53522cc6d9d6d0240-dirty -X github.com/containers/podman/v4/libpod/define.buildInfo=1647642154 -X github.com/containers/podman/v4/libpod/config._installPrefix=/usr/local -X github.com/containers/podman/v4/libpod/config._etcDir=/usr/local/etc -X github.com/containers/common/pkg/config.additionalHelperBinariesDir= ' \
                -tags "remote exclude_graphdriver_btrfs btrfs_noversion exclude_graphdriver_devicemapper containers_image_openpgp" \
                -o bin/darwin/podman ./cmd/podman
STEP 1/1: FROM ubi8/ubi-minimal:latest
COMMIT
--> 281e62f51d4
281e62f51d47ff41e03e8f9a66fcd8b0a80a1f06445f5b4c3831c591c6d72cbe

@ghost ghost changed the title OS X podman build fails with Dockerfile not in context macOS podman build fails with Dockerfile not in context Mar 19, 2022
@Luap99
Copy link
Member

Luap99 commented Mar 21, 2022

Duplicate of #13293
@cdoern PTAL at the suggested change

@Luap99 Luap99 closed this as completed Mar 21, 2022
@cdoern
Copy link
Contributor

cdoern commented Mar 21, 2022

thanks for the suggestion @Cameron-Kurotori I have a similar solution on my end and will push it to #13531 soon

the issue is, we do want to skip the root dir sometimes but the way nTar is written we are skipping basically everything so I just made a quick patch to check for some particular cases.

@github-actions github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Sep 20, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 20, 2023
# for free to subscribe to this conversation on GitHub. Already have an account? #.
Labels
kind/bug Categorizes issue or PR as related to a bug. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. remote Problem is in podman-remote
Projects
None yet
Development

No branches or pull requests

2 participants