Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

x-initialSync should always copy files and directories when --watch is passed #12254

Open
mvhatch opened this issue Oct 30, 2024 · 5 comments
Open

Comments

@mvhatch
Copy link

mvhatch commented Oct 30, 2024

Description

Running docker compose up --watch <service> does not copy files or directories when the file/directory is modified before image creation, even when setting x-initialSync: true. Why not just copy the files/directories without checking its modified timestamp?

@ndeloof
Copy link
Contributor

ndeloof commented Nov 6, 2024

x-initialSync is an experimental extension (as the tag name suggests). Actual behavior is subject to changes.
We only sync files which aren't up-to-date with image as such a copy can impact a large number of files and can slow down initial start for no reason.
Can you please clarify your usage? Why would you need to sync files which are already up-to-date in built image ?

@mvhatch
Copy link
Author

mvhatch commented Nov 6, 2024

I have a monorepo with dozens of services. I have a single compose.yml file with a top-level anchor the defines some common attributes for a service, including a docker image. This docker image only provides some common utilities and dependencies, but it does not copy any files from the monorepo. Instead, I'd like to specify a develop attribute for each service in my compose.yml, and watch a subset of files in my monorepo.

So to answer your question:

Why would you need to sync files which are already up-to-date in built image ?

Because there are no files in my built image. This seems to be a fairly common pattern (see thread here asking for the same thing)

Does docker compose cp not copy the files a user requests because it could be a large number of files? Can you allow end users to copy a large number of files w/ x-InitialSync in the same vein? Alternatively, could you allow docker compose --watch to watch a subset of files, even if some of those files are already being bind-mounted?

@ndeloof
Copy link
Contributor

ndeloof commented Nov 11, 2024

Because there are no files in my built image

Does this mean you can't run docker compose up with this image, without an extra step to copy content ?

Example on #11102 based on Traeffic demonstrate this misconception. watch should be used to make development iteration simpler and faster, but application should be able to run without. A workaround is to add a Dockerfile to add required files, so that image you run has required content:

services:
  traefik:
    build:
      dockerfile-inline: | 
        FROM: traefik:v2.10.7
        COPY ./traefik /etc/traefik
    develop:
      watch:
        - action: sync+restart
          path: ./traefik
          target: /etc/traefik

@mvhatch
Copy link
Author

mvhatch commented Nov 11, 2024

Does this mean you can't run docker compose up with this image, without an extra step to copy content ?

Not entirely the case. Our Dockerfile has a volume section that uses env var interpolation (e.g. ${APP_HOST_MOUNT}) that is properly set when executing make run. This bind mounts the developer's filesystem into the container. However, make watch sets the value to /dev/null (because docker compose watch is unable to watch anything under a directory that is bind mounted) and then copies the contents if the filesystem into the docker container.

It's just a little annoying to do this hack, and I was hoping to get this functionality directly from x-initialSync: true.

@mmx86
Copy link

mmx86 commented Nov 21, 2024

In my case I'm syncing nginx configs directory from two different sources:

  1. Service related.
  2. Dev environment related (some conf.d entires, some templates).

Second one is not synced initially.

    develop:
      watch:
        - action: sync+restart
          path: ./my-service/nginx/
          target: /etc/nginx/
        - action: sync+restart
          path: ./dev/my-service/nginx/
          target: /etc/nginx/
          x-initialSync: true

# for free to join this conversation on GitHub. Already have an account? # to comment
Projects
None yet
Development

No branches or pull requests

3 participants