allowing docker to use cached base image in build

[JeffDownie]

This fixes the last comment in issue #34 - and adds a flag to allow not pulling the base image down on every build.

I'm not 100% sure that the name I've given to the option is good, but I couldn't get the EnvVar options to work with a bool flag that was true-by-default correctly :/. This being my first go code ever, I'm happy to take any suggestions/help regarding this!

example usage:

  publish:
    image: plugins/drone-docker:<NEW_VERSION>
    repo: <MY_REPO>
    use_cache: true
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

Needs to be run as trusted for the volume obviously!

[bradrydzewski]

Thanks! added some minor comments, but one resolved will be good to merge

[bradrydzewski]

I think this could be condensed to a single line:

if build.Pull {
  args = append(args, "--pull=true")
}

[bradrydzewski]

also make sure to run gofmt on your changes. This will ensure the codebase conforms to the official Go style guidelines (in this case tags for indentation)

[JeffDownie]

Doooh.. now I feel silly! I knew there was a way of doing this without negating the option, thanks!

[bradrydzewski]

recommend change to Pull

[bradrydzewski]

recommend change to:

c.BoolT("pull-image")

[bradrydzewski]

also note we need to use c.BoolT here

[bradrydzewski]

recommend change to:

cli.BoolTFlag{
  Name: "pull_image",
  Usage: "pull a newer version of the image on build",
  EnvVar: "PLUGIN_PULL_IMAGE",
}

[bradrydzewski]

note that we cannot use pull here. This is a reserved keywork in the yaml and will not get passed to the plugin. So instead we use pull_image

[bradrydzewski]

can you revert this file? You can add editor specific files to .git/info/exclude

[bradrydzewski]

I should also mention that mounting a volume should not be required, and may result in unexpected behavior and certain settings not working because you are bypassing the docker-in-docker daemon that is executing inside this plugin container.

This plugin is meant to give the ability to build and publish images without compromising the host machine. So if you are mounting a docker volume, I would recommend not using this plugin and instead running docker commands directly:

pipeline:
  publish:
-   image: plugins/docker
+   image: docker
    commands:
      - docker build .
      - docker tag ...
      - ...
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

If you want to use the docker plugin with layer caching, the recommended solution is to use one of the drone caching plugins. These plugins give you the ability to cache files and folders and restore them at the start of your build.

pipeline:
  restore_cache:
    image: <cache plugin>
    restore: true
    mount: /drone/docker
  publish:
    image: plugins/docker
    repo: foo/bar
    storage_path: /drone/docker  # volume shared by all containers in the pipeline
  rebuild_cache:
    image: <cache plugin>
    restore: true
    mount: /drone/docker

In the above example we make sure the docker storage path is in the shared volume so that is accessible by all steps in the pipeline. The first step (restore) will restore the docker storage path from the cache. The last step will update it. This is the recommended way to persist state between builds without having to mount a volume (although there is a cache plugin that does this too, if you want a local cache)

[JeffDownie]

@bradrydzewski I've made the changes - didn't know about this gofmt tool, nice!

It might be worth mentioning, I had a lot of trouble running my own version of this plugin on my own drone server, because it is hardcoded to run as privileged in the actual drone agent - it might be worth documenting this somewhere!

I also came across an issue where I tried to make the storage_path a relative path, which caused the docker daemon to hang on startup. (I wanted to store the docker stuff in a subdirectory of the workspace, so it would definitely be shared between everything in the pipeline to cache it like you suggested)

[JeffDownie]

@bradrydzewski I've been trying to use the above caching method with this plugin, and I can't seem to get it to work nicely - how can you avoid the cache growing in size with old images continually? There's no obvious way of cleaning the docker storage of everything except an image?

The only obvious solution that comes to my mind is not using any drone caching plugin and instead pulling down a previous version of the image from the registry on this image's startup. This feels a little excessive for a plugin though - I may just write my own commands onto a docker-in-docker container to get the caching strategy I need.

[bradrydzewski]

maybe add a prune: true flag to the yaml, and if true it runs docker image prune. This will remove all dangling layers. Maybe add a prune_filter: ... to set the docker image prune --filter flag for more control. Some cache plugins also let you control the duration of the cache (ie 30 days). Pruning and cache expiration can both probably help reduce this problem.

I may just write my own commands onto a docker-in-docker container to get the caching strategy I need.

That is fine too. This is not intended to be a one-size-fits-all plugin. It works well for most people that use it, but certainly not everyone. I therefore highly encourage forking and / or creating alternate versions of plugins tailored to fit specific requirements and use cases.

[clayrisser]

@bradrydzewski, what cache plugin should I use for <cache plugin>?