Add Support for "Overwrite" instruction in appspec.yml "Files" section

[colinbjohnson]

Feature Request:

Would like the ability to have an "overwrite" instruction in the appspec.yml "Files" section.

Use:

In my initial testing, I wanted to clobber all files in /var/www/html with new files. This resulted in a failure because the files already existed. Specifically, the following failure: 2015-06-13 20:58:40 ERROR [codedeploy-agent(7589)]: InstanceAgent::Plugins::CodeDeployPlugin::CommandPoller: Error during perform: RuntimeError - File already exists at location /var/www/html/index.html - /opt/codedeploy-agent/lib/instance_agent/plugins/codedeploy/installer.rb:113:ingenerate_normal_copy'`

Desired Behavior:

I'd like to be able to create an appspec.yml file with the following section:
files:
- source: web
destination: /var/www/html/
overwrite: true
Notice I've added the "overwrite" instruction, which is currently not offered.

Current Workaround:

I've been working around by utilizing the hook section to do file copies. The hooks/shell script workaround is sufficient, rendering this a lower priority feature request.

[surybala]

Hi,
Thanks for the feedback. I've added this feature request to our backlog.

[CNG]

I would also love this feature.

[madisonmay]

Add one more name to the list.

[kingdango]

Of course we need this feature!
But it should not be simply overwrite: true -- it should allow for a few options:
overwrite: none (default)
overwrite: purge
overwrite: merge

Purge removes all files in the destination before performing copying the new files.

Merge leaves the destination as-is, adds the files from source and will replace an files in the destination that also exist in the source.

As for now I guess I need to stage my source files in a temp location and then run a script to achieve this. 🐼 😢

[bslatner]

+1 for this feature as well.

Side note: the Windows agent does NOT exhibit this behavior. I've been writing several appspecs for Windows services and I've never run into this. Only when we went to deploy to an Ubuntu server for the first time did we see this behavior.

[JoshuaEstes]

+1

[etcsayeghr]

+1

[ixaxaar]

+1

[vitalyzeme]

+1

[praddc]

+1

[gabriellhrn]

+1

[JoshuaEstes]

So, I hope this helps other people find a solution. In my situation I work a lot with Magento so I deal with PHP, nginx. Composer is also tossed in the mix. I found a solution that works for me and for my case. If anyone has any comments or suggestions, please let me know.

---

I wanted to be have the production directory synced with the repository that I was using so any file modifications would be reflected. Not just updates to files, but delete the files that are no longer required. I think this is what everyone really wants. The solution I found was to deploy the code into a temporary directory (/var/www/release) and then rsync that directory over to the production directory (/var/www/magento).

appspec.yaml

version: 0.0
os: linux
files:
    - source: ./
      destination: /var/www/release/
permissions:
    - object: /var/www/magento/
      owner: www-data
      group: www-data
hooks:
    BeforeInstall:
        - location: scripts/beforeInstall.bash
    AfterInstall:
        - location: scripts/afterInstall.bash

scripts/beforeInstall.bash

#!/usr/bin/env bash

# I want to make sure that the directory is clean and has nothing left over from
# previous deployments. The servers auto scale so the directory may or may not
# exist.
if [ -d /var/www/release ]; then
    rm -rf /var/www/release
fi
mkdir -vp /var/www/release

scripts/afterInstall.bash

#!/usr/bin/env bash

# I have left a few things in here and will explain this further (see below)
rsync --delete-before --verbose --archive --exclude "htdocs/media/" --exclude ".*" --exclude "htdocs/var/" --exclude "htdocs/app/etc/local.xml" /var/www/release/ /var/www/magento/ > /var/log/deploy.log

The rsync command is what allows you to overwrite all the files you need and it also deletes files if they are deleted from the repository. In my case I am deploying a git repo and there are times when extensions/files are removed and those need to be removed on the server.

There are also directories that are excluded from the rsync command. You do not need to sync cache directories and in my case the htdocs/media directory is a mounted S3 bucket and there are lots of files we do not need to delete.

All of the output is put into a deployment log (/var/log/deploy.log) file that I can use in case something goes wrong and can see the exact files deleted and synced over. When I was first experimenting with this, I ran rsync in dry-run mode and checked the log file to make sure files were synced correctly. I HIGHLY recommend you do the same.

---

Some things I want to try when I make the time, is to rsync files directly from the code-deploy directory. For each deployment, AWS CodeDeploy puts the code into a specific directory which you can reach by some of the environment variables it sets. This should speed up the deploy a little since it isn't putting the files in two places.

There are some things that have been left out of the scripts. Certain directories are chowned with www-data user/group since the files are served used nginx. You may need something similar. The code that places Magento into maintenance mode and removes it from maintenance mode has also been removed since this needs to be generic. You may need similar logic.

---

You can read more about the various hooks and when they occur in the docs at http://docs.aws.amazon.com/codedeploy/latest/userguide/app-spec-ref.html and also see the environment variables that can be used within the scripts.

[maorhayoun]

+1

[sverdier]

+1