Integration Testing

Setup a development container following these instructions. Follow the next five steps to conduct an integration test from within the container.

1. Test the Environment Variables
2. Making the First Backup
3. Restoring the First Backup
4. Scheduling an Unattended Backup
5. Stopping the Test

Test the Environment Variables

From within the container, run the following command to validate the environment variables have been properly initialized.

container:~$ restic-unattended list

The output should look similar to this:

VARIABLE            	SET	DESCRIPTION                                                                 
B2_ACCOUNT_ID_FILE  	Yes	Name of file containing the Account ID or applicationKeyId for Backblaze B2	
B2_ACCOUNT_KEY_FILE 	Yes	Name of file containing the Account Key or applicationKey for Backblaze B2 	
RESTIC_PASSWORD_FILE	Yes	Name of file containing the restic password                                	
RESTIC_REPOSITORY   	Yes	Location of the repository                                                     	

To review all supported environment variables instead, run restic-unattended list -a.

Making the First Backup

Now let's create some test data to test the backup functionality. Create a file test.txt in the folder ~/backup and enter some sample text.

The Docker volume /data/backup is mounted as read-only by default. Typically it is shared with another container that has full ownership of the volume. The UID and GID of the user running the container need to be the same.

container:~$ mkdir -p ~/backup
container:~$ printf "This is a sample file to test restic backup and restore" > ~/backup/test.txt

Now test the configuration by creating the first backup. Use the following command to create a backup on the spot and to init the remote repository if needed.

container:~$ restic-unattended backup -p ~/backup --init

After some processing, the output should look similar to this.

Starting backup operation of path '/home/restic/backup'
no parent snapshot found, will read all files

Files:           1 new,     0 changed,     0 unmodified
Dirs:            3 new,     0 changed,     0 unmodified
Added to the repo: 1.414 KiB

processed 1 files, 55 B in 0:07
snapshot xxxxxxxx saved
Finished backup operation of path '/home/restic/backup'

Restoring the First Backup

Now perform a restore operation to test if the backup did work. Use the following command to restore the data to the /data/restore folder, which has been created by Docker during initialization. Please note that restore uses the latest available snapshot by default, but can be instructed to use a specific snapshot instead. See restic-unattended restore -h and restic-unattended snapshots -h for more details.

container:~$ restic-unattended restore /data/restore

The output should look similar to this.

Starting restore operation for snapshot 'latest'
restoring <Snapshot XXXXXXXX of [/home/restic/backup] at DATE by USER> to /data/restore

Finished restore operation for snapshot 'latest'

Navigate to the /data/restore folder to verify the data has been restored correctly.

Scheduling an Unattended Backup

Test the scheduled backup functionality once the one-off backup is working correctly. Restic-unattended has a built-in scheduler capable of interpreting cron specifications. It supports several keywords and optional seconds too (emulating the alternative cron format provided by Quartz). See the cron documentation for more details. For testing purposes, we will instruct restic-unattended to perform a backup every minute at zero seconds, and a forget operation every minute at 30 seconds. The folder to backup is specified by the -p flag. It can also be configured as an environment variable instead. The forget operation instructs restic to remove old snapshots according to a policy. In this case, we instruct restic to keep the last 5 snapshots. See restic-unattended forget -h for more options.

The scheduler fires backup and forget jobs at the specified intervals. In this test, the data to be backed up is very small and the jobs are both expected to finish in less than 30 seconds. The scheduler processes one job at a time, following a First In, First Out (FIFO) policy. If a current job is still running, the next job is delayed until the current job has finished. A maximum of 5 jobs is kept at any time. Additional jobs will be dropped when the maximum capacity has been reached. In practice, it is recommended to time the typical duration of your jobs and to set a realistic schedule for both types of jobs.

Test the scheduling functionality with the following command.

Due to limitations of the used software libraries, the ~ expansion does not work when using -p=~/backup. Use a format without the = #stead.

container:~$ restic-unattended schedule '0 * * * * *' -p ~/backup --forget '30 * * * * *' --keep-last 5

The schedule job keeps on running until you hit ctrl-c. You should see logging output similar to the below example. The timestamps have been removed for brevity (and can also be omitted by using the flag --logformat=default).

The first section displays several initialization messages.

INFO   | Executing schedule command
INFO   | Scheduling job 'backup' with cron spec '0 * * * * *'
INFO   | First 'backup' job scheduled to run at 'TIME'
INFO   | Scheduling job 'forget' with cron spec '30 * * * * *'
INFO   | First 'forget' job scheduled to run at 'TIME'

The forget operation shows output similar to this once it started running.

INFO   | Starting forget operation
INFO   | Applying Policy: keep 5 latest snapshots
INFO   | keep 3 snapshots:
INFO   | ID        Time  Host  Tags  Reasons        Paths
INFO   | -------------------------------------------------------
INFO   | XXXXXXXX  TIME  host        last snapshot  /data/backup
INFO   | YYYYYYYY  TIME  host        last snapshot  /data/backup
INFO   | ZZZZZZZZ  TIME  host        last snapshot  /data/backup
INFO   | -------------------------------------------------------
INFO   | 3 snapshots
INFO   | keep 4 snapshots:
INFO   | ID        Time  Host  Tags  Reasons        Paths
INFO   | -------------------------------------------------------
INFO   | WWWWWWWW  TIME  host        last snapshot  /data/backup
INFO   | XXXXXXXX  TIME  host        last snapshot  /data/backup
INFO   | YYYYYYYY  TIME  host        last snapshot  /data/backup
INFO   | ZZZZZZZZ  TIME  host        last snapshot  /data/backup
INFO   | -------------------------------------------------------
INFO   | 4 snapshots
INFO   | keep 1 snapshots:
INFO   | ID        Time  Host  Tags  Reasons        Paths
INFO   | -------------------------------------------------------
INFO   | ZZZZZZZZ  TIME  host        last snapshot  /data/backup
INFO   | -------------------------------------------------------
INFO   | 1 snapshots
INFO   | Finished forget operation

The backup operation shows output similar to this.

INFO   | Starting backup operation of path '/home/restic/backup'
INFO   | Files:           0 new,     0 changed,     1 unmodified
INFO   | Dirs:            0 new,     2 changed,     0 unmodified
INFO   | Added to the repo: 342 B
INFO   | processed 1 files, 3.045 KiB in 0:03
INFO   | snapshot XXXXXXXX saved
INFO   | Finished backup operation of path '/home/restic/backup'

Hit ctrl-c to stop the scheduler.

WARN   | Worker processing canceled
FATAL  | Error running schedule command error="Cron processing interrupted"

Stopping the Test

Stop execution of the container by entering exit on the container's command line. Dbm then automatically removes the created container and network.

Stopping restic-unattended_restic_1 ... done
Removing restic-unattended_restic_1 ... done
Removing network restic
Done.