Backup Manager

Backup Manager is an extensible tool to backup data from various cloud providers as well as GitHub, Wordpress, servers and databases.

Currently supported:

• GitHub
• Google Drive
• Google Photos
• Linux Server
• MySQL Database
• WordPress
• Dropbox
• PostgreSQL Database

Google and Dropbox will need some minor preparations, see below for details

Prerequisites

sudo apt install python3 python3-urllib3 sshpass mysql-client postgresql-client git

pip3 install -r requirements.txt

Add accounts

To add an account to backup run

$ python3 main.py --add

and follow the instructions

Backup accounts

To start the backup run

$ python3 main.py backup [alias1, alias2]

If you provide aliases, only those accounts will be backed up.

Set up E-Mail notifications

Note that this feature will currently only work with Gmail

On first start the script will ask for Gmail credentials for notifications.

If you choose to enable this feature, enter your Gmail-Account as "Mail user" and an "App Password" as "Mail password".

Check out this link for how to obtain an app password.

Set up Google-Backup

1. Open google cloud console
2. Choose or create a project
3. Activate Drive API
4. Activate Photos API
5. Create consent page
6. Choose "External"
7. Enter a name, support email and contact email
8. Click "Save and continue"
9. Click "Add or remove scopes"
10. Select ".../auth/drive.readonly"
11. Select ".../auth/photoslibrary.readonly"
12. Click "Save and continue"
13. Enter yourself as a test user
14. Click "Save and continue"
15. Open credentials page
16. Click on "Create Credentials" -> OAuth-Client-ID -> Desktop Application
17. Download the Client ID JSON
18. When prompted by the script, paste the content of that json

Set up Dropbox-Backup

1. Open Dropbox developers page
2. Click "Create app"
3. Select "Dropbox API"
4. Select "Full Dropbox"
5. Give it a name
6. Click "Create app"
7. Under "Generated access token" click "Generate"
8. When prompted by the script, paste that token

Set up GitHub-Backup

Follow this documentation to create your personal access token.

You will be prompted for it when adding a GitHub backup

Using with Docker

Backup Manager also works in a Docker environment.

Adding accounts

To add new accounts, run:

docker run --rm -it -v "/path/to/backups:/app/backups" -v "/path/to/log:/app/log" -v "/path/to/config/config.json:/app/config/config.json" paranerd/backup-manager add

Then follow the adding process.

You may replace the backup path however you like, just remember to reflect it properly when adding.

Running the backup

Analogous to adding, the backup command is as follows:

docker run --rm -it -v "/path/to/backups:/app/backups" -v "/path/to/config/config.json:/app/config/config.json" paranerd/backup-manager backup [alias1, alias2]

How it works

1. GitHub
   • Archive
     • Goes through all your repositories and backs up the latest release
     • Saves each repository as [repository_name]-[tag].zip (e.g. my-project-1.0.zip)
     • Older versions of the same repository will be removed (so there's only the most recent version backed up)
   • Non-archive
     • Uses git (requires git to be set up) to clone or pull --rebase
2. GitHub Gist
   • Archive
     • Goes through all your gists and backs up the latest release
     • Saves each repository as [gist-id]-[name of first file]-[hashed-update-datetime].zip (e.g. my-project-1.0.zip)
     • Older versions of the same gist will be removed (so there's only the most recent version backed up)
   • Non-archive
     • Uses git (requires git to be set up) to clone or pull --rebase
3. Google Drive
   • Mirrors the folder structure of your Drive to your backup-path
   • For regular files it compares the MD5-Hash to determine whether to download it or not (to avoid overhead)
   • Google Docs and Spreadsheets (for which no MD5 is provided by the API), will be backed up every time and exported as .pdf and .xlsx respectively
4. Google Photos
   • Uses album-names to create a folder structure
   • An album "2018-12-24 Christmas" will be backed up to /your/backup/folder/2018/2018-01-01 Christmas/
   • All other albums go to /your/backup/folder/0000/[albumname]
   • Since there's no MD5-Hashes for images either, the script simply uses the original filename to determine if it should download the image or not
5. Linux Server
   • Uses SSH to connect to the server
   • Archive
     • Uses zip on the server to compress, then backs up as many versions as specified
   • Non-archive
     • Uses rsync
6. MySQL Database
   • Uses mysqldump to dump the database
7. WordPress
   • Simply a combination of "Linux Server" and "MySQL Database"
8. Dropbox
   • Mirrors the folder structure of your Dropbox to your backup-path
   • Checks for content hash to determine whether to download it or not (to avoid overhead)