Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Updated Backup section #4103

Merged
merged 10 commits into from
Jul 24, 2020
57 changes: 53 additions & 4 deletions docs/4.3/admin-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -2219,8 +2219,10 @@ As an extra precaution you might want to backup your application prior to upgrad
to SHA-512 based signatures. To upgrade an existing cluster, set the following in
your teleport.yaml:

```bash
teleport:
ca_signature_algo: “rsa-sha2-512”
```

After updating to 4.3+ rotate the cluster CA [following these docs](#certificate-rotation).

Expand Down Expand Up @@ -2259,11 +2261,58 @@ When upgrading multiple clusters:

## Backing Up Teleport

When planning a backup of Teleport, it's important to know what is where and the
importance of each component. Teleports Proxies and Nodes are stateless, and thus
only `teleport.yaml` should be backed up.

The Auth server is Teleports brains, and depending on the backend should be backup
in a regular cadance.

For example a customer running Teleport on AWS with etd have these key items of data:

| What | Where ( Example AWS Customer ) |
|-|-|
| Local Users ( not SSO ) | DynamoDB |
| Certificate Authorities | DynamoDB |
| Trusted Clusters | DynamoDB |
| Connectors: SSO | DynamoDB / File System |
| RBAC | DynamoDB / File System |
| teleport.yaml | File System |
| teleport.service | File System |
| license.pem | File System |
| TLS /. | ( File System / Outside Scope ) |

For this customer, we would Recommend using [AWS best practices](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/BackupRestore.html) for backing up DynamoDB. If DynamoDB is used for
the audit log, logged events have a TTL of 1 year.

| Backend | Recommend Backup Strategy |
|-|-|
| dir ( local filesystem ) | Copy `data_dir/storage` and use `tctl get all` to get local state. |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd rephrase the strategy as: "Backup /var/lib/teleport/storage directory and the output of tctl get all".
Also, if a user backs up /var/lib/teleport, is there any reason to do tctl get all?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the idea with tctl get all was that the format would be YAML and therefore backend agnostic, so theoretically you could dump out the CAs, users etc and then restore these into a different backend.

| DynamoDB | [Follow AWS Guidelines for Backup & Restore](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/BackupRestore.html) |
| etcD | [Follow etcD Guidleines for Disaster Recovery ](https://etcd.io/docs/v2/admin_guide) |
| Firestore | [Follow GCP Guidlines for Automated Backups](https://firebase.google.com/docs/database/backups) |

### Config Files

We would recommend you store configs as discrete files in git.

### GitOps

If running Teleport at scale it's important for teams to have an automated way to
restore Teleport. At a high level, this is our recommend approach:

- Persist and backup your backend
- Share that backend among auth servers
- Store your configs as discrete files in git
- Have your CI run `tctl create -f *.yaml` from that git directory, without restarting teleport service / pods in k8s
- If you lose your backend, this setup still restores the essential configs

## Migrating Backends.
As of version v4.1 you can now quickly export a collection of resources from
Teleport. This feature set works best for local and etcd, it's currently experimental
for AWS/GCP.
Teleport. This feature was designed to help customers migrate from local storage
to etcd.

Using `tctl get all` will backup.
Using `tctl get all` will retrive the below items:

- Users
- Certificate Authorities
Expand All @@ -2274,7 +2323,7 @@ Using `tctl get all` will backup.
- OIDC [Teleport Enterprise]
- Roles [Teleport Enterprise]

When backing up Teleport you'll need to backup up your auth server's `data_dir/storage` directly.
When migrating backend you'll should backup up your auth server's `data_dir/storage` directly.

**Example of backing up and restoring a cluster.**

Expand Down