From a7093bd2c9560b5088c9e6d2f9b7c9067d1bf6a5 Mon Sep 17 00:00:00 2001 From: Jon Udell Date: Thu, 2 Jan 2025 15:56:53 -0800 Subject: [PATCH] begin updating create snapshots --- docs/query/snapshots.md | 62 ++++++++++++++--------------------------- 1 file changed, 21 insertions(+), 41 deletions(-) diff --git a/docs/query/snapshots.md b/docs/query/snapshots.md index e5fd0c7..dfe123a 100644 --- a/docs/query/snapshots.md +++ b/docs/query/snapshots.md @@ -2,101 +2,81 @@ title: Snapshots --- ->[!NOTE] -> cloned from steampipe. how much is applicable? i know i can snap/restore locally, cloud stuff out of scope? - # Snapshots -Steampipe allows you to take **snapshots**. A snapshot is a saved view of your query results that you can view as a [dashboard in Powerpipe](https://powerpipe.io/docs/run/dashboard) All data and metadata for a snapshot is contained in a JSON file which can be saved and viewed locally in the Powerpipe dashboard or uploaded to [Turbot Pipes](https://turbot.com/pipes/docs). Snapshots in Turbot Pipes may be shared with other Turbot Pipes users or made public (shared with anyone that has the link). +Tailpipe enables you to take **snapshots**. A snapshot is a saved view of a benchmark run or dashboard that you can view as a [dashboard in Powerpipe](https://powerpipe.io/docs/run/dashboard) All data and metadata for a snapshot is contained in a JSON file which can be saved and viewed locally in the Powerpipe dashboard or uploaded to [Turbot Pipes](https://turbot.com/pipes/docs). Snapshots in Turbot Pipes may be shared with other Turbot Pipes users or made public (shared with anyone that has the link). -You can create Turbot Pipes snapshots directly from the Steampipe CLI, however if you wish to subsequently [modify](https://turbot.com/pipes/docs/dashboards#managing-snapshots) them (add/remove tags, change visibility) or delete them, you must do so from the Turbot Pipes console. You may [browse the snapshot list](https://turbot.com/pipes/docs/dashboards#browsing-snapshots) in Turbot Pipes by clicking the **Snapshots** button on the top of your workspace's **Dashboards** page. +You can create Turbot Pipes snapshots directly from the Powerpipe CLI, however if you wish to subsequently [modify](https://turbot.com/pipes/docs/dashboards#managing-snapshots) them (add/remove tags, change visibility) or delete them, you must do so from the Turbot Pipes console. You may [browse the snapshot list](https://turbot.com/pipes/docs/dashboards#browsing-snapshots) in Turbot Pipes by clicking the **Snapshots** button on the top of your workspace's **Dashboards** page. ## Taking Snapshots -> To upload snapshots to Turbot Pipes, you must either [log in via the `steampipe login` command](/docs/reference/cli/login) or create an [API token](https://turbot.com/pipes/docs/profile#tokens) and pass it via the [`--pipes-token`](/docs/reference/cli/overview#global-flags) flag or [`PIPES_TOKEN`](/docs/reference/env-vars/pipes_token) environment variable. +To upload snapshots to Turbot Pipes, you must either [log in via the `powerpipe login` command](https://powerpipe.io/docs/reference/cli/login) or create an [API token](https://turbot.com/pipes/docs/profile#tokens) and pass it via the [`--pipes-token`](/docs/reference/cli/overview#global-flags) flag or [`PIPES_TOKEN`](/docs/reference/env-vars/pipes_token) environment variable. To take a snapshot and save it to [Turbot Pipes](https://turbot.com/pipes/docs), simply add the `--snapshot` flag to your command. ```bash -steampipe query --snapshot "select * from aws_ec2_instance" +powerpipe query run "select * from aws_cloudtrail_log order by tp_date desc limit 1000" --snapshot ``` +```bash +powerpipe benchmark run cloudtrail_log_detections --snapshot +``` + + The `--snapshot` flag will create a snapshot with `workspace` visibility in your user workspace. A snapshot with `workspace` visibility is visible only to users that have access to the workspace in which the snapshot resides -- A user must be authenticated to Turbot Pipes with permissions on the workspace. If you want to create a snapshot that can be shared with *anyone*, use the `--share` flag instead. This will create the snapshot with `anyone_with_link` visibility: ```bash -steampipe query --share "select * from aws_ec2_instance" +powerpipe benchmark run cloudtrail_log_detections --share ``` You can set a snapshot title in Turbot Pipes with the `--snapshot-title` argument. ```bash -steampipe query --share --snapshot-title "Public Buckets" "select name from aws_s3_bucket where bucket_policy_is_public" +powerpipe query run "select * from aws_cloudtrail_log order by tp_date desc limit 1000" --share --snapshot-title "Recent Cloudtrail log lines" ``` - If you wish to save to the snapshot to a different workspace, such as an org workspace, you can use the `--snapshot-location` argument with `--share` or `--snapshot`: ```bash -steampipe query --share --snapshot-location vandelay-industries/latex "select * from aws_ec2_instance" -``` - -Note that the previous command ran the query against the *local* database, but saved the snapshot to the `vandelay-industries/latex` workspace. If you want to run the query against the remote `vandelay-industries/latex` database AND store the snapshot there, you can also add the `--database-location` argument: - -```bash -steampipe query --share --snapshot-location vandelay-industries/latex \ - --workspace-database vandelay-industries/latex \ - "select * from aws_ec2_instance" -``` - -Steampipe provides a shortcut for this though. The `--workspace` flag supports [passing the cloud workspace](/docs/managing/workspaces#implicit-workspaces): -```bash - -steampipe query --snapshot --workspace vandelay-industries/latex "select * from aws_ec2_instance" +powerpipe query run "select * from aws_cloudtrail_log order by tp_date desc limit 1000" --share --snapshot-location my-org/my-workspace ``` -While not a common case, you can even run a query against a Turbot Pipes workspace database, but store the snapshot in an entirely different Turbot Pipes workspace: -```bash - -steampipe query --share --snapshot-location vandelay-industries/latex \ - --workspace vandelay-industries/latex-prod \ - "select * from aws_ec2_instance" -``` - - ## Tagging Snapshots You may want to tag your snapshots to make it easier to organize them. You can use the `--snapshot-tag` argument to add a tag: ```bash -steampipe query --snapshot-tag env=local --snapshot \ - "select * from aws_ec2_instance" +powerpipe benchmark run cloudtrail_log_detections --snapshot-tag:source:cloudtrail ``` Simply repeat the flag to add more than one tag: ```bash -steampipe query --snapshot-tag env=local --snapshot --snapshot-tag owner=george \ - "select * from aws_ec2_instance" - +powerpipe benchmark run cloudtrail_log_detections --snapshot-tag:source:cloudtrail --snapshot-tag:env:prod ``` - ## Saving Snapshots to Local Files Turbot Pipes makes it easy to save and share your snapshots, however it is not strictly required; You can save and view snapshots using only the CLI. -You can specify a local path in the `--snapshot-location` argument or `STEAMPIPE_SNAPSHOT_LOCATION` environment variable to save your snapshots to a directory in your filesystem: +You can specify a local path in the `--snapshot-location` argument or `TAILPIPE_SNAPSHOT_LOCATION` environment variable to save your snapshots to a directory in your filesystem: ```bash +powerpipe benchmark run cloudtrail_log_detections --snapshot-location . + steampipe query --snapshot --snapshot-location . "select * from aws_account" ``` You can also set `snapshot_location` in a [workspace](/docs/managing/workspaces) if you wish to make it the default location. +>[!NOTE] +> remaining conversion tbd, assuming this is on the right track + + Alternatively, you can use the `--export` argument to export a query in the Steampipe snapshot format. This will create a file with a `.sps` extension in the current directory: