Skip to content

Commit

Permalink
Add row enirchment info and remove unsupported section from plugin ma…
Browse files Browse the repository at this point in the history
…nagement guide
  • Loading branch information
cbruno10 committed Feb 6, 2025
1 parent 878a264 commit f2d6f5c
Show file tree
Hide file tree
Showing 2 changed files with 19 additions and 23 deletions.
27 changes: 5 additions & 22 deletions docs/manage/plugin.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,15 +26,15 @@ For example, to install the 0.118.0 version of the aws plugin:
$ tailpipe plugin install aws@0.118.0
```

This will download the aws plugin version 0.118.0 (the one with the `0.118.0` tag) from the Hub registry.
This will download the aws plugin version 0.118.0 (the one with the `0.118.0` tag) from the Hub registry.

## Installing from a SemVer Constraint

Plugins should follow [semantic versioning](https://semver.org/) guidelines, and they are tagged in the Hub registry with a **version tag** that specifies their *exact version* in the `major.minor.patch` format (e.g. `1.0.1`).

The intent of the version tag is that it is immutable - while it is technically possible to move the version tag to a different image version, this should not be done.

Installing with a semver constraint allows you to "lock" (or pin) to a specific set of releases which match the contraints.
Installing with a semver constraint allows you to "lock" (or pin) to a specific set of releases which match the contraints.

If you install via `tailpipe plugin install aws@^1`, for example, `tailpipe plugin update` (and auto-updates) will only update to versions greater than `1.0.0` but less than `2.0.0`.

Expand Down Expand Up @@ -72,7 +72,7 @@ $ tailpipe plugin install aws@2.x.x
```

- To install the latest version locked to a specific minor version:
```bash
```bash
$ tailpipe plugin install aws@~2.1
# or
$ tailpipe plugin install aws@2.1.x
Expand All @@ -86,23 +86,6 @@ Tailpipe plugins are packaged in OCI format and can be hosted and installed from
$ tailpipe plugin install us-docker.pkg.dev/myproject/myrepo/myplugin@mytag
```

## Installing from a File

A plugin binary can be installed manually, and this is often convenient when developing the plugin. Tailpipe will attempt to load any plugin that is referred to in a `connection` configuration:
- The plugin binary file must have a `.plugin` extension
- The plugin binary must reside in a subdirectory of the `~/.tailpipe/plugins/` directory and must be the ONLY `.plugin` file in that subdirectory
- The `connection` must specify the path (relative to `~/.tailpipe/plugins/`) to the plugin in the `plugin` argument

For example, consider a `myplugin` plugin that you have developed. To install it:
- Create a subdirectory `.tailpipe/plugins/local/myplugin`
- Name your plugin binary `myplugin.plugin`, and copy it to `.tailpipe/plugins/local/myplugin/myplugin.plugin`
- Create a `~/.tailpipe/config/myplugin.tpc` config file containing a connection definition that points to your plugin:
```hcl
connection "myplugin" {
plugin = "local/myplugin"
}
```
## Viewing Installed Plugins
You can list the installed plugins with the `tailpipe plugin list` command:

Expand All @@ -120,7 +103,7 @@ To update a plugin to the latest version for a given stream, you can use the `t

```
tailpipe plugin update plugin_name[@stream]
```
```

The syntax and semantics are identical to the install command - `tailpipe plugin update aws` will get the latest aws plugin, `tailpipe plugin update aws@1` will get the latest in the 1.x major stream, etc.

Expand All @@ -136,7 +119,7 @@ You can uninstall a plugin with the `tailpipe plugin uninstall` command:

```
tailpipe plugin uninstall [plugin]
```
```

## Tailpipe Plugin Registry Support Lifecycle

Expand Down
15 changes: 14 additions & 1 deletion docs/reference/develop/plugin-release-checklist.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,17 @@ Every table and column has a description. These are consistent across tables. Th

## Table and Column Design

<input type="checkbox"/> <b>Row enrichment</b>

The table enriches the row with the following required [common fields](https://tailpipe.io/docs/manage/table#common-fields):
- `tp_date` - The date the event was originally generated.
- `tp_id` - A unique identifier for the row. In Turbot plugins, typically set to an [xid](https://github.com/rs/xid).
- `tp_index` - The index used to partition the data, e.g., AWS account ID, GitHub organization, hostname.
- `tp_ingest_timestamp` - The timestamp when the event was ingested into the system.
- `tp_timestamp` - The timestamp when the event was originally generated.

Additional common fields, like `tp_ips` and `tp_source_location` should be added based on what is available in the log type.

<input type="checkbox"/> <b>Default file layout</b>

For each supported artifact source, the table defines a default `FileLayout` that uses the most common pattern. For instance, the `aws_cloudtrail_log` uses the standard AWS log format as the default `FileLayout` for the `aws_s3_bucket` source as part of its [GetSourceMetadata function](https://github.com/turbot/tailpipe-plugin-aws/blob/6b2620d49330aff84f8879e2740fabb39fc87b79/tables/cloudtrail_log/cloudtrail_log_table.go#L26-L28).
Expand Down Expand Up @@ -144,7 +155,9 @@ The color matches the provider's brand guidelines, typically stated on a page li

The description in `docs/index.md` is appropriate for the provider. The [AWS plugin](https://hub.tailpipe.io/plugins/turbot/aws), for example, uses:

> AWS provides on-demand cloud computing platforms and APIs to authenticated customers on a metered pay-as-you-go basis.
```
AWS provides on-demand cloud computing platforms and APIs to authenticated customers on a metered pay-as-you-go basis.
```

The opening sentence of the Wikipedia page for the provider can be a good source of guidance here.

Expand Down

0 comments on commit f2d6f5c

Please # to comment.