Skip to content

Commit

Permalink
AMP-120733 add documentation about DBT integration
Browse files Browse the repository at this point in the history
  • Loading branch information
xiaowuc1 committed Dec 18, 2024
1 parent c46e702 commit 1d19816
Showing 1 changed file with 102 additions and 0 deletions.
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
---
id: 06feb1be-15e3-419a-95ea-85a497dc07f5
blueprint: warehouse_native_amplitude
title: 'Warehouse Native DBT Integration'
this_article_will_help_you:
- 'efficiently manage multiple models with DBT integration'
landing: false
exclude_from_sitemap: false
---
The warehouse-native DBT integration helps you manage your warehouse native models.
Instead of managing one model at a time, you can create or edit multiple models
leveraging the DBT manifest file.

## Approach

This section provides details about how to annotate the metadata for the DBT manifest file
to automatically generate all data models for you.

### Supported operations

Warehouse-native Amplitude supports both create and update operations with the manifest file. For each table present in the manifest file, multiple data models can be

Check warning on line 21 in content/collections/warehouse_native_amplitude/en/warehouse-native-dbt-integration.md

View workflow job for this annotation

GitHub Actions / runner / vale

[vale] reported by reviewdog 🐶 [Amplitude.Passive] 'be created' looks like passive voice. Raw Output: {"message": "[Amplitude.Passive] 'be created' looks like passive voice.", "location": {"path": "content/collections/warehouse_native_amplitude/en/warehouse-native-dbt-integration.md", "range": {"start": {"line": 21, "column": 165}}}, "severity": "WARNING"}
created.

To create data models based on a table, add `"amplitude_meta"` as a key to the
metadata for the table. The value associated with the `"amplitude_meta"` key must
be a dictionary with a single key, `"data_models"`. The value associated with
the `"data_models"` key must be a list of data model descriptions.

A data model description is a dictionary with three keys, `"name"`, `"type"`,
and `"special_columns"`. The value associated with the `"name"` key must be a
globally unique string among all data model descriptions, and is used to
identify the specific data model. If this name isn't present among the
data models already present, then a new data model will be created with this name.

Check warning on line 33 in content/collections/warehouse_native_amplitude/en/warehouse-native-dbt-integration.md

View workflow job for this annotation

GitHub Actions / runner / vale

[vale] reported by reviewdog 🐶 [Amplitude.Will] Future tense! Avoid using 'will be'. Raw Output: {"message": "[Amplitude.Will] Future tense! Avoid using 'will be'.", "location": {"path": "content/collections/warehouse_native_amplitude/en/warehouse-native-dbt-integration.md", "range": {"start": {"line": 33, "column": 52}}}, "severity": "WARNING"}

Check warning on line 33 in content/collections/warehouse_native_amplitude/en/warehouse-native-dbt-integration.md

View workflow job for this annotation

GitHub Actions / runner / vale

[vale] reported by reviewdog 🐶 [Amplitude.Passive] 'be created' looks like passive voice. Raw Output: {"message": "[Amplitude.Passive] 'be created' looks like passive voice.", "location": {"path": "content/collections/warehouse_native_amplitude/en/warehouse-native-dbt-integration.md", "range": {"start": {"line": 33, "column": 57}}}, "severity": "WARNING"}
Otherwise, the data model with that name will be overwritten with the specified

Check warning on line 34 in content/collections/warehouse_native_amplitude/en/warehouse-native-dbt-integration.md

View workflow job for this annotation

GitHub Actions / runner / vale

[vale] reported by reviewdog 🐶 [Amplitude.Will] Future tense! Avoid using 'will be'. Raw Output: {"message": "[Amplitude.Will] Future tense! Avoid using 'will be'.", "location": {"path": "content/collections/warehouse_native_amplitude/en/warehouse-native-dbt-integration.md", "range": {"start": {"line": 34, "column": 42}}}, "severity": "WARNING"}
configuration.

The value associated with the `"type"` key defines
the type of data model and must be one of the following:
- `"EVENT"`
- `"EVENT_PROPERTIES"`
- `"CURRENT_USER_PROPERTIES"`
- `"HISTORICAL_USER_PROPERTIES"`
- `"CURRENT_GROUP_PROPERTIES"`
- `"HISTORICAL_GROUP_PROPERTIES"`

The value associated with the `"special_columns"` key must be a dictionary with
specified keys as follows:
| Data Model Type | Keys Expected |
| ------------------------------- | ------------------------------------------- |
| `"EVENT"` | `"unique_id"`, `"event_time"` |
| `"EVENT_PROPERTIES"` | `"event_type"` |
| `"CURRENT_USER_PROPERTIES"` | `"unique_id"` |
| `"HISTORICAL_USER_PROPERTIES"` | `"unique_id"`, `"start_time"`, `"end_time"` |
| `"CURRENT_GROUP_PROPERTIES"` | `"unique_id"` |
| `"HISTORICAL_GROUP_PROPERTIES"` | `"unique_id"`, `"start_time"`, `"end_time"` |

Each key in the dictionary associated with the `"special_columns"` should be associated
with a string corresponding to the key in the `"columns"` dictionary.

{{partial:admonition type="note" heading="What happens to models that aren't configured in the manifest file?"}}
Amplitude ignores and doesn't alter models that aren't in the manifest file.
{{/partial:admonition}}

{{partial:admonition type="note" heading="What happens to tables which don't specify the `amplitude_meta` key?"}}
Amplitude ignores all such tables.
{{/partial:admonition}}

## Example

```json
{
"nodes": {
"event_table": {
"database": "database",
"schema": "schema",
"name": "table_name",
"meta": {
"amplitude_meta": {
"data_models": [
{
"name": "data_model_name",
"type": "EVENT",
"special_columns": {
"unique_id": "unique_id_col",
"event_time": "event_time_col"
}
}
]
}
},
"columns": {
"event_time_col": {
"name": "event_time_col"
},
"unique_id_col": {
"name": "unique_id_col"
}
}
}
}
}
```

0 comments on commit 1d19816

Please # to comment.