Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Automatically create database if it doesn't exist #24

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

rabidaudio
Copy link

With Meltano, I like to namespace each extractor to a new database to avoid conflicts (like tap-github and tap-shopify both having users tables) by setting the database name dynamically:

config:
  athena_database: $MELTANO_EXTRACTOR_NAMESPACE

However this means the database needs to exist before the tap can be run.

This change runs CREATE DATABASE IF NOT EXISTS before uploading.

Because meltano/sdk#246 hasn't been implemented, there's not a clean place to put a setup step to run once before the whole target run. Instead this will run for each sink in the constructor, which is not ideal but it should be fine.

@ndrluis
Copy link
Member

ndrluis commented Jan 6, 2023

Hello @rabidaudio Thank you for your contribution.

In our use case at Gupy we have the expectation to create the database manually, because of that I think that would be nice to have a setting to enable this feature.

What do you think about it?

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants