Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

feat: Add Aurora Data API engine spec #16535

Merged
merged 2 commits into from
Sep 4, 2021

Conversation

betodealmeida
Copy link
Member

SUMMARY

Add a DB engine spec for the Aurora Data API.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

Querying a table:

Screen Shot 2021-08-31 at 4 35 31 PM

I also fixed Exasol and Drill not showing up:

Screen Shot 2021-08-31 at 4 51 48 PM

TESTING INSTRUCTIONS

To add an Aurora PostgreSQL database:

Configure the SQLAlchemy URI: postgresql+auroradataapi://{aws access id}:{aws secret key}@/{db_name}
Configure the engine parameters in "Other":

{
  "connect_args": {
    "aurora_cluster_arn": "{cluster ARN}",
    "secret_arn": "{secret ARN}",
    "region_name": "us-east-2"
  }
}

ADDITIONAL INFORMATION

  • Has associated issue:
  • Required feature flags:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

@betodealmeida betodealmeida changed the title WIP feat: Add Aurora Data API engine spec Aug 31, 2021
Copy link
Member

@eschutho eschutho left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is great!

@codecov
Copy link

codecov bot commented Sep 1, 2021

Codecov Report

Merging #16535 (f5c280c) into master (5f8dff1) will decrease coverage by 0.21%.
The diff coverage is 82.35%.

❗ Current head f5c280c differs from pull request most recent head 4bd20a0. Consider uploading reports for the commit 4bd20a0 to get more accurate results
Impacted file tree graph

@@            Coverage Diff             @@
##           master   #16535      +/-   ##
==========================================
- Coverage   76.71%   76.50%   -0.22%     
==========================================
  Files        1002     1003       +1     
  Lines       53823    53836      +13     
  Branches     7287     7287              
==========================================
- Hits        41292    41187     -105     
- Misses      12291    12409     +118     
  Partials      240      240              
Flag Coverage Δ
hive ?
mysql 81.55% <82.35%> (+0.04%) ⬆️
postgres 81.54% <82.35%> (-0.04%) ⬇️
presto ?
python 81.67% <82.35%> (-0.42%) ⬇️
sqlite 81.23% <82.35%> (+0.04%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
superset/db_engine_specs/rockset.py 53.12% <ø> (+1.60%) ⬆️
superset/db_engine_specs/__init__.py 50.00% <25.00%> (ø)
superset/db_engine_specs/aurora.py 100.00% <100.00%> (ø)
superset/models/slice.py 86.41% <100.00%> (ø)
superset/db_engines/hive.py 0.00% <0.00%> (-82.15%) ⬇️
superset/db_engine_specs/hive.py 69.80% <0.00%> (-16.87%) ⬇️
superset/db_engine_specs/presto.py 83.47% <0.00%> (-6.91%) ⬇️
superset/views/database/mixins.py 81.03% <0.00%> (-1.73%) ⬇️
superset/connectors/sqla/models.py 88.04% <0.00%> (-1.66%) ⬇️
superset/db_engine_specs/base.py 88.00% <0.00%> (-0.39%) ⬇️
... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5f8dff1...4bd20a0. Read the comment docs.

@betodealmeida betodealmeida added the need:merge The PR is ready to be merged label Sep 2, 2021
@betodealmeida betodealmeida merged commit 359383b into apache:master Sep 4, 2021
villebro pushed a commit to preset-io/superset that referenced this pull request Sep 6, 2021
* feat: Add Aurora Data API engine spec

* Fix lint
opus-42 pushed a commit to opus-42/incubator-superset that referenced this pull request Nov 14, 2021
* feat: Add Aurora Data API engine spec

* Fix lint
villebro added a commit that referenced this pull request Nov 15, 2021
* add support for adhoc columns to api and sqla model

* fix some types

* fix duplicates in column names

* fix more lint

* fix schema and dedup

* clean up some logic

* first pass at fixing viz.py

* Add frontend support for adhoc columns

* Add title edit

* Fix showing custom title

* Use column name as default value in sql editor

* fix: Adds a loading message when needed in the Select component (#16531)

* fix(tests): make parquet select deterministic with order by (#16570)

* bump emotion to help with cache clobbering (#16559)

* fix: Support Jinja template functions in global async queries (#16412)

* Support Jinja template functions in async queries

* Pylint

* Add tests for async tasks

* Remove redundant has_request_context check

* fix: impersonate user label/tooltip (#16573)

* docs: update for small typos (#16568)

* feat: Add Aurora Data API engine spec (#16535)

* feat: Add Aurora Data API engine spec

* Fix lint

* refactor: sql_json view endpoint: encapsulate ctas parameters (#16548)

* refactor sql_json view endpoint: encapsulate ctas parameters

* fix failed tests

* fix failed tests and ci issues

* refactor sql_json view endpoint: separate concern into ad hod method (#16595)

* feat: Experimental cross-filter plugins (#16594)

* fix:fix get permission function

* feat: add cross filter chart in charts gallery under FF

* chore(deps): bump superset-ui to 0.18.2 (#16601)

* update type guard references

* fix imports

* update series_columns schema

* Add changes that got lost in rebase

* Use current columns name or expression as sql editor init value

* add integration test and do minor fixes

* Bump superset-ui

* fix linting issue

* bump superset-ui to 0.18.22

* resolve merge conflict

* lint

* fix select filter infinite loop

* bump superset-ui to 0.18.23

* Fix auto setting column popover title

* Enable adhoc columns only if UX_BETA enabled

* put back removed test

* Move popover height and width to constants

* Refactor big ternary expression

Co-authored-by: Kamil Gabryjelski <kamil.gabryjelski@gmail.com>
Co-authored-by: Michael S. Molina <70410625+michael-s-molina@users.noreply.github.com>
Co-authored-by: Elizabeth Thompson <eschutho@gmail.com>
Co-authored-by: Rob DiCiuccio <rob.diciuccio@gmail.com>
Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
Co-authored-by: joeADSP <75027008+joeADSP@users.noreply.github.com>
Co-authored-by: ofekisr <35701650+ofekisr@users.noreply.github.com>
Co-authored-by: simcha90 <56388545+simcha90@users.noreply.github.com>
QAlexBall pushed a commit to QAlexBall/superset that referenced this pull request Dec 28, 2021
* feat: Add Aurora Data API engine spec

* Fix lint
AAfghahi pushed a commit that referenced this pull request Jan 10, 2022
* add support for adhoc columns to api and sqla model

* fix some types

* fix duplicates in column names

* fix more lint

* fix schema and dedup

* clean up some logic

* first pass at fixing viz.py

* Add frontend support for adhoc columns

* Add title edit

* Fix showing custom title

* Use column name as default value in sql editor

* fix: Adds a loading message when needed in the Select component (#16531)

* fix(tests): make parquet select deterministic with order by (#16570)

* bump emotion to help with cache clobbering (#16559)

* fix: Support Jinja template functions in global async queries (#16412)

* Support Jinja template functions in async queries

* Pylint

* Add tests for async tasks

* Remove redundant has_request_context check

* fix: impersonate user label/tooltip (#16573)

* docs: update for small typos (#16568)

* feat: Add Aurora Data API engine spec (#16535)

* feat: Add Aurora Data API engine spec

* Fix lint

* refactor: sql_json view endpoint: encapsulate ctas parameters (#16548)

* refactor sql_json view endpoint: encapsulate ctas parameters

* fix failed tests

* fix failed tests and ci issues

* refactor sql_json view endpoint: separate concern into ad hod method (#16595)

* feat: Experimental cross-filter plugins (#16594)

* fix:fix get permission function

* feat: add cross filter chart in charts gallery under FF

* chore(deps): bump superset-ui to 0.18.2 (#16601)

* update type guard references

* fix imports

* update series_columns schema

* Add changes that got lost in rebase

* Use current columns name or expression as sql editor init value

* add integration test and do minor fixes

* Bump superset-ui

* fix linting issue

* bump superset-ui to 0.18.22

* resolve merge conflict

* lint

* fix select filter infinite loop

* bump superset-ui to 0.18.23

* Fix auto setting column popover title

* Enable adhoc columns only if UX_BETA enabled

* put back removed test

* Move popover height and width to constants

* Refactor big ternary expression

Co-authored-by: Kamil Gabryjelski <kamil.gabryjelski@gmail.com>
Co-authored-by: Michael S. Molina <70410625+michael-s-molina@users.noreply.github.com>
Co-authored-by: Elizabeth Thompson <eschutho@gmail.com>
Co-authored-by: Rob DiCiuccio <rob.diciuccio@gmail.com>
Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
Co-authored-by: joeADSP <75027008+joeADSP@users.noreply.github.com>
Co-authored-by: ofekisr <35701650+ofekisr@users.noreply.github.com>
Co-authored-by: simcha90 <56388545+simcha90@users.noreply.github.com>
@mistercrunch mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 1.4.0 labels Mar 13, 2024
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels need:merge The PR is ready to be merged size/M 🚢 1.4.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants