Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

[ADAP-1012] Support for new agate data type in Spark #935

Closed
gshank opened this issue Nov 7, 2023 · 0 comments · Fixed by #936
Closed

[ADAP-1012] Support for new agate data type in Spark #935

gshank opened this issue Nov 7, 2023 · 0 comments · Fixed by #936

Comments

@gshank
Copy link
Contributor

gshank commented Nov 7, 2023

dbt Core changes

To support integers in the "show" command, a new agate data_type, Integer, was added in dbt Core in core/dbt/clients/agate_helper.py. The agate table creation code is also used when creating empty tables for seeds, such as when a seed file has headers but doesn't have any rows. Because the "convert_agate_type" method did not include the new Integer type, "None" was returned and the sql generated by the macro "create_csv_table" was incorrect.

dbt Core pull request dbt-labs/dbt-core#9004 adds a new "convert_integer_type" method and adds Integer to the list of agate types in "convert_agate_type".

Adapter changes

This pull request will pull in the new empty seed test and provide a "convert_integer_type" method.

@github-actions github-actions bot changed the title Support for new agate data type in Spark [ADAP-1012] Support for new agate data type in Spark Nov 7, 2023
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant