- support Hudi, Delta, Iceberg natively supported in Glue through datalake_format parameter
- upgrade default Glue version to 4.0
- Support delta tables in Athena managed by Lake Formation
- adding support for database AWS Lake Formation tag management
- fix the boto3 version to use lakeformation tags
- Fixed msck repair call for Delta non-partitioned tables
- re-use the glue-session to run multiple dbt run commands
- adding support for AWS Lake Formation tag management
- adding support for AWS Lake Formation data filtering and row, column, cell level security
- add compatibility with dbt 1.6
- fix Snapshot for Hudi
- use session to build client
- enable data skipping using hudi metadata in read path
- enable models to substitute sql with pyspark variables
- fix the naming for query execution timeout
- add assumeRole feature for AWS API Call
- fix get response for insert_overwrite issue
- Replace delete session per stop session
- add compatibility with dbt 1.5
- add multithreading
- add hudi bulk insert shuffle parallelism
- fix hudi snapshots without parameters
- run pre_hooks before create tmp table
- fix typos in sample_profiles.yml and README.md
- add Hudi related extensions #167
- add add execute_pyspark capability for registering python/pyspark UDFs #163
- add compatibility with dbt 1.4.1
- add compatibility with dbt 1.4.0
- Fix HUDI merge strategy
- Fix exception handling for correct DBT report
- Update READEME.md to use Apache Iceberg Connector for AWS Glue v0.14.0
- add ability to override any hudi option
- add details on Iceberg documentation (provide details on commit locking, catalog alias. Provide least privilege IAM Permission for DynamoDB Commit Locking. Correct typos)
- add customizable DynamoDB table name for Iceberg Commit Locking.
- Refactoring of unused --conf parameter for Iceberg file format (warehouse_path)
- Fix error catching for seed feature
- implementation of Iceberg append, merge, and insert_overwrite operation and refacto of the existing create and create or replace implementation.
- add method to create dbt snapshots from Iceberg tables
- update README.md
- add version.py to fix
dbt --version
- improve support for Boto backoff and retry logic
- improvement of seed
- add seed_format and seed_mode to configuration
- Add error checking for HUDI incremental materializations
- Specify location for tmp table
- Add method to create a default location for Iceberg without using final trailing slash. The fix will make Iceberg tables readable from query engines like Trino.
- Add support for Iceberg table materializion, and iceberg_table_replace materializion
- Added default_arguments param for Glue to add custom Glue configuration options.
- Include config
full_refreshflag when materialization is incremental
- Updated dependencies to support dbt-core 1.3.0
- Force database parameter must be omitted or have the same value as schema Github Issue Link
- Fix duplicates when using partitions changes with Hudi/Merge incremental materialization Github Issue Link
- Added a function to add an end space in case of single quote at the end of a query. Ex: WHERE column='foo' Github Issue Link
- #80: Fix default glue version on documentation
- Changing default glue version and fixing a typo. Github Issue Link
- Changing on Readme file the pip and python commands by python3 and pip3. This resolves potential issues when python2 is installed too.
- #45: Modified Connection argument for Glue Session and table relation information for incremental mode
- Modified Connection argument for Glue Session
- Updated get_relation method to return relation as dict instead of list. Github Issue Link
- Added Conf param for Glue to add custom spark configuration options.
- Updated glue.sql.sources.partitionOverwriteMode to spark.sql.sources.partitionOverwriteMode to work partition overwrite properly.
- Override default types for STRING from TEXT to STRING