Snapshotting with schema changes doesn't work (dbt-databricks)

[pramoddoodala]

Hello,

Having issues with snapshots, i was able to setup snapshots and had success when i ran dbt snapshot multiple times. But it fails when i rename or add a new column on the model, dbt snapshot fails to recognize new column or renamed column

cannot resolve new_table_column_name in INSERT clause given columns

The table is created by a simple model:

select 
     1 as id 
     --,1 as new_table_column_name
     ,current_timestamp as updated_at 
union all
select 
    5 as id
    --,1 as new_table_column_name
    ,current_timestamp as updated_at

and snapshot:

{% snapshot test_snap %}

{{
    config(
      target_schema='snapshots',
      unique_key='id',

      strategy='timestamp',
      updated_at='updated_at'
    )
}}

select * from dbo.my_first_dbt_model

{% endsnapshot %}

I run dbt run followed by dbt snapshot to create the snapshot. Then uncomment the commented row in the original table model to modify the table schema:

select 
     1 as id 
     ,1 as new_table_column_name
     ,current_timestamp as updated_at 
union all
select 
    5 as id
    ,1 as new_table_column_name
    ,current_timestamp as updated_at

Running dbt run and dbt snapshot again produces the aforementioned error. Changing the project profile to connect to a BigQuery database and running the exact same models and procedure produces no errors. The same behaviour is present when using check strategy for the snapshot instead of timestamping (and removing the updated_at column).

Current Setup:

Core:
  - installed: 1.4.5
  - latest:    1.5.0 - Update available!

Plugins:
  - spark:      1.4.1 - Update available!
  - databricks: 1.4.2 - Update available!

[dataders]

this looks like this is a bug to me, @pramoddoodala. I suggest converting this Discussion into an Issue.