Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Add support for Snowflake database #108

Closed
wants to merge 47 commits into from
Closed

Add support for Snowflake database #108

wants to merge 47 commits into from

Conversation

mbobrovskyi
Copy link
Contributor

No description provided.

@mbobrovskyi mbobrovskyi reopened this Dec 18, 2017
Copy link

@BenFradet BenFradet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good 👍 , thanks for the PR!

I just have a couple of remarks / questions.

affected += aff
}
}
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

problem here is that we only return the last error, what we can do to remedy that is either to:

  • not split on ; if that's possible and execute only one query with all the different statements
  • accumulate the errors
  • short-circuit on the first error

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's snowflake specific - it performs only one statement per request, so we should loop through all the statements. What i've done - i added a circuit on first error.

}
}

return QueryStatus{query, query.Path, int(affected), err}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we rather change the type of affected in QueryStatus to int64?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, I don't think so, int64 is just snowflakes' driver type of affected rows, I think we may do a type assertion to int to not change the other codebase.

log.Fatal(err)
}

fmt.Println("CONFIG:", configStr)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe we should remove this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, i did this, it's unnecessary here

@alexanderdean
Copy link
Member

alexanderdean commented Dec 19, 2017

Thanks @BenFradet - if you could action Ben's feedback @mbobrovsky that would be great.

@coveralls
Copy link

Coverage Status

Coverage decreased (-2.1%) to 30.483% when pulling 1d01e5b on mbobrovsky:snowflake-support into 0ae72da on snowplow:master.

@snowplow snowplow deleted a comment from coveralls Dec 20, 2017
@snowplow snowplow deleted a comment from coveralls Dec 20, 2017
aff, _ := res.RowsAffected()
affected += aff
}
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we return an error if len(strings.TrimSpace(script)) <= 0?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so.
Should we?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's always good to signal to the user that her/his input is malformed instead of silently dismissing it, no?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, we can notify about the empty query string, or empty query file, but should we do it inside RunQuery() method?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On second thought, this is just a matter of the last split being empty isn't it? In this case I think it's fine.

@alexanderdean
Copy link
Member

@BenFradet am I right in thinking there are still changes for @mbobrovsky to make before you give the green tick?

@BenFradet
Copy link

@alexanderdean nope, just changed to approved

@alexanderdean alexanderdean added this to the Version 0.6.0 milestone Dec 24, 2017
@alexanderdean
Copy link
Member

Thanks scheduling.

@alexanderdean alexanderdean changed the title Snowflake support Add database support for Snowflake Dec 24, 2017
@alexanderdean alexanderdean changed the title Add database support for Snowflake Add support for Snowflake database Dec 24, 2017
@alexanderdean alexanderdean self-requested a review December 24, 2017 23:54
@alexanderdean
Copy link
Member

alexanderdean commented Dec 24, 2017

Hey @mbobrovsky - I started to integrate this for UA testing before I noticed that Travis is failing:

https://travis-ci.org/snowplow/sql-runner/builds/318704666?utm_source=github_status&utm_medium=notification

The integration test suite is still using Golang 1.6 and 1.7, whereas it looks like the gosnowflake driver is only tested with 1.8 and 1.9:

https://github.com/snowflakedb/gosnowflake/blob/master/.travis.yml

You will need to update your PR so that tests pass (obviously). I've created a couple of tickets for you:

To ensure a clean git history, you must add these into your branch before the current Snowflake commit.

Does that make sense? Let me know if any questions.

@mbobrovskyi
Copy link
Contributor Author

@alexanderdean What exactly should I do? As I undestood you commited changes on #110 , but not on #112 . Also, I can't figure out what branch is it on.
So should I just update my project to this commit with updated golang version and than add all the commits I've made before for snowflake suport?
Please, let me know what branch I should update from.

@alexanderdean
Copy link
Member

alexanderdean commented Dec 25, 2017

Hey @mbobrovsky:

Thanks!

@alexanderdean
Copy link
Member

alexanderdean commented Dec 25, 2017

I believe godep update -goversion is what you need as part of #112 (from an environment running 1.9.2)

@coveralls
Copy link

coveralls commented Dec 26, 2017

Coverage Status

Coverage decreased (-2.1%) to 30.483% when pulling 64243fa on mbobrovsky:snowflake-support into 0ae72da on snowplow:master.

@BenFradet
Copy link

@mbobrovsky travis passes 👍 , could you rebase so that only 3 commits (corresponding to the 3 issues) are left:

  • Remove Go versions 1.6 and 1.7 from .travis.yml (closes #110)
  • Bump Go version to 1.9.2 (closes #112)
  • Add support for Snowflake database (closes #114)

@coveralls
Copy link

coveralls commented Dec 26, 2017

Coverage Status

Coverage decreased (-2.1%) to 30.483% when pulling 716f7b3 on mbobrovsky:snowflake-support into 0ae72da on snowplow:master.

@mbobrovskyi
Copy link
Contributor Author

@BenFradet Did I do right?

@alexanderdean
Copy link
Member

Rebase to only 3 commits please

@mbobrovskyi
Copy link
Contributor Author

Let's I create a new branch, and then create a new PR, it will be easier

@alexanderdean
Copy link
Member

Go for it, closing this one

# for free to join this conversation on GitHub. Already have an account? # to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants