Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

BigQueryConnectException should log offsets for failed rows #273

Open
muscovitebob opened this issue Jun 7, 2020 · 0 comments
Open

BigQueryConnectException should log offsets for failed rows #273

muscovitebob opened this issue Jun 7, 2020 · 0 comments

Comments

@muscovitebob
Copy link

muscovitebob commented Jun 7, 2020

Currently when a BigQueryConnectException is thrown, the formatInsertAllErrors printer outputs some information about which rows failed and with what error. Unfortunately the end user cannot correlate these rows at all back to the actual message that failed to insert as I see it. We should propagate the offset information to this exception if possible and print it when this exception is thrown so that users get actionable information. From my understanding this would entail modifying the BigQueryWriter to work with lists of Tuple[Offset, InsertAllRequest.RowToInsert] or some similar structure instead of just the rows, but I would appreciate guidance on how to go about this.

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant