Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Resumable upload for BigQuery data import #1143

Closed
dwsupplee opened this issue Mar 1, 2016 · 2 comments
Closed

Resumable upload for BigQuery data import #1143

dwsupplee opened this issue Mar 1, 2016 · 2 comments
Assignees
Labels
api: bigquery Issues related to the BigQuery API.

Comments

@dwsupplee
Copy link
Contributor

It looks as though the bigquery api supports resumable upload, but there is only a resumable upload module built for storage.

This may be enough, given that you can import data to bigquery from storage, but I just wanted to check in and see if it made sense to include resumable upload support directly for bigquery as well.

@stephenplusplus stephenplusplus added enhancement api: bigquery Issues related to the BigQuery API. labels Mar 3, 2016
@stephenplusplus
Copy link
Contributor

I think the Storage -> BigQuery import is a better alternative here. Resumable uploads come with a lot of overhead in the implementation as well as on the user-- persisting a config file is unexpected.

Anyone feel free to discuss and re-open if this is something we need to support.

@jscinoz
Copy link

jscinoz commented Jan 24, 2017

Is there any reason that this can't be done under the hood as part of table.createWriteStream from the Bigquery module, just as it's done for file.createWriteStream in the GCS module?

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
api: bigquery Issues related to the BigQuery API.
Projects
None yet
Development

No branches or pull requests

3 participants