Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

Ingest Methodology #377

Closed
vforgione opened this issue Jul 23, 2018 · 0 comments
Closed

Ingest Methodology #377

vforgione opened this issue Jul 23, 2018 · 0 comments
Labels
Epic Organizes stories into larger objectives.

Comments

@vforgione
Copy link
Member

Currently we are only working with one ingest method: download the entire source, load it, truncate existing tables/views, and swapping the new version.

This is a weak point when considering large data sets. We need to consider different sources, source document types, sizes, relative speed and stability of loading the sources.
@vforgione vforgione added the Epic Organizes stories into larger objectives. label Jul 23, 2018
This was referenced Jul 23, 2018
Closed
Closed
# for free to join this conversation on GitHub. Already have an account? # to comment
Assignees
No one assigned
Labels
Epic Organizes stories into larger objectives.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@vforgione