Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Automatically check if uploaded files are avro or not. #41

Open
hfjn opened this issue Sep 5, 2019 · 1 comment
Open

Automatically check if uploaded files are avro or not. #41

hfjn opened this issue Sep 5, 2019 · 1 comment
Assignees
Labels
good first issue Good for newcomers

Comments

@hfjn
Copy link
Contributor

hfjn commented Sep 5, 2019

What happens if I forget to set this flag but the data is avro? Wouldn't it be possible to detect avro or not automatically based on the first few lines?

Originally posted by @hfjn in #39

@MrTrustworthy
Copy link
Contributor

Avro files have a header, see https://avro.apache.org/docs/1.8.1/spec.html#Object+Container+Files
I think in pyconnect we read something from the avro message struct to make sure it's avro or something... maybe check there too

@hfjn hfjn added the good first issue Good for newcomers label Sep 23, 2019
@annasweb annasweb self-assigned this Oct 4, 2019
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

3 participants