Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Get an Exception when reading Avro File #1729

Open
richiesgr opened this issue Nov 30, 2020 · 0 comments
Open

Get an Exception when reading Avro File #1729

richiesgr opened this issue Nov 30, 2020 · 0 comments

Comments

@richiesgr
Copy link

Hi
I would like to test writing data as Avro instead of parquet
I've configured Secor to write Avro file
After some time I get and the process is stopped

java.lang.RuntimeException: Failed to apply upload policy
	at com.pinterest.secor.consumer.Consumer.checkUploadPolicy(Consumer.java:198)
	at com.pinterest.secor.consumer.Consumer.run(Consumer.java:179)
Caused by: java.util.NoSuchElementException
	at org.apache.avro.file.DataFileStream.next(DataFileStream.java:249)
	at org.apache.avro.file.DataFileStream.next(DataFileStream.java:235)
	at com.pinterest.secor.io.impl.AvroFileReaderWriterFactory$AvroFileReader.next(AvroFileReaderWriterFactory.java:101)
	at com.pinterest.secor.uploader.Uploader.trim(Uploader.java:228)
	at com.pinterest.secor.uploader.Uploader.trimFiles(Uploader.java:272)
	at com.pinterest.secor.uploader.Uploader.checkTopicPartition(Uploader.java:335)
	at com.pinterest.secor.uploader.Uploader.applyPolicy(Uploader.java:363)
	at com.pinterest.secor.consumer.Consumer.checkUploadPolicy(Consumer.java:196)

meaning the reader try to read the record in the file and it's empty
Triggered by
public D next(D reuse) throws IOException { if (!hasNext()) throw new NoSuchElementException(); D result = reader.read(reuse, datumIn); if (0 == --blockRemaining) { blockFinished(); } return result; }

Any help thanks

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant