Skip to content

Update the parquet code prune_pages_in_one_row_group to use the StatisticsExtractor #11480

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom
Closed
Tracked by #10922
alamb opened this issue Jul 15, 2024 · 0 comments · Fixed by #11483
Closed
Tracked by #10922

Update the parquet code prune_pages_in_one_row_group to use the StatisticsExtractor #11480

alamb opened this issue Jul 15, 2024 · 0 comments · Fixed by #11483
Assignees
@alamb

Comments

@alamb
Copy link
Contributor

alamb commented Jul 15, 2024 •
edited
Loading

Part of #10922

Part of the new statistics extractor code we are moving upstream to arrow-rs helps extracting datapage statistics.

In order to ensure we have the right API we should also update the code in prune_pages_in_one_row_group (source) to use the new StatisticsExtractor
@alamb alamb mentioned this issue Jul 15, 2024
Closed
23 tasks
@alamb alamb self-assigned this Jul 15, 2024
@alamb alamb changed the title Update the parquet code prune_pages_in_one_row_group (source) to use the new StatisticsExtractor code Update the parquet code prune_pages_in_one_row_group to use the StatisticsExtractor Jul 15, 2024
This was referenced Jul 15, 2024
Merged
Merged
Merged
Merged
# for free to join this conversation on GitHub. Already have an account? # to comment
Assignees

@alamb alamb

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Update parquet page pruning code to use the StatisticsExtractor alamb/datafusion
1 participant
@alamb