Skip to content

Statistics::total_byte_size does not account for projection in FileScanConfig::with_projection #14936

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Open
Tracked by #8227
alamb opened this issue Feb 28, 2025 · 1 comment
Labels
bug Something isn't working

Comments

@alamb
Copy link
Contributor

alamb commented Feb 28, 2025

Describe the bug

As @blaginin found in #14685, the statistics when a File is projected (aka only a subset of the columns are present) is incorrect

Specifically, the projected statistics have the same total_byte_size as the input. However, given only a subset of columns are selected this will mean that the total_byte_size should actually be lower

To Reproduce

See tests referenced in #14685

Expected behavior

total_byte_size should take into account the subset of columns

Additional context

No response

@blaginin
Copy link
Contributor

Should be easier to fix after #15352

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants