Skip to content

Query produces different results after physical plan round trip to bytes #14253

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Closed
Tracked by #14008
robtandy opened this issue Jan 23, 2025 · 4 comments · Fixed by #14465
Closed
Tracked by #14008

Query produces different results after physical plan round trip to bytes #14253

robtandy opened this issue Jan 23, 2025 · 4 comments · Fixed by #14465
Assignees
Labels
bug Something isn't working

Comments

@robtandy
Copy link
Contributor

Describe the bug

Serializing a physical plan to bytes and then recreating it produces a different output. This was observed on TPCH query 3.

Note though, that it was also observed that if you remove the config option: datafusion.execution.parquet.pushdown_filters the discrepancy goes away.

I'll work up a PR to improve the round trip tests to hopefully isolate the problem.

To Reproduce

A reproducible example is here
https://github.com/robtandy/datafusion-physical-plan-round-trip/blob/main/src/main.rs

Expected behavior

Physical plans should produce the same output after a round trip to proto and back.

Additional context

No response

@robtandy robtandy added the bug Something isn't working label Jan 23, 2025
@alamb
Copy link
Contributor

alamb commented Jan 27, 2025

This certainly sounds not good (and an excellent find)

@alamb alamb mentioned this issue Jan 27, 2025
32 tasks
@alamb
Copy link
Contributor

alamb commented Jan 27, 2025

Added to the list of things to look at before release on

@blaginin
Copy link
Contributor

blaginin commented Feb 3, 2025

Looks like the way we serialize pushdown_filters is wrong. It's true in the initial struct but false after deserialization

@blaginin
Copy link
Contributor

blaginin commented Feb 3, 2025

take

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants