Skip to content

PR: Improve push down filter of join #13184 introduces correctness issues #13211

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Closed
eejbyfeldt opened this issue Nov 1, 2024 · 3 comments · Fixed by #13229
Closed

PR: Improve push down filter of join #13184 introduces correctness issues #13211

eejbyfeldt opened this issue Nov 1, 2024 · 3 comments · Fixed by #13229
Labels
bug Something isn't working

Comments

@eejbyfeldt
Copy link
Contributor

eejbyfeldt commented Nov 1, 2024

Describe the bug

Recently merged PR "Improve push down filter of join #13184" introduces correctness issues.

I did not look closely at the code, but the example in the PR description seems incorrect. See reproducer for why.

To Reproduce

> CREATE TABLE t1 (c0 CHAR, c1 INT) AS VALUES ('a', 1);
CREATE TABLE t2 (c0 CHAR, c1 INT);
SELECT * FROM t1 LEFT JOIN t2 ON t1.c0 = t2.c0 WHERE abs(t2.c1) > 5;

0 row(s) fetched. 
Elapsed 0.011 seconds.

0 row(s) fetched. 
Elapsed 0.006 seconds.

+----+----+----+----+
| c0 | c1 | c0 | c1 |
+----+----+----+----+
| a  | 1  |    |    |
+----+----+----+----+

Expected behavior

Before that commit Datafusion correct produced empty output

DataFusion CLI v42.2.0
> CREATE TABLE t1 (c0 CHAR, c1 INT) AS VALUES ('a', 1);
CREATE TABLE t2 (c0 CHAR, c1 INT);
SELECT * FROM t1 LEFT JOIN t2 ON t1.c0 = t2.c0 WHERE abs(t2.c1) > 5;

0 row(s) fetched. 
Elapsed 0.015 seconds.

0 row(s) fetched. 
Elapsed 0.008 seconds.

+----+----+----+----+
| c0 | c1 | c0 | c1 |
+----+----+----+----+
+----+----+----+----+
0 row(s) fetched. 
Elapsed 0.019 seconds.

Additional context

No response

@eejbyfeldt eejbyfeldt added the bug Something isn't working label Nov 1, 2024
@alamb
Copy link
Contributor

alamb commented Nov 1, 2024

FYI @JasonLi-cn and @Dandandan

@Dandandan
Copy link
Contributor

Oops... we need to revert it, will probably do so the weekend if noone else does it.

@eejbyfeldt
Copy link
Contributor Author

Created #13229 for reverting

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants