Skip to content

Eliminate more outer joins by supporting more expressions #13232

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom
Open
Tracked by #15885
Dandandan opened this issue Nov 2, 2024 · 3 comments
Open
Tracked by #15885

Eliminate more outer joins by supporting more expressions #13232

Dandandan opened this issue Nov 2, 2024 · 3 comments
Labels
enhancement New feature or request optimizer Optimizer rules performance Make DataFusion faster

Comments

@Dandandan
Copy link
Contributor

Dandandan commented Nov 2, 2024 •
edited
Loading

Is your feature request related to a problem or challenge?

This PR by @JasonLi-cn tried to support more pushdown of filters #13184 , however the implementation wasn't 100% right as it pushed down more predicates, but didn't change the join type (left to inner).

Describe the solution you'd like

Add support for a wider range of expressions in EliminateOuterJoin, notably in extract_non_nullable_columns.
We lack support of more complex nested expressions, quite some binary operators, UDFs etc. such as:

  • abs(x) > 1
  • a + a > c + b

We should aim to support any operator that returns null when the input value is null (most binary / unary operators).

Describe alternatives you've considered

No response

Additional context

No response
@Dandandan Dandandan added the enhancement New feature or request label Nov 2, 2024
@Dandandan Dandandan mentioned this issue Nov 2, 2024
Merged
@Dandandan Dandandan changed the title Eliminate more outer joins by supporting more expressions, like UDFs Eliminate more outer joins by supporting more expressions Nov 2, 2024
@Dandandan Dandandan added performance Make DataFusion faster optimizer Optimizer rules labels Nov 2, 2024
@Rachelint
Copy link
Contributor

take

@JasonLi-cn JasonLi-cn mentioned this issue Nov 4, 2024
Closed
@JasonLi-cn
Copy link
Contributor

Rachelint

I have implemented a preliminary version and am happy to solve this Issue together. 😄

pr: #13249

@Rachelint Rachelint removed their assignment Nov 4, 2024
@Rachelint
Copy link
Contributor

Rachelint commented Nov 4, 2024 •
edited
Loading

Rachelint

I have implemented a preliminary version and am happy to solve this Issue together. 😄

pr: #13249

😆 OK, I am learning it!

@suibianwanwank suibianwanwank mentioned this issue Mar 15, 2025
Closed
@alamb alamb mentioned this issue Apr 28, 2025
Open
22 tasks
# for free to join this conversation on GitHub. Already have an account? # to comment
Assignees
No one assigned
Labels
enhancement New feature or request optimizer Optimizer rules performance Make DataFusion faster
Projects
None yet
Milestone
No milestone
3 participants
@Dandandan @JasonLi-cn @Rachelint