-
Notifications
You must be signed in to change notification settings - Fork 1.4k
Add Expr::try_as_col
, deprecate Expr::try_into_col
(speed up optimizer)
#10448
New issue
Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? # to your account
Conversation
Thanks @sadboy 🙏 |
Expr::try_as_col
, deprecate Expr::try_into_col
Expr::try_as_col
, deprecate Expr::try_into_col
(speed up optimizer)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Most places seem to still be cloning it, but I guess baby steps 😅
Yes indeed -- baby steps. Thank you for the review @tustvold Another thing that would likely help non trivially would be to introduce a version of |
accumu.insert(l.try_into_col()?); | ||
accumu.insert(r.try_into_col()?); | ||
let Some(l) = l.try_as_col().cloned() else { | ||
return internal_err!( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think making the error here (rather than in try_into_col
) results in a more specific error message that will be easier to debug if we ever hit it
Which issue does this PR close?
Part of #9637
I ran into this while working on #10444
Rationale for this change
There are places in the code that check if an expr is a column by using
Expr::try_into_col()
which has two potential performance problems:Column
(and thus a string) to check if an expr is aExpr::Column
DataFusionError
to check if an expr is not aExpr::Column
What changes are included in this PR?
Expr::try_as_col
Expr::try_into_col
Expr::try_into_col
Are these changes tested?
By existing tests
Are there any user-facing changes?
A new function / less required copying
I doubt we will see any significant performance changes however it does reduce allocations during planning