-
Notifications
You must be signed in to change notification settings - Fork 1.5k
rewrite array_append/array_prepend
to remove deplicate codes
#8108
New issue
Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? # to your account
Conversation
@alamb @jayzhan211 PTAL ^ . ^ |
Signed-off-by: veeupup <code@tanweime.com>
array_append/array_prepend
to remove deplicate codes
Thanks @Veeupup -- I'll check it out shortly |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you @Veeupup ! Could you also move the related tests to sqllogictest?
My expected way for array_append/array_prepend is building array via MutableArrayData, so I need to think about whether row converter is the better choice or not. |
I wrote another function with MutableArrayData and it is a lot faster. 30000 micro second vs 300 micro second. @Veeupup Can you also try with MutableArrayData and compare it yourself, if you also got the same result, we can go with MutableArrayData approach. |
Sure! I'll make it later |
Appreciate your advice! I'll try this way and benchmark it then |
hi @Weijun-H ! it seems that I do not need to do anything? (Or just remove ut I have checked the |
Signed-off-by: veeupup <code@tanweime.com>
Signed-off-by: veeupup <code@tanweime.com>
@jayzhan211 I have implemented the array_append in MutableArray and RowConverter and it shows that Mutable performs better too. Here are the benchmarks for sqllogictests:
|
Very nice! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you @Veeupup and @jayzhan211 -- this looks a lot nicer to me. I think there is even more performance to be gained here by avoiding concat
(and using ArrayData to build up the final output) but this is a lot better than what we have currently.
Thanks a lot 🙏
Which issue does this PR close?
One issue from #7988
array_append
&&array_prepend
.Rationale for this change
What changes are included in this PR?
Are these changes tested?
Are there any user-facing changes?