-
Notifications
You must be signed in to change notification settings - Fork 1.5k
Implementation of array_intersect
#8081
New issue
Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? # to your account
Conversation
9b050db
to
522f698
Compare
hi @edmondop @jayzhan211 , I have a draft implementation for array_intersect, I use a hashmap to store the first array values and find values that appear in the hashmap. And it works just fine and looks like a general approach? But I encounter a problem for Float32 and Float64, we can not just use |
Perhaps you could use https://docs.rs/arrow-row/latest/arrow_row/? This would have the added benefit of supporting more complex types, e.g. lists of lists, and not adding additional codegen (#7988). The notes I wrote on #6981 (comment) might be helpful for this |
d10a12d
to
793a330
Compare
array_intersect
793a330
to
31dbcb1
Compare
31dbcb1
to
a2cf813
Compare
a2cf813
to
41663a9
Compare
6b7198f
to
f12525c
Compare
f12525c
to
36e0b9b
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Roughly LGTM
@alamb PTAL : ) |
@Veeupup Triggered the ci, please fix it. |
199df3a
to
2d3f0c7
Compare
Signed-off-by: veeupup <code@tanweime.com>
Signed-off-by: veeupup <code@tanweime.com> x
2d3f0c7
to
d55fa60
Compare
@xudong963 hi try fixing ci locally, maybe you can help me trigger it again or review it? : ) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like a nice implementation to me -- thank you @Veeupup @Dandandan @tustvold @xudong963 and @Dandandan -- what a team effort ❤️
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, welcome to arrow-datafusion
Which issue does this PR close?
Closes #6978
Rationale for this change
Using RowConverter to implement
array_intersect
, no duplicate code for different data types,What changes are included in this PR?
Are these changes tested?
Are there any user-facing changes?