Skip to content

feat: array_contains #6618

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Merged
merged 7 commits into from
Jun 27, 2023
Merged

feat: array_contains #6618

merged 7 commits into from
Jun 27, 2023

Conversation

izveigor
Copy link
Contributor

@izveigor izveigor commented Jun 9, 2023

Which issue does this PR close?

Closes #6557

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Yes

Are there any user-facing changes?

Yes

@github-actions github-actions bot added core Core DataFusion crate logical-expr Logical plan and expressions physical-expr Changes to the physical-expr crates sqllogictest SQL Logic Tests (.slt) labels Jun 9, 2023
@izveigor izveigor marked this pull request as draft June 9, 2023 18:37
@izveigor izveigor marked this pull request as ready for review June 23, 2023 18:59
@@ -1070,6 +1071,70 @@ pub fn array_ndims(args: &[ColumnarValue]) -> Result<ColumnarValue> {
]))))
}

macro_rules! contains {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't quite understand this comment -- are you saying that it would be better to use the in_list kernel rather than flattening it?

@izveigor
Copy link
Contributor Author

@alamb I wonder if you review this PR if you have free time.

@alamb
Copy link
Contributor

alamb commented Jun 24, 2023

I will find time to review this PR, but maybe not until Monday

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great @izveigor -- thank you very much for this contribution.

I agree with your comments in #6557 that it would be interesting to add operators like @> such as postgres has https://www.postgresql.org/docs/current/functions-array.html. Should we file some follow on tickets to try and crowdsource that work?

@@ -1070,6 +1071,70 @@ pub fn array_ndims(args: &[ColumnarValue]) -> Result<ColumnarValue> {
]))))
}

macro_rules! contains {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't quite understand this comment -- are you saying that it would be better to use the in_list kernel rather than flattening it?

@@ -179,23 +179,24 @@ Unlike to some databases the math functions in Datafusion works the same way as

## Array Expressions
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❤️

@alamb
Copy link
Contributor

alamb commented Jun 26, 2023

The other thing I think would help this PR is some tests for the array version in .slt (rather than just scalar) but I will write those as a follow on PR (as I want to get some experience with these functions myself).

@izveigor
Copy link
Contributor Author

@alamb I think it would be better to create a separate array function ticket with columns (Since I considered only scalars as arguments). (See the tickets: #6693 and #6709).

@alamb
Copy link
Contributor

alamb commented Jun 26, 2023

@alamb I think it would be better to create a separate array function ticket with columns (Since I considered only scalars as arguments). (See the tickets: #6693 and #6709).

Indeed @izveigor -- in fact I found exactly the same bug when writing the tests -- #6771

@alamb alamb merged commit 1dd1fbd into apache:main Jun 27, 2023
@alamb
Copy link
Contributor

alamb commented Jun 27, 2023

Thanks again @izveigor

maxburke pushed a commit to urbanlogiq/arrow-datafusion that referenced this pull request Jul 14, 2023
* feat: array_contains

* feat: regen.sh

* docs: array_contains

* fix: merge

* Update docs/source/user-guide/sql/scalar_functions.md

---------

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
core Core DataFusion crate logical-expr Logical plan and expressions physical-expr Changes to the physical-expr crates sqllogictest SQL Logic Tests (.slt)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

New array method array_contains
2 participants