Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

[SPARK-39857][SQL] V2ExpressionBuilder uses the wrong LiteralValue data type for In predicate #6180

Closed
amahussein opened this issue Aug 1, 2022 · 0 comments
Labels
audit_3.3.0 Audit related tasks for 3.3.0 audit_3.4.0 Audit related tasks for 3.4.0 P1 Nice to have for release

Comments

@amahussein
Copy link
Collaborator

amahussein commented Aug 1, 2022

Spark-context

apache/spark@6b2492628c

What changes were proposed in SPARK pull request?

There is a bug where InSet.dataType (which is BooleanType) is used to build the LiteralValue while building V2 In Predicate .

The new PR fixes the bug by using InSet.child.dataType

Why are the changes needed in Spark?

Bug Fix.

Does this PR introduce any user-facing change?

  • No

Why it might affect RAPIDS?

  • GpuInSet.sql() and GpuInSet.toString() seem to be doing the right thing.
  • If the plugin did not have the bug, then the question why we did not see that bug in Spark before?
    If this is a legit concern, then we may need to investigate if more tests are needed to catch such difference in behavior between GPU and CPU.

Impact on Testing?

Maybe more improvement to tests?

Requires Doc update?

No.

@amahussein amahussein added ? - Needs Triage Need team to review and classify audit_3.3.0 Audit related tasks for 3.3.0 audit_3.4.0 Audit related tasks for 3.4.0 labels Aug 1, 2022
@amahussein amahussein removed the ? - Needs Triage Need team to review and classify label Aug 2, 2022
@amahussein amahussein reopened this Aug 2, 2022
@amahussein amahussein added the ? - Needs Triage Need team to review and classify label Aug 2, 2022
@sameerz sameerz added P1 Nice to have for release and removed ? - Needs Triage Need team to review and classify labels Aug 2, 2022
@jlowe jlowe closed this as not planned Won't fix, can't repro, duplicate, stale Mar 7, 2024
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
audit_3.3.0 Audit related tasks for 3.3.0 audit_3.4.0 Audit related tasks for 3.4.0 P1 Nice to have for release
Projects
None yet
Development

No branches or pull requests

3 participants