Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

[BUG] Spark-4.0: Test failure in DecimalBinaryOpSuite #12007

Open
Tracked by #12001
nartal1 opened this issue Jan 23, 2025 · 1 comment
Open
Tracked by #12001

[BUG] Spark-4.0: Test failure in DecimalBinaryOpSuite #12007

nartal1 opened this issue Jan 23, 2025 · 1 comment
Labels
bug Something isn't working Spark 4.0+ Spark 4.0+ issues test Only impacts tests

Comments

@nartal1
Copy link
Collaborator

nartal1 commented Jan 23, 2025

Describe the bug
Below unit test(scala test) is failing on Spark-4.0 in DecimalBinaryOpSuite:

- SPARK-24957: average with decimal followed by aggregation returning wrong result *** FAILED ***
  java.lang.IllegalArgumentException: Part of the plan is not columnar class org.apache.spark.sql.execution.aggregate.HashAggregateExec
HashAggregate(keys=[text#85], functions=[sum(avg_res#87)], output=[text#85, sum(avg_res)#94])
+- HashAggregate(keys=[text#85], functions=[partial_sum(avg_res#87)], output=[text#85, sum#105, isEmpty#106])
   +- HashAggregate(keys=[text#85], functions=[avg(number#86)], output=[text#85, avg_res#87])
      +- HashAggregate(keys=[text#85], functions=[partial_avg(number#86)], output=[text#85, sum#109, count#110L])
         +- Exchange SinglePartition, REPARTITION_BY_NUM, [plan_id=120]
            +- LocalTableScan [text#85, number#86]
  at com.nvidia.spark.rapids.GpuTransitionOverrides.assertIsOnTheGpu(GpuTransitionOverrides.scala:671)

Disabling ANSI mode would fix this test failure in the short term.

Steps/Code to reproduce bug

$  mvn package -f scala2.13/pom.xml -pl tests -am -Dbuildver=400 -Dsuffixes=".*DecimalBinaryOpSuite"

@nartal1 nartal1 added ? - Needs Triage Need team to review and classify bug Something isn't working Spark 4.0+ Spark 4.0+ issues test Only impacts tests labels Jan 23, 2025
@nartal1 nartal1 changed the title [BUG] Spark-4.0: Tests failures in DecimalBinaryOpSuite [BUG] Spark-4.0: Test failure in DecimalBinaryOpSuite Jan 23, 2025
@mattahrens mattahrens removed the ? - Needs Triage Need team to review and classify label Jan 29, 2025
@nartal1
Copy link
Collaborator Author

nartal1 commented Jan 29, 2025

We will have to skip these tests for Spark-4.0. i.e disable ANSI mode for these failing tests.
We need to enable the tests once #5114 if fixed. Mention the issue number to track.

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
bug Something isn't working Spark 4.0+ Spark 4.0+ issues test Only impacts tests
Projects
None yet
Development

No branches or pull requests

2 participants