Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

Fix Miscellaneous Expression tests on Databricks 14.3 #11537

Closed
razajafri opened this issue Sep 27, 2024 · 0 comments · Fixed by #11670
Closed

Fix Miscellaneous Expression tests on Databricks 14.3 #11537

razajafri opened this issue Sep 27, 2024 · 0 comments · Fixed by #11670
Assignees
@mythrocks

Comments

@razajafri
Copy link
Collaborator

Build the plugin against the Databricks 14.3 cluster using #11467. Once built successfully run the Misc Expression tests by TESTS=misc_expr_test.py jenkins/databricks/test.sh

The following tests fail

[gw4] [ 61%] FAILED ../../src/main/python/misc_expr_test.py::test_raise_error
mythrocks added a commit to mythrocks/spark-rapids that referenced this issue Oct 28, 2024
@mythrocks
Fix misc_expr_test for [databricks] 14.3
40639b0 
Fixes NVIDIA#11537.

This commit addresses the failure of the `test_raise_error` test
in `misc_expr_test.py` for Databricks 14.3.

This is an extension of NVIDIA#11129, where this test was skipped for
Apache Spark 4.0.  The failure on Databricks 14.3 shares the same
cause as in Spark 4.0, i.e. a backward-incompatible Spark change
in the signature of RaiseError, as introduced in
https://issues.apache.org/jira/browse/SPARK-44838.

The work to support this change in a Spark-RAPIDS shim will be
tracked in NVIDIA#10969.  This test will be skipped until that work
is completed.

Signed-off-by: MithunR <mithunr@nvidia.com>
@mythrocks mythrocks mentioned this issue Oct 28, 2024
Merged
@mythrocks mythrocks self-assigned this Oct 29, 2024
# for free to join this conversation on GitHub. Already have an account? # to comment
Assignees

@mythrocks mythrocks

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix misc_expr_test for [databricks] 14.3 mythrocks/spark-rapids
2 participants
@mythrocks @razajafri