Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

[BUG] get_json_test.py::test_get_json_object_quoted_question fails on Spark 4 with mismatched output #11130

Open
mythrocks opened this issue Jul 3, 2024 · 0 comments
Assignees
Labels
bug Something isn't working Spark 4.0+ Spark 4.0+ issues

Comments

@mythrocks
Copy link
Collaborator

get_json_test.py::test_get_json_object_quoted_question fails on Spark 4 with mismatched output:

---------------------------- Captured stderr setup -----------------------------
2024-07-02 23:57:30 INFO     Running test 'src/main/python/get_json_test.py::test_get_json_object_quoted_question[DATAGEN_SEED=1719964646, TZ=UTC]'
------------------------------ Captured log setup ------------------------------
INFO     __pytest_worker_logger__:spark_init_internal.py:256 Running test 'src/main/python/get_json_test.py::test_get_json_object_quoted_question[DATAGEN_SEED=1719964646, TZ=UTC]'
----------------------------- Captured stdout call -----------------------------
### CPU RUN ###
### GPU RUN ###
### COLLECT: GPU TOOK 0.5124051570892334 CPU TOOK 1.7997944355010986 ###
--- CPU OUTPUT
+++ GPU OUTPUT
@@ -1 +1 @@
-Row(question='QUESTION')
+Row(question=None)

Seems to work alright on prior Spark versions. We'll need this functionality sorted out before we get a clean run in Spark 4.

I'm disabling this test as part of #11129.

@mythrocks mythrocks added bug Something isn't working ? - Needs Triage Need team to review and classify labels Jul 3, 2024
@mattahrens mattahrens added Spark 4.0+ Spark 4.0+ issues and removed ? - Needs Triage Need team to review and classify labels Jul 9, 2024
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
bug Something isn't working Spark 4.0+ Spark 4.0+ issues
Projects
None yet
Development

No branches or pull requests

3 participants