Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

[BUG] GetJsonObject fails at spark unit test $.store.basket[0][*].b #10217

Closed
revans2 opened this issue Jan 18, 2024 · 2 comments · Fixed by #10581
Closed

[BUG] GetJsonObject fails at spark unit test $.store.basket[0][*].b #10217

revans2 opened this issue Jan 18, 2024 · 2 comments · Fixed by #10581
Assignees
Labels
bug Something isn't working

Comments

@revans2
Copy link
Collaborator

revans2 commented Jan 18, 2024

Describe the bug
In the spark unit test for GetJsonObject with a path of $.store.basket[0][*].b and data of

  val json =
    """
      |{"store":{"fruit":[{"weight":8,"type":"apple"},{"weight":9,"type":"pear"}],
      |"basket":[[1,2,{"b":"y","a":"x"}],[3,4],[5,6]],"book":[{"author":"Nigel Rees",
      |"title":"Sayings of the Century","category":"reference","price":8.95},
      |{"author":"Herman Melville","title":"Moby Dick","category":"fiction","price":8.99,
      |"isbn":"0-553-21311-3"},{"author":"J. R. R. Tolkien","title":"The Lord of the Rings",
      |"category":"fiction","reader":[{"age":25,"name":"bob"},{"age":26,"name":"jack"}],
      |"price":22.99,"isbn":"0-395-19395-8"}],"bicycle":{"price":19.95,"color":"red"}},
      |"email":"amy@only_for_json_udf_test.net","owner":"amy","zip code":"94025",
      |"fb:testid":"1234"}
      |""".stripMargin

we return null, meaning it didn't match anything, but the CPU say it matches ["y"]. https://jsonpath.com/ also produces the same answer as the CPU.

@revans2 revans2 added bug Something isn't working ? - Needs Triage Need team to review and classify labels Jan 18, 2024
@mattahrens mattahrens removed the ? - Needs Triage Need team to review and classify label Jan 30, 2024
@thirtiseven
Copy link
Collaborator

A minimal repro: {"a":[{"b":"c"},2]} and $.a[*].b will return ["c"] on cpu and jsonpath.com, but null on gpu.

This is related to the [*] and I think it's a cudf bug. Will add fallback for paths containing [*] first, which is also a requirement from #10216

@res-life
Copy link
Collaborator

Will be fixed by: NVIDIA/spark-rapids-jni#1868

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants