Skip to content

Nested coalesce #6074

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Open
crepererum opened this issue Apr 20, 2023 · 3 comments
Open

Nested coalesce #6074

crepererum opened this issue Apr 20, 2023 · 3 comments
Labels
enhancement New feature or request

Comments

@crepererum
Copy link
Contributor

crepererum commented Apr 20, 2023

Is your feature request related to a problem or challenge?

coalsece currently does not support structs. If it would however, I think it should only consider top-level NULLs, so coalsece(NULL, {a: 1, b: NULL}, {a: 2, b: 3}) = {a: 1, b: NULL}. However sometimes it would be handy to fill in fields recursively. So coalsece_nested(NULL, {a: 1, b: NULL}, {a: 2, b: 3}) = {a: 1, b: 3}

Describe the solution you'd like

Have a coalesce variant that supports structs by traversing them recursively. So:

coalesce_nested(
  NULL,
  {
    a: 1,
    b: NULL,
    c: NULL,
    d: NULL,
  },
  {
    a: 2,
    b: NULL,
    c: {a: NULL},
    d: {a: 2, b: NULL},
  },
  {
    a: 3,
    b: NULL,
    c: NULL,
    d: {a: 3, b: 3},
  },
)

=

{
  a: 1,
  b: NULL,
  c: {a: NULL},
  d: {a: 2, b: 3},
}

Describe alternatives you've considered

-

Additional context

A user that fully knows the type could manually construct this using coalesce and named_struct (proposed in #5861).

Prior art:

@crepererum crepererum added the enhancement New feature or request label Apr 20, 2023
@crepererum
Copy link
Contributor Author

If this is a desired feature to have in DataFusion, I can contribute it. I have this nearly ready because we probably need a (somewhat simpler version of it) in InfluxDB IOx anyways.

@alamb
Copy link
Contributor

alamb commented Apr 24, 2023

I personally think we should start in IOx -- I am not sure how widely useful this is to the DataFusion community as a whole -- maybe others can weigh in

@alamb
Copy link
Contributor

alamb commented Apr 24, 2023

Added to structured type tracking ticket: #2326

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants