Skip to content

Internal error generate_series: Internal error: could not cast value to .....arrow_array::types::Date32Type` #11922

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Closed
alamb opened this issue Aug 9, 2024 · 3 comments · Fixed by #12002
Assignees
Labels
bug Something isn't working

Comments

@alamb
Copy link
Contributor

alamb commented Aug 9, 2024

Describe the bug

When running a query like

select generate_series(start, '1993-03-01', INTERVAL '1 year') from date_table;

You get an error:

DataFusion error: Internal error: could not cast value to arrow_array::array::primitive_array::PrimitiveArray<arrow_array::types::Date32Type>.
This was likely caused by a bug in DataFusion's code and we would welcome that you file an bug report in our issue tracker

To Reproduce

DataFusion CLI v40.0.0
> CREATE TABLE date_table(
  start DATE,
  stop DATE,
  step INTERVAL
) AS VALUES
  (DATE '1992-01-01', DATE '1993-01-02', INTERVAL '1' MONTH),
  (DATE '1993-02-01', DATE '1993-01-01', INTERVAL '-1' DAY),
  (DATE '1989-04-01', DATE '1993-03-01', INTERVAL '1' YEAR);
0 row(s) fetched.
Elapsed 0.016 seconds.

> select generate_series(start, '1993-03-01', INTERVAL '1 year') from date_table;

Internal error: could not cast value to arrow_array::array::primitive_array::PrimitiveArray<arrow_array::types::Date32Type>.
This was likely caused by a bug in DataFusion's code and we would welcome that you file an bug report in our issue tracker

Expected behavior

I expect you get the same answer as if you explicitly cast the second argument to date:

> select generate_series(start, '1993-03-01'::DATE, INTERVAL '1 year') from date_table;
+-------------------------------------------------------------------------------------------------------------------------------------------+
| generate_series(date_table.start,Utf8("1993-03-01"),IntervalMonthDayNano("IntervalMonthDayNano { months: 12, days: 0, nanoseconds: 0 }")) |
+-------------------------------------------------------------------------------------------------------------------------------------------+
| [1992-01-01, 1993-01-01]                                                                                                                  |
| [1993-02-01]                                                                                                                              |
| [1989-04-01, 1990-04-01, 1991-04-01, 1992-04-01]                                                                                          |
+-------------------------------------------------------------------------------------------------------------------------------------------+
3 row(s) fetched.
Elapsed 0.001 seconds.

Additional context

I added a test here: #11921

@alamb alamb added the bug Something isn't working label Aug 9, 2024
@getChan
Copy link
Contributor

getChan commented Aug 15, 2024

take

@getChan
Copy link
Contributor

getChan commented Aug 15, 2024

I think that the query returning an error is normal.
A valid query should have start and stop arguments of the same date type. - test code has str type in stop
Therefore, instead of fixing the bug, I will proceed with the following steps:

  • Modify the test code: select generate_series(start, DATE '1993-03-01', INTERVAL '1 year') from date_table;
  • When invalid argument type, return a more informative error.
    • If TypeSignature::ExactOrNull is generated in the future, that can be resolved by changing the function’s type signature.

@getChan
Copy link
Contributor

getChan commented Aug 18, 2024

as-is test is valid.
so did not modify test.
but did fix function support string type.

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants