Extract Parquet statistics from Interval column

[marvinlanhenke]

Which issue does this PR close?

Closes #10752.

Rationale for this change

Since parquet does not support statistics for Interval columns, this PR only prepares test cases and some necessary setup.
Once parquet supports statistics, a follow-up PR will be needed to finish the implementation.

What changes are included in this PR?

• added test cases for interval columns
• added get_statistic match arm (stub)
• added comments to highlight the not yet supported parts
• for now the test_cases should_panic

Are these changes tested?

Are there any user-facing changes?

No.

[marvinlanhenke]

@alamb PTAL.

As described in the PR I tried to prepare as much as possible, although statistics are not yet supported.

Yet another issue: The IntervalUnit::MonthDayNano is also not supported.

So I'll guess we have to open 2 tickets in arrow-rs?

[alamb]

Thank you @marvinlanhenke -- this looks very nice 👌 -- I left some comments but I think it is quite close.

Note that there are some conflicts, likely with #10711 which merged a few hours ago

I will file a ticket upstream in arrow-rs to support writing statistics for intervals (thank you for the code pointers)

[alamb]

@alamb PTAL.

As described in the PR I tried to prepare as much as possible, although statistics are not yet supported.

Yet another issue: The IntervalUnit::MonthDayNano is also not supported.

So I'll guess we have to open 2 tickets in arrow-rs?

I filed apache/arrow-rs#5847. I didn't quite understand your suggestion about two tickets

[marvinlanhenke]

@alamb PTAL.
As described in the PR I tried to prepare as much as possible, although statistics are not yet supported.
Yet another issue: The IntervalUnit::MonthDayNano is also not supported.
So I'll guess we have to open 2 tickets in arrow-rs?

I filed apache/arrow-rs#5847. I didn't quite understand your suggestion about two tickets

I was thinking about two tickets:

1. Interval statistics not supported; possibly due to those lines
2. Type IntervalUnit::MonthDayNano not supported at all by the arrow_writer

I think those are two distinct issues?

[marvinlanhenke]

@alamb thanks for the review. I have adressed your comments PTAL.

[alamb]

1. Interval statistics not supported; possibly due to those lines
2. Type IntervalUnit::MonthDayNano not supported at all by the arrow_writer

I think those are two distinct issues?

Yes I think you are right -- any chance you can file a ticket in arrow-rs to track writing IntervalMonthDayNano?

[marvinlanhenke]

1. Interval statistics not supported; possibly due to those lines
2. Type IntervalUnit::MonthDayNano not supported at all by the arrow_writer

I think those are two distinct issues?

Yes I think you are right -- any chance you can file a ticket in arrow-rs to track writing IntervalMonthDayNano?

Sure, I can do that.

I filed: apache/arrow-rs#5849

[alamb]

Looks good to me -- thank you very much @marvinlanhenke

[comphead]

lgtm @marvinlanhenke kudos for the test description

[tustvold]

As per the format specification, it is incorrect to read statistics for interval columns, as there is no defined sort order - https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#interval

[marvinlanhenke]

@alamb
Since the support for Interval is not planned or possible, should we remove the added code here? As it might never be supported, we should probably get rid of potentially dead code? Less code less problems...

[alamb]

@alamb Since the support for Interval is not planned or possible, should we remove the added code here? As it might never be supported, we should probably get rid of potentially dead code? Less code less problems...

I think removing it would be fine @marvinlanhenke -- thank you