Skip to content

feat: impl display for DataType::List #7051

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

irenjj
Copy link
Contributor

@irenjj irenjj commented Jan 31, 2025

Which issue does this PR close?

Closes #7048

Rationale for this change

What changes are included in this PR?

Are there any user-facing changes?

@github-actions github-actions bot added the arrow Changes to the arrow crate label Jan 31, 2025
@tustvold tustvold added api-change Changes to the arrow API next-major-release the PR has API changes and it waiting on the next major version labels Jan 31, 2025
Copy link
Contributor

@tustvold tustvold left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me, although it will need to wait for the next breaking release as it is a breaking change

DataType::List(Arc::new(Field::new_list_field(DataType::UInt64, false)));
let list_data_type_string = list_data_type.to_string();
let expected_string = "List(UInt64)";
assert_eq!(list_data_type_string, expected_string);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we could get a test of a nested list as well

@irenjj irenjj requested a review from tustvold February 1, 2025 11:26
@irenjj
Copy link
Contributor Author

irenjj commented Feb 1, 2025

Thanks @tustvold , I have made the changes as you suggested. Could you please help review it again?

Copy link
Contributor

@tustvold tustvold left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've taken another look, but just to set expectations this won't be merged until main opens up for the next breaking release in a months time

@@ -458,7 +458,16 @@ pub enum UnionMode {

impl fmt::Display for DataType {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
write!(f, "{self:?}")
match self {
DataType::List(field) => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We also need to print the field name here if it isn't the default of item

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right -- so if field.name() != "item" perhaps it could look like

List(Int8;N, field='foo')

Likewise, if there is metadata I think it should also be displayed like

List(Int8;N, field='foo', metadata)

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @irenjj and @tustvold

@Blizzara
Copy link
Contributor

Blizzara commented Apr 7, 2025

The Arrow docstring on datatype says the Display should be also reversible - look at the code of datatype_parse it doesn't seem like lists were parseable even before, but maybe it'd be good to add as part of this PR given the new format should be easier to parse? :)

@@ -458,7 +458,25 @@ pub enum UnionMode {

impl fmt::Display for DataType {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
write!(f, "{self:?}")
match self {
DataType::List(field) => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe add the same for LargeList and FixedSizeList?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Blizzara for your advice, that's good idea!

@goldmedal
Copy link
Contributor

Thanks @irenjj for working on this. I agree with @Blizzara. The display result should be reversible. I prefer to format the List type like the following way:

            DataType::List(field) => {
                write!(f, "List({})", field.data_type())
            }

The result would be something like List(Int64). It can be parsed by parse_data_type.
I think other list types can be formatted in the same way.
Then, it's better to add the corresponding test cases at list_datatypes, where we will test the round-trip of data type parsing.

fn list_datatypes() -> Vec<DataType> {

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
api-change Changes to the arrow API arrow Changes to the arrow crate next-major-release the PR has API changes and it waiting on the next major version
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Improve human readable display for DataType::List
5 participants