[Parquet][Python] Read and write file/column metadata using pandas attrs

[asfimport]

Related: pandas-dev/pandas#20521

What the general thoughts are to use DataFrame.attrs and Series.attrs for reading and writing metadata to/from parquet?

For example, here is how the metadata would be written:

pdf = pandas.DataFrame({"a": [1]})
pdf.attrs = {"name": "my custom dataset"}
pdf.a.attrs = {"long_name": "Description about data", "nodata": -1, "units": "metre"}
pdf.to_parquet("file.parquet")

Then, when loading in the data:

pdf = pandas.read_parquet("file.parquet")
pdf.attrs

{"name": "my custom dataset"}

pdf.a.attrs

{"long_name": "Description about data", "nodata": -1, "units": "metre"}

 

 

Reporter: Alan Snow

_{Note: This issue was originally created as ARROW-12823. Please see the migration documentation for further details.}

[asfimport]

Alan Snow:
Seems like writing metadata could happen in get_column_metadata

Possibly add an "attrs" item so it doesn't conflict with "metadata".