Skip to content

ENH: Non-nullable extension types #40554

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Closed
gsakkis opened this issue Mar 21, 2021 · 5 comments
Closed

ENH: Non-nullable extension types #40554

gsakkis opened this issue Mar 21, 2021 · 5 comments
Labels
Closing Candidate May be closeable, needs more eyeballs Enhancement ExtensionArray Extending pandas with custom dtypes or arrays. Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate

Comments

@gsakkis
Copy link

gsakkis commented Mar 21, 2021

As far as I can tell, all custom extension types are nullable. It would be useful to provide a way to create non-nullable extensions. One way to support this could be by setting the na_value attribute to NotImplemented. Thoughts?

@gsakkis gsakkis added Enhancement Needs Triage Issue that has not been reviewed by a pandas team member labels Mar 21, 2021
@dsaxton
Copy link
Member

dsaxton commented Mar 21, 2021

Sounds like you want something like a database constraint but for DataFrames? That's an interesting idea, although I suspect this might be easiest to accomplish in end-user code by simply asserting that a given column contains no null values, rather than creating whole new data types.

@dsaxton dsaxton added Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate and removed Needs Triage Issue that has not been reviewed by a pandas team member labels Mar 21, 2021
@gsakkis
Copy link
Author

gsakkis commented Mar 22, 2021

IMO nullability is part of the type definition, it's not some extra constraint to be tested by the end-user code. Checking the constraint in the extension code itself (e.g. in __init__ and/or _from_sequence) would be an improvement but even that is not sufficient for my use case. Ideally nullability should be an inspectable property of the type, discoverable by a new pd.api.types.is_nullable_dtype(arr_or_dtype) function.

@jbrockmendel
Copy link
Member

xref #40574

@lithomas1 lithomas1 added the ExtensionArray Extending pandas with custom dtypes or arrays. label May 25, 2021
@jbrockmendel
Copy link
Member

A 3rd party EA could just raise if any NA-like is passed to the relevant constructor or __setitem__. I guess we could document this? Otherwise not clear that this is actionable.

@jbrockmendel jbrockmendel added the Closing Candidate May be closeable, needs more eyeballs label Jul 27, 2023
@mroeschke
Copy link
Member

Looks like there's not much support for this feature from the core team so closing

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
Closing Candidate May be closeable, needs more eyeballs Enhancement ExtensionArray Extending pandas with custom dtypes or arrays. Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
Projects
None yet
Development

No branches or pull requests

5 participants