Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

feat!: harden dtype handling for scalars #418

Merged

Conversation

douglasdavis
Copy link
Collaborator

@douglasdavis douglasdavis commented Nov 15, 2023

This PR hardens treatment of meta/dtypes for Scalar collections. It changes the Scalar.__init__ and new_scalar_object function, which should really only find use inside of dask-awkward, but it's still possible this can break things.

The PR makes sure that all Scalars have a meta and a dtype, stemming from conversation with @agoose77 on Slack, so I'd be happy to get your feedback here :)

@douglasdavis douglasdavis force-pushed the scalar-dtype-and-meta-handling branch from 1315bca to 423bc04 Compare November 15, 2023 19:17
@codecov-commenter
Copy link

codecov-commenter commented Nov 15, 2023

Codecov Report

Attention: 6 lines in your changes are missing coverage. Please review.

Comparison is base (9001227) 93.75% compared to head (dfcd294) 93.55%.

Files Patch % Lines
src/dask_awkward/lib/core.py 85.71% 5 Missing ⚠️
src/dask_awkward/lib/structure.py 0.00% 1 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #418      +/-   ##
==========================================
- Coverage   93.75%   93.55%   -0.20%     
==========================================
  Files          23       23              
  Lines        3139     3152      +13     
==========================================
+ Hits         2943     2949       +6     
- Misses        196      203       +7     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

self._meta: Any = self._check_meta(meta)
if meta is not None and dtype is None:
self._meta = self._check_meta(meta)
self._dtype = np.dtype(self._meta.type.content.primitive)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the meta always has a NumpyType type, we could also just use self._meta.layout.dtype. I think that might be slightly better, because it uses Awkward's dtype→primitive conversion.

@agoose77
Copy link
Collaborator

OK, looks good (though I made a change)! It seems then that scalars are now always implemented as an ak.Array of length 1. I assume that this is so that _meta from dask-awkward collections is always None or an ak.Array for simplicity?

@douglasdavis
Copy link
Collaborator Author

_meta from dask-awkward collections is always None or an ak.Array for simplicity?

Yes that's what I'm going for here!

@douglasdavis douglasdavis merged commit fa80052 into dask-contrib:main Nov 30, 2023
22 checks passed
@douglasdavis douglasdavis deleted the scalar-dtype-and-meta-handling branch November 30, 2023 16:49
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants