New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

[protocol] USE_BITMASK is ambiguous #252

Open

pitrou opened this issue Sep 5, 2023 · 0 comments

Labels

interchange-protocol

pitrou commented Sep 5, 2023

The description for USE_BITMASK does not specify in which order the bits of a byte are to be considered (from MSB to LSB or LSB to MSB).

FTR, Arrow goes from LSB to MSB, i.e. bit 0 representents the validity of the first array element, bit 1 the second element, and so on:

>>> a = pa.array([1, None, 2, 3])
>>> a.buffers()
[<pyarrow.Buffer address=0x7f3dc8209000 size=1 is_cpu=True is_mutable=True>,
 <pyarrow.Buffer address=0x7f3dc8209040 size=32 is_cpu=True is_mutable=True>]
# buffer 0 is the validity bitmap
>>> bin(a.buffers()[0][0])
'0b1101'

pitrou changed the title ~~USE_BITMASK is ambiguous~~ [protocol] USE_BITMASK is ambiguous

jorisvandenbossche added the interchange-protocol label

# for free to join this conversation on GitHub. Already have an account? # to comment