Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

[FEA] Fixed-point support for to_arrow and from_arrow #7116

Closed
razajafri opened this issue Jan 11, 2021 · 5 comments · Fixed by #7609
Closed

[FEA] Fixed-point support for to_arrow and from_arrow #7116

razajafri opened this issue Jan 11, 2021 · 5 comments · Fixed by #7609
Assignees
@shwina @codereport
Labels
feature request New feature or request libcudf Affects libcudf (C++/CUDA) code. Spark Functionality that helps Spark RAPIDS

Comments

@razajafri
Copy link
Contributor

Is your feature request related to a problem? Please describe.
Spark supports fixed-point precision for conversion to/from Arrow

Describe the solution you'd like
We need to support to_arrow and from_arrow for fixed-point precision

Additional context
This builds upon #3556
@razajafri razajafri added feature request New feature or request Needs Triage Need team to review and classify Spark Functionality that helps Spark RAPIDS labels Jan 11, 2021
@razajafri razajafri mentioned this issue Jan 11, 2021
Closed
27 tasks
@jrhemstad
Copy link
Contributor

This will be a little tricky since Arrow uses 128bit integers for its fixed-point decimal representation. We'll need some conversion logic and such.

@kkraus14
Copy link
Collaborator

I believe @shwina has worked out the logic in Python in #6715 and ideally we could move that directly to libcudf.

@harrism harrism mentioned this issue Jan 20, 2021
Closed
45 tasks
@kkraus14 kkraus14 added libcudf Affects libcudf (C++/CUDA) code. and removed Needs Triage Need team to review and classify labels Jan 20, 2021
@tgravescs tgravescs mentioned this issue Feb 1, 2021
Open
@github-actions
Copy link

github-actions bot commented Mar 5, 2021

This issue has been labeled inactive-30d due to no recent activity in the past 30 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed. This issue will be labeled inactive-90d if there is no activity in the next 60 days.

@kkraus14
Copy link
Collaborator

kkraus14 commented Mar 5, 2021

This is planned to be tackled in 0.19.

@shwina
Copy link
Contributor

shwina commented Mar 16, 2021

@codereport and I will be pairing on this, PR should be up shortly!

@shwina shwina mentioned this issue Mar 16, 2021
Merged
rapids-bot bot pushed a commit that referenced this issue Apr 12, 2021
@shwina
Add to/from arrow for fixed point columns (#7609)
f593e34 
Closes #7116

Authors:
  - Ashwin Srinath (https://github.com/shwina)
  - Conor Hoekstra (https://github.com/codereport)

Approvers:
  - Keith Kraus (https://github.com/kkraus14)
  - GALI PREM SAGAR (https://github.com/galipremsagar)
  - Vukasin Milovanovic (https://github.com/vuule)
  - Jake Hemstad (https://github.com/jrhemstad)

URL: #7609
devavret pushed a commit to harrism/cudf that referenced this issue Apr 14, 2021
@shwina @devavret
Add to/from arrow for fixed point columns (rapidsai#7609)
e54bffd 
Closes rapidsai#7116

Authors:
  - Ashwin Srinath (https://github.com/shwina)
  - Conor Hoekstra (https://github.com/codereport)

Approvers:
  - Keith Kraus (https://github.com/kkraus14)
  - GALI PREM SAGAR (https://github.com/galipremsagar)
  - Vukasin Milovanovic (https://github.com/vuule)
  - Jake Hemstad (https://github.com/jrhemstad)

URL: rapidsai#7609
# for free to join this conversation on GitHub. Already have an account? # to comment
Assignees

@shwina shwina

@codereport codereport

Labels
feature request New feature or request libcudf Affects libcudf (C++/CUDA) code. Spark Functionality that helps Spark RAPIDS
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add to/from arrow for fixed point columns shwina/cudf
5 participants
@shwina @kkraus14 @razajafri @jrhemstad @codereport