Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

Support str_to_map [databricks] #4636

Merged
merged 77 commits into from
Feb 23, 2022
Merged

Support str_to_map [databricks] #4636

merged 77 commits into from
Feb 23, 2022

Conversation

ttnghia
Copy link
Collaborator

@ttnghia ttnghia commented Jan 26, 2022 •
edited
Loading

@ttnghia
Add GpuOverride
77a760d 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Implement skeleton
02cf8b8 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Fix type check
6d49cf6 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
WIP
0ff0e3f 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Simplify code
5e6d23b 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Fix delimiter check
7832760 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Add comments
6b4ffe5 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Implement GpuCreateMap.createMapFromKeysValuesAsStructs
2a64e97 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Using GpuCreateMap.createMapFromKeysValuesAsStructs to create map f…
08bb1b2 
…rom split strings

Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia ttnghia added feature request New feature or request SQL part of the SQL/Dataframe plugin cudf_dependency An issue or PR with this label depends on a new feature in cudf labels Jan 26, 2022
@ttnghia ttnghia requested review from andygrove and revans2 January 26, 2022 23:23
@ttnghia ttnghia self-assigned this Jan 26, 2022
@ttnghia
Update comment
ac99eff 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Change input for createMapFromKeysValuesAsStructs
657fcc3 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Fix toMap method
fca136a 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
MISC
5c3e755 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Merge branch 'branch-22.04' into str_to_map
a654cb2
@ttnghia
Update copyright year
310f331 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Remove unused constructors
a6d70ac 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Rename variables
395e41f 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Fix type check
f3074e4 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Rename variable
c5d6780 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@sameerz sameerz linked an issue Jan 31, 2022 that may be closed by this pull request
[FEA] Support str_to_map function #3542
Closed
@ttnghia
Use makeStructView instead of makeStruct to avoid copying data
afd1f16 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Add regex tests
6231d5b 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Update compatibility.md
dbfb5ac 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Update description
b05de28 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Update docs
e610f9b 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia ttnghia marked this pull request as ready for review February 15, 2022 17:55
@ttnghia ttnghia requested a review from revans2 February 15, 2022 17:55
@ttnghia
Update test
656df3d 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Merge branch 'branch-22.04' into str_to_map
db65868 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>

# Conflicts:
#	integration_tests/src/main/python/map_test.py
@ttnghia
Rename test
57ba669 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Copy link
Collaborator Author

ttnghia commented Feb 18, 2022

build

@ttnghia
Copy link
Collaborator Author

ttnghia commented Feb 18, 2022 •
edited
Loading

@revans2 @andygrove Should we test build this with [databricks]?

@revans2
Copy link
Collaborator

revans2 commented Feb 18, 2022

@revans2 @andygrove Should we test build this with [databricks]?

I think databricks would be good just so we can be sure that it compiles. It looks like you are getting a compile error because GpuStringToMap is not inheriting from the shimmed class that lets us support newer versions of Spark ShimExpression.

@ttnghia
Inherit ShimExpression
0bab1db 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia ttnghia changed the title Support str_to_map Support str_to_map [databricks] Feb 18, 2022
@ttnghia
Copy link
Collaborator Author

ttnghia commented Feb 18, 2022

build

revans2
revans2 previously approved these changes Feb 18, 2022
View reviewed changes
@ttnghia
Update special pattern to almost always generate duplicate keys
c237cb9 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Merge branch 'branch-22.04' into str_to_map
f08c093
@ttnghia
Fix python import
4ef4a8e 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Copy link
Collaborator Author

ttnghia commented Feb 18, 2022

build

@ttnghia ttnghia requested a review from revans2 February 22, 2022 20:29
revans2
revans2 previously approved these changes Feb 23, 2022
View reviewed changes
@ttnghia
Merge branch 'branch-22.04' into str_to_map
7c05021 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>

# Conflicts:
#	integration_tests/src/main/python/map_test.py
@ttnghia
Copy link
Collaborator Author

ttnghia commented Feb 23, 2022

build

@ttnghia ttnghia merged commit 7a87978 into NVIDIA:branch-22.04 Feb 23, 2022
@ttnghia ttnghia deleted the str_to_map branch February 23, 2022 20:40
# for free to join this conversation on GitHub. Already have an account? # to comment
Reviewers

@revans2 revans2 revans2 approved these changes

@andygrove andygrove Awaiting requested review from andygrove

Assignees

@ttnghia ttnghia

Labels
cudf_dependency An issue or PR with this label depends on a new feature in cudf feature request New feature or request SQL part of the SQL/Dataframe plugin
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[FEA] Support regular expression delimiters for str_to_map [FEA] Support str_to_map function
3 participants
@ttnghia @revans2 @andygrove