Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Fix Spark 3.2.1 build failures and make it non-snapshot #4508

Merged
merged 13 commits into from
Jan 28, 2022

Conversation

tgravescs
Copy link
Collaborator

Spark has 3.2.1 up for vote, I was testing and found it wouldn't build. The issue here is that https://issues.apache.org/jira/browse/SPARK-37705 was merged in but our internal build didn't pick it up due to spark haven't already changed the version number to 3.2.2-snapshot.

This is WIP because it has temporary repository addition to build against the 3.2.1 rc1 version. Once it's officially released I will update this to remove that repository. Putting up to run through build and get any comments early.

fixes #4500

@tgravescs tgravescs added bug Something isn't working build Related to CI / CD or cleanly building labels Jan 12, 2022
@tgravescs tgravescs added this to the Jan 10 - Jan 28 milestone Jan 12, 2022
@tgravescs tgravescs self-assigned this Jan 12, 2022
@tgravescs
Copy link
Collaborator Author

build

Copy link
Contributor

@jlowe jlowe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm once the temp repo changes are removed.

@tgravescs
Copy link
Collaborator Author

build

@tgravescs
Copy link
Collaborator Author

build

@tgravescs
Copy link
Collaborator Author

premerged timed out for some reason, looking

@tgravescs
Copy link
Collaborator Author

saw some pretty big pauses for now apparent reason. Just going to rekick and see what happens:

[2022-01-12T20:07:30.312Z] ^[[36mDiscovery starting.^[[0m
[2022-01-12T20:08:12.503Z] [gw2]^[[36m [ 94%] ^[[0m^[[32mPASSED^[[0m ../../src/main/python/udf_test.py::test_cogroup_apply_udf[Short(not_null)][IGNORE_ORDER, ALLOW_NON_GPU(FlatMapCoGroupsInPandasExec,PythonUDF,Alias)]
[2022-01-12T20:08:38.034Z] ^[[36mDiscovery completed in 1 minute.^[[0m

@tgravescs
Copy link
Collaborator Author

build

@tgravescs tgravescs changed the title [WIP] Fix Spark 3.2.1 build failures and make it non-snapshot Fix Spark 3.2.1 build failures and make it non-snapshot Jan 28, 2022
@tgravescs
Copy link
Collaborator Author

build

@tgravescs tgravescs marked this pull request as ready for review January 28, 2022 14:39
@tgravescs
Copy link
Collaborator Author

3.2.1 is released, I upmerged and am running the tests

@tgravescs tgravescs merged commit 73c832b into NVIDIA:branch-22.02 Jan 28, 2022
@pxLi pxLi changed the title Fix Spark 3.2.1 build failures and make it non-snapshot Fix Spark 3.2.1 build failures and make it non-snapshot Feb 7, 2022
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
bug Something isn't working build Related to CI / CD or cleanly building
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] Build failures against Spark 3.2.1 rc1 and make 3.2.1 non snapshot
3 participants