Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

[BUG] Column vector leak in conditionals_test.py #4541

Closed
andygrove opened this issue Jan 15, 2022 · 0 comments · Fixed by #4552
Closed

[BUG] Column vector leak in conditionals_test.py #4541

andygrove opened this issue Jan 15, 2022 · 0 comments · Fixed by #4552
Assignees
Labels
bug Something isn't working

Comments

@andygrove
Copy link
Contributor

Describe the bug

I am seeing this in branch-22.02

TEST_PARALLEL=1 ./run_pyspark_from_build.sh -k test_conditional_with_side_effects

...

../../src/main/python/conditionals_test.py::test_conditional_with_side_effects_col_col[Integer] PASSED [ 25%]
../../src/main/python/conditionals_test.py::test_conditional_with_side_effects_col_scalar[Integer] PASSED [ 50%]
../../src/main/python/conditionals_test.py::test_conditional_with_side_effects_cast[String] PASSED [ 75%]
../../src/main/python/conditionals_test.py::test_conditional_with_side_effects_case_when[String] PASSED [100%]

=============================== warnings summary ===============================
../../../../../../../opt/spark-3.0.1-bin-hadoop3.2/python/lib/pyspark.zip/pyspark/sql/context.py:75
  /opt/spark-3.0.1-bin-hadoop3.2/python/lib/pyspark.zip/pyspark/sql/context.py:75: DeprecationWarning: Deprecated in 3.0.0. Use SparkSession.builder.getOrCreate() instead.

-- Docs: https://docs.pytest.org/en/stable/warnings.html
=============== 4 passed, 13803 deselected, 1 warning in 10.10s ================
22/01/15 03:16:39 ERROR ColumnVector: A DEVICE COLUMN VECTOR WAS LEAKED (ID: 6212 7f5e20059a30)
22/01/15 03:16:39 ERROR ColumnVector: A DEVICE COLUMN VECTOR WAS LEAKED (ID: 6660 7f5e2c049070)
22/01/15 03:16:39 ERROR ColumnVector: A DEVICE COLUMN VECTOR WAS LEAKED (ID: 5853 7f5d840681e0)
22/01/15 03:16:39 ERROR ColumnVector: A DEVICE COLUMN VECTOR WAS LEAKED (ID: 9029 7f5df01798c0)
22/01/15 03:16:39 ERROR ColumnVector: A DEVICE COLUMN VECTOR WAS LEAKED (ID: 6944 7f5db4056e70)
22/01/15 03:16:39 ERROR ColumnVector: A DEVICE COLUMN VECTOR WAS LEAKED (ID: 7809 7f5dc4067fc0)
22/01/15 03:16:39 ERROR ColumnVector: A DEVICE COLUMN VECTOR WAS LEAKED (ID: 7015 7f5dd804bcb0)

Steps/Code to reproduce bug
See above

Expected behavior
Should not leak.

Environment details (please complete the following information)
N/A

Additional context
None

@andygrove andygrove added bug Something isn't working ? - Needs Triage Need team to review and classify labels Jan 15, 2022
@andygrove andygrove added this to the Jan 10 - Jan 28 milestone Jan 15, 2022
@andygrove andygrove self-assigned this Jan 15, 2022
@sameerz sameerz removed the ? - Needs Triage Need team to review and classify label Jan 18, 2022
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants