Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

[Audit][SPARK-41660][SQL] Only propagate metadata columns if they are used #7524

Open
wbo4958 opened this issue Jan 18, 2023 · 0 comments
Open

[Audit][SPARK-41660][SQL] Only propagate metadata columns if they are used #7524

wbo4958 opened this issue Jan 18, 2023 · 0 comments
Labels
audit_3.3.0 Audit related tasks for 3.3.0 audit_3.4.0 Audit related tasks for 3.4.0 bug Something isn't working

Comments

@wbo4958
Copy link
Collaborator

wbo4958 commented Jan 18, 2023

This PR apache/spark@bd864a085f only propagates the wanted metadata columns in LogicalPlan which may cause GpuFileSan/Column pruning to fail. We'd better verify it.
@wbo4958 wbo4958 added bug Something isn't working ? - Needs Triage Need team to review and classify labels Jan 18, 2023
@sameerz sameerz added audit_3.4.0 Audit related tasks for 3.4.0 and removed ? - Needs Triage Need team to review and classify labels Jan 24, 2023
@NVnavkumar NVnavkumar mentioned this issue May 17, 2023
Open
27 tasks
@tgravescs tgravescs added the audit_3.3.0 Audit related tasks for 3.3.0 label May 26, 2023
# for free to join this conversation on GitHub. Already have an account? # to comment
Assignees
No one assigned
Labels
audit_3.3.0 Audit related tasks for 3.3.0 audit_3.4.0 Audit related tasks for 3.4.0 bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@wbo4958 @tgravescs @sameerz