Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

[BUG] Compile error for Spark330 because of VectorizedColumnReader constructor added a new parameter. #5123

Closed
res-life opened this issue Apr 1, 2022 · 1 comment · Fixed by #5124
Assignees
Labels
audit_3.3.0 Audit related tasks for 3.3.0 bug Something isn't working

Comments

@res-life
Copy link
Collaborator

res-life commented Apr 1, 2022

Describe the bug

mvn clean install -Dbuildver=330 -DskipTests
not enough arguments for constructor VectorizedColumnReader

[INFO] Compiler bridge file: /home/chong/.sbt/1.0/zinc/org.scala-sbt/org.scala-sbt-compiler-bridge_2.12-1.3.1-bin_2.12.15__52.0-1.3.1_20191012T045515.jar
[INFO] Compiling 267 Scala sources and 29 Java sources to /home/chong/code/2-spark-rapids/sql-plugin/target/spark330/classes ...
[ERROR] [Error] /home/chong/code/2-spark-rapids/sql-plugin/src/main/321+/scala/org/apache/spark/sql/execution/datasources/parquet/rapids/shims/ShimVectorizedColumnReader.scala:58: not enough arguments for constructor VectorizedColumnReader: (x$1: org.apache.parquet.column.ColumnDescriptor, x$2: org.apache.parquet.schema.LogicalTypeAnnotation, x$3: org.apache.parquet.column.page.PageReader, x$4: java.util.PrimitiveIterator.OfLong, x$5: java.time.ZoneId, x$6: String, x$7: String, x$8: String, x$9: String, x$10: org.apache.parquet.VersionParser.ParsedVersion)org.apache.spark.sql.execution.datasources.parquet.VectorizedColumnReader.
Unspecified value parameter x$10.
[ERROR] [Error] /home/chong/code/2-spark-rapids/sql-plugin/src/main/321+/scala/org/apache/spark/sql/execution/datasources/parquet/rapids/shims/ShimVectorizedColumnReader.scala:20: Unused import
[ERROR] two errors found
@res-life res-life added bug Something isn't working ? - Needs Triage Need team to review and classify audit_3.3.0 Audit related tasks for 3.3.0 labels Apr 1, 2022
@pxLi
Copy link
Member

pxLi commented Apr 1, 2022

related to apache/spark#35262 which affected 330+

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
audit_3.3.0 Audit related tasks for 3.3.0 bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants