Skip to content

Commit

Permalink
YARN-11657. Remove protobuf-2.5 from hadoop-yarn-api module (#6575)
Browse files Browse the repository at this point in the history
The import of protobuf-java-2.5 in the hadoop-yarn-api module
is downgraded from "compile" to "provided"

This removes it from share/hadoop/yarn/lib/protobuf-java-2.5.0.jar

It is still found under
share/hadoop/yarn/timelineservice/lib/protobuf-java-2.5.0.jar
  • Loading branch information
steveloughran authored Feb 23, 2024
1 parent ad8b654 commit 209729b
Show file tree
Hide file tree
Showing 5 changed files with 10 additions and 13 deletions.
10 changes: 7 additions & 3 deletions BUILDING.txt
Original file line number Diff line number Diff line change
Expand Up @@ -315,12 +315,12 @@ Controlling the redistribution of the protobuf-2.5 dependency

The protobuf 2.5.0 library is used at compile time to compile the class
org.apache.hadoop.ipc.ProtobufHelper; this class known to have been used by
external projects in the past. Protobuf 2.5 is not used elsewhere in
external projects in the past. Protobuf 2.5 is not used directly in
the Hadoop codebase; alongside the move to Protobuf 3.x a private successor
class, org.apache.hadoop.ipc.internal.ShadedProtobufHelper is now used.

The hadoop-common module no longer exports its compile-time dependency on
protobuf-2.5. Hadoop distributions no longer include it.
protobuf-java-2.5.
Any application declaring a dependency on hadoop-commmon will no longer get
the artifact added to their classpath.
If is still required, then they must explicitly declare it:
Expand All @@ -337,10 +337,14 @@ Controlling the redistribution of the protobuf-2.5 dependency

-Dcommon.protobuf2.scope=compile

If this is done then protobuf-2.5.0.jar will again be exported as a
If this is done then protobuf-java-2.5.0.jar will again be exported as a
hadoop-common dependency, and included in the share/hadoop/common/lib/
directory of any Hadoop distribution built.

Note that protobuf-java-2.5.0.jar is still placed in
share/hadoop/yarn/timelineservice/lib; this is needed by the hbase client
library.

----------------------------------------------------------------------------------
Building components separately

Expand Down
3 changes: 2 additions & 1 deletion LICENSE-binary
Original file line number Diff line number Diff line change
Expand Up @@ -395,9 +395,10 @@ hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/d3.v3.js
hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/d3-3.5.17.min.js
leveldb v1.13

com.google.protobuf:protobuf-java:2.5.0
com.google.protobuf:protobuf-java:3.6.1
com.google.re2j:re2j:1.1
com.jcraft:jsch:0.1.54
com.jcraft:jsch:0.1.55
com.thoughtworks.paranamer:paranamer:2.3
jakarta.activation:jakarta.activation-api:1.2.1
org.fusesource.leveldbjni:leveldbjni-all:1.8
Expand Down
5 changes: 0 additions & 5 deletions hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -62,11 +62,6 @@
<artifactId>hadoop-annotations</artifactId>
</dependency>

<dependency>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-java</artifactId>
</dependency>

<dependency>
<groupId>org.apache.hadoop.thirdparty</groupId>
<artifactId>hadoop-shaded-protobuf_3_21</artifactId>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@
<dependency>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-java</artifactId>
<scope>${transient.protobuf2.scope}</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -112,10 +112,6 @@
<groupId>com.google.inject.extensions</groupId>
<artifactId>guice-servlet</artifactId>
</dependency>
<dependency>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-java</artifactId>
</dependency>

<dependency>
<groupId>io.dropwizard.metrics</groupId>
Expand Down

0 comments on commit 209729b

Please # to comment.