Removing Protobuf 2.5

[jojochuang]

Hi I spent some time trying to get rid of protobuf 2.5. Took several approaches:

(1) Hadoop recommends migrating to ProtobufRpcEngine2 where it uses shaded protobuf classes org.apache.hadoop.thirdparty.protobuf.*. Tried to this approach but it's a huge mess. Not only does the RPC messages need to change to implement the shaded protobuf classes, rocksdb codec and column family must change as well. Grpc needs special treatment to use the new RPC messages too. What's worse, it implies we will be in lockstep with the Hadoop thirdparty protobuf version. That just doesn't seem right because Ozone don't talk to Hadoop or HDFS.

(2) Bump protobuf 2.5 to 3.x in the pom.xml, use protoc 3 instead of protoc 2.5 to compile protobuf message definition files. I can confirm that Hadoop's ProtobufRpcEngine works with protobuf 3. It doesn't require ProtobufRpcEngine2 to use protobuf3.
Advantage: minimum change.
Drawback: every test passes except with Hadoop 2 (ozonesecure-mr). In an Hadoop 2 environment, the class path does not have protobuf 3 jars. In addition, it probably won't work with Spark, or any other downstream application clients. (PoC: https://github.com/jojochuang/ozone/commits/refs/heads/remove_protobuf2.5_take2/)

(3) I've been thinking we should create a shaded uber filesystem jar, that in addition to approach (2), include & relocate Hadoop's ProtobufRpcEngine, protobuf3, Grpc (possibly via ozone-filesystem-shaded), so that everything related to com.google.protobuf get relocated to org.apache.ozone.shaded.com.google.protobuf. That way, it should work.
This approach has an additional benefit that Spark will no longer require ozone-filesystem-hadoop3-client to use Ozone. The problem observed in HDDS-6926 is due to class path collision. If the client has ProtobufRpcEngine class relocated, it won't collide with the one coming from hadoop-client-api.