Using source code deployment, after upgrading from version 3.8.1 to 3.8.2, the Kafka component cannot start and remains in Restarting state #2903
Labels
bug
Categorizes issue or PR as related to a bug.
OpenIM Server Version
3.8.2
Operating System and CPU Architecture
Linux (AMD)
Deployment Method
Source Code Deployment
Bug Description and Steps to Reproduce
The current version is 3.8.1, with source code deployment. The following are the upgrade steps:
View Kafka logs:
kafka 23:06:12.85 INFO ==> ** Starting Kafka **
[2024-11-28 23:06:13,750] INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$)
[2024-11-28 23:06:14,111] INFO Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation (org.apache.zookeeper.common.X509Util)
[2024-11-28 23:06:14,317] INFO Registered signal handlers for TERM, INT, HUP (org.apache.kafka.common.utils.LoggingSignalHandler)
[2024-11-28 23:06:14,322] INFO [ControllerServer id=0] Starting controller (kafka.server.ControllerServer)
[2024-11-28 23:06:14,791] INFO Updated connection-accept-rate max connection creation rate to 2147483647 (kafka.network.ConnectionQuotas)
[2024-11-28 23:06:14,856] INFO [SocketServer listenerType=CONTROLLER, nodeId=0] Created data-plane acceptor and processors for endpoint : ListenerName(CONTROLLER) (kafka.network.SocketServer)
[2024-11-28 23:06:14,859] INFO [SharedServer id=0] Starting SharedServer (kafka.server.SharedServer)
[2024-11-28 23:06:14,921] ERROR [SharedServer id=0] Got exception while starting SharedServer (kafka.server.SharedServer)
java.io.IOException: Could not read file /bitnami/kafka/data/__cluster_metadata-0/00000000000006012624-0000000019.checkpoint
at kafka.log.LogLoader.$anonfun$removeTempFilesAndCollectSwapFiles$2(LogLoader.scala:228)
at scala.collection.TraversableLike$WithFilter.$anonfun$foreach$1(TraversableLike.scala:985)
at scala.collection.IndexedSeqOptimized.foreach(IndexedSeqOptimized.scala:36)
at scala.collection.IndexedSeqOptimized.foreach$(IndexedSeqOptimized.scala:33)
at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:198)
at scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:984)
at kafka.log.LogLoader.removeTempFilesAndCollectSwapFiles(LogLoader.scala:226)
at kafka.log.LogLoader.load(LogLoader.scala:101)
at kafka.log.UnifiedLog$.apply(UnifiedLog.scala:1804)
at kafka.raft.KafkaMetadataLog$.apply(KafkaMetadataLog.scala:588)
at kafka.raft.KafkaRaftManager.buildMetadataLog(RaftManager.scala:270)
at kafka.raft.KafkaRaftManager.(RaftManager.scala:170)
at kafka.server.SharedServer.start(SharedServer.scala:247)
at kafka.server.SharedServer.startForController(SharedServer.scala:129)
at kafka.server.ControllerServer.startup(ControllerServer.scala:197)
at kafka.server.KafkaRaftServer.$anonfun$startup$1(KafkaRaftServer.scala:95)
at kafka.server.KafkaRaftServer.$anonfun$startup$1$adapted(KafkaRaftServer.scala:95)
at scala.Option.foreach(Option.scala:407)
at kafka.server.KafkaRaftServer.startup(KafkaRaftServer.scala:95)
at kafka.Kafka$.main(Kafka.scala:113)
at kafka.Kafka.main(Kafka.scala)
[2024-11-28 23:06:14,926] INFO [ControllerServer id=0] Waiting for controller quorum voters future (kafka.server.ControllerServer)
[2024-11-28 23:06:14,926] INFO [ControllerServer id=0] Finished waiting for controller quorum voters future (kafka.server.ControllerServer)
[2024-11-28 23:06:14,928] ERROR Encountered fatal fault: caught exception (org.apache.kafka.server.fault.ProcessTerminatingFaultHandler)
java.lang.NullPointerException: Cannot invoke "kafka.raft.KafkaRaftManager.apiVersions()" because the return value of "kafka.server.SharedServer.raftManager()" is null
at kafka.server.ControllerServer.startup(ControllerServer.scala:210)
at kafka.server.KafkaRaftServer.$anonfun$startup$1(KafkaRaftServer.scala:95)
at kafka.server.KafkaRaftServer.$anonfun$startup$1$adapted(KafkaRaftServer.scala:95)
at scala.Option.foreach(Option.scala:407)
at kafka.server.KafkaRaftServer.startup(KafkaRaftServer.scala:95)
at kafka.Kafka$.main(Kafka.scala:113)
at kafka.Kafka.main(Kafka.scala)
kafka 23:06:16.74 INFO ==>
kafka 23:06:16.74 INFO ==> Welcome to the Bitnami kafka container
kafka 23:06:16.74 INFO ==> Subscribe to project updates by watching https://github.com/bitnami/containers
kafka 23:06:16.74 INFO ==> Submit issues and feature requests at https://github.com/bitnami/containers/issues
kafka 23:06:16.75 INFO ==>
kafka 23:06:16.75 INFO ==> ** Starting Kafka setup **
kafka 23:06:16.89 INFO ==> Initializing KRaft storage metadata
kafka 23:06:16.89 INFO ==> Formatting storage directories to add metadata...
All of the log directories are already formatted.
kafka 23:06:18.88 INFO ==> ** Kafka setup finished! **
kafka 23:06:18.90 INFO ==> ** Starting Kafka **
[2024-11-28 23:06:19,800] INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$)
[2024-11-28 23:06:20,151] INFO Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation (org.apache.zookeeper.common.X509Util)
[2024-11-28 23:06:20,331] INFO Registered signal handlers for TERM, INT, HUP (org.apache.kafka.common.utils.LoggingSignalHandler)
[2024-11-28 23:06:20,336] INFO [ControllerServer id=0] Starting controller (kafka.server.ControllerServer)
[2024-11-28 23:06:20,859] INFO Updated connection-accept-rate max connection creation rate to 2147483647 (kafka.network.ConnectionQuotas)
[2024-11-28 23:06:20,948] INFO [SocketServer listenerType=CONTROLLER, nodeId=0] Created data-plane acceptor and processors for endpoint : ListenerName(CONTROLLER) (kafka.network.SocketServer)
[2024-11-28 23:06:20,951] INFO [SharedServer id=0] Starting SharedServer (kafka.server.SharedServer)
[2024-11-28 23:06:21,028] ERROR [SharedServer id=0] Got exception while starting SharedServer (kafka.server.SharedServer)
java.io.IOException: Could not read file /bitnami/kafka/data/__cluster_metadata-0/00000000000006012624-0000000019.checkpoint
at kafka.log.LogLoader.$anonfun$removeTempFilesAndCollectSwapFiles$2(LogLoader.scala:228)
at scala.collection.TraversableLike$WithFilter.$anonfun$foreach$1(TraversableLike.scala:985)
at scala.collection.IndexedSeqOptimized.foreach(IndexedSeqOptimized.scala:36)
at scala.collection.IndexedSeqOptimized.foreach$(IndexedSeqOptimized.scala:33)
at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:198)
at scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:984)
at kafka.log.LogLoader.removeTempFilesAndCollectSwapFiles(LogLoader.scala:226)
at kafka.log.LogLoader.load(LogLoader.scala:101)
at kafka.log.UnifiedLog$.apply(UnifiedLog.scala:1804)
at kafka.raft.KafkaMetadataLog$.apply(KafkaMetadataLog.scala:588)
at kafka.raft.KafkaRaftManager.buildMetadataLog(RaftManager.scala:270)
at kafka.raft.KafkaRaftManager.(RaftManager.scala:170)
at kafka.server.SharedServer.start(SharedServer.scala:247)
at kafka.server.SharedServer.startForController(SharedServer.scala:129)
at kafka.server.ControllerServer.startup(ControllerServer.scala:197)
at kafka.server.KafkaRaftServer.$anonfun$startup$1(KafkaRaftServer.scala:95)
at kafka.server.KafkaRaftServer.$anonfun$startup$1$adapted(KafkaRaftServer.scala:95)
at scala.Option.foreach(Option.scala:407)
at kafka.server.KafkaRaftServer.startup(KafkaRaftServer.scala:95)
at kafka.Kafka$.main(Kafka.scala:113)
at kafka.Kafka.main(Kafka.scala)
[2024-11-28 23:06:21,032] INFO [ControllerServer id=0] Waiting for controller quorum voters future (kafka.server.ControllerServer)
[2024-11-28 23:06:21,032] INFO [ControllerServer id=0] Finished waiting for controller quorum voters future (kafka.server.ControllerServer)
[2024-11-28 23:06:21,034] ERROR Encountered fatal fault: caught exception (org.apache.kafka.server.fault.ProcessTerminatingFaultHandler)
java.lang.NullPointerException: Cannot invoke "kafka.raft.KafkaRaftManager.apiVersions()" because the return value of "kafka.server.SharedServer.raftManager()" is null
at kafka.server.ControllerServer.startup(ControllerServer.scala:210)
at kafka.server.KafkaRaftServer.$anonfun$startup$1(KafkaRaftServer.scala:95)
at kafka.server.KafkaRaftServer.$anonfun$startup$1$adapted(KafkaRaftServer.scala:95)
at scala.Option.foreach(Option.scala:407)
at kafka.server.KafkaRaftServer.startup(KafkaRaftServer.scala:95)
at kafka.Kafka$.main(Kafka.scala:113)
at kafka.Kafka.main(Kafka.scala)
Screenshots Link
No response
The text was updated successfully, but these errors were encountered: