kafka 啟動報錯 missingTopicsFatal is true的解決

Posted on by

kafka 啟動報錯：missingTopicsFatal is true

報錯信息

org.springframework.context.ApplicationContextException: Failed to start bean ‘org.springframework.kafka.config.internalKafkaListenerEndpointRegistry’; nested exception is java.lang.IllegalStateException: Topic(s) [async] is/are not present and missingTopicsFatal is true
at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:185) ~[spring-context-5.2.0.RELEASE.jar!/:5.2.0.RELEASE]
at org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:53) ~[spring-context-5.2.0.RELEASE.jar!/:5.2.0.RELEASE]
at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:360) ~[spring-context-5.2.0.RELEASE.jar!/:5.2.0.RELEASE]
at org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:158) ~[spring-context-5.2.0.RELEASE.jar!/:5.2.0.RELEASE]
at org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:122) ~[spring-context-5.2.0.RELEASE.jar!/:5.2.0.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:894) ~[spring-context-5.2.0.RELEASE.jar!/:5.2.0.RELEASE]
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.finishRefresh(ServletWebServerApplicationContext.java:162) ~[spring-boot-2.2.0.RELEASE.jar!/:2.2.0.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:553) ~[spring-context-5.2.0.RELEASE.jar!/:5.2.0.RELEASE]
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:141) ~[spring-boot-2.2.0.RELEASE.jar!/:2.2.0.RELEASE]

報錯原因：消費監聽接口監聽的主題不存在時，默認會報錯

解決方法：

配置文件中將listener的屬性missingTopicsFatal設置為false

spring:
  kafka:
    listener:
      missing-topics-fatal: false

kafka常見報錯警告異常解決

1.自行修改server.properties中broker id後造成與meta.properties中數據不匹配

kafka.common.InconsistentBrokerIdException: Configured broker.id 2 doesn’t match stored broker.id 1 in meta.properties. If you moved your data, make sure your configured broker.id matches. If you intend to create a new broker, you should remove all data in your data directories (log.dirs).

2.__consumer_offsets這個topic是由kafka自動創建的，默認49個，這個topic是不能被刪除的！

Error while executing topic command : Topic __consumer_offsets is a kafka internal topic and is not allowed to be marked for deletion.

為什麼這裡會是這樣存儲__consumer_offsets的呢：

  • [1.] 將所有 N Broker 和待分配的 i 個 Partition 排序
  • [2.] 將第 i 個 Partition 分配到第(i mod n)個 Broker 上

3.刪除topic時

Topic frist is already marked for deletion.

在這裡插入圖片描述

徹底刪除topic：

  • [1. ] 刪除Topic，delete.topic.enable=true這裡要設置為true
  • [2. ] 刪除log日志
  • [3. ] 刪除ZK中的Topic記錄

這裡是引用

命令：./bin/zkCli.sh

找到topic所在的目錄：ls /brokers/topics

執行命令：rmr /brokers/topics/{topic name}即可，此時topic被徹底刪除。

另外被標記為marked for deletion的topic你可以在zookeeper客戶端中通過命令獲得：ls /admin/delete_topics/{topic name}，如果你刪除瞭此處的topic，那麼marked for deletion 標記消失

當然，這裡是你delete.topic.enable設置為false時Zookeeper下才會有這個節點。

以上為個人經驗，希望能給大傢一個參考，也希望大傢多多支持WalkonNet。

推薦閱讀：