How to run / configure outside kubernetes/strimzi

[julienmmm]

Hello,

I am trying to run the kafka-lag-exporter as a classic stand alone application.
To do that, I've built the project with "sbt clean dist" which has produced a zip file under target/universal. After unzipping I am able to start the exporter with "./bin/kafka-lag-exporter".
My problem is that it is not very clear for me how you can configure a static cluster definition, the documentation stated that it is possible to create an application.conf file but I could not find any documentation on its structure.

Thanks for your help!

Julien

[seglo]

Hi Julien,

Thanks for trying out Kafka Lag Exporter. The standalone run use case is indeed something I haven't spent time to document. You can use the reference.conf to determine what configuration you can override in your application.conf. For example, to setup several static clusters you might define:

kafka-lag-exporter {
  clusters = [
    {
       name = "clusterA"
       bootstrap-brokers = "b-1.cluster-a.xyzcorp.com:9092,b-2.cluster-a.xyzcorp.com:9092"
    }
    {
       name = "clusterB"
       bootstrap-brokers = "b-1.cluster-b.xyzcorp.com:9092,b-2.cluster-b.xyzcorp.com:9092"
    }
  ]
}

Please share your experiences running the project in standalone mode. I would welcome a section in the README documenting this.

[NeQuissimus]

I had this up and running pretty quickly like so:

docker run -p 8000:8000 -e KAFKA_LAG_EXPORTER_CLUSTERS.0.name="dev" -e KAFKA_LAG_EXPORTER_CLUSTERS.0.bootstrap-brokers="testbroker1:9092" lightbend/kafka-lag-exporter:0.4.0

You can then hit http://localhost:8000

[julienmmm]

With this simple application.conf file:

kafka-lag-exporter {
  port = 9107
  poll-interval = 30 seconds
  lookup-table-size = 60
  client-group-id = "kafkalagexporter"
  kafka-client-timeout = 10 seconds
  clusters = [
    {
       name = "kafkatst"
       bootstrap-brokers = "kafkatst01:9092,kafkatst02:9092,kafkatst03:9092"
    }
  ]
  watchers = {
    strimzi = "false"
  }
}

akka {
  loggers = ["akka.event.slf4j.Slf4jLogger"]
  loglevel = "DEBUG"
  logging-filter = "akka.event.slf4j.Slf4jLoggingFilter"
}

I have successfully run the lag exporter by executing kafka-lag-exporter -Dconfig.file=application.conf

However I am not very familiar with scala applications and Ithink that this configuraton file could be better (especially related to logging).

[seglo]

This looks good @julienmmm. The reference.conf serves as a place to define defaults when an application.conf is not provided and/or fallbacks when one is provided. You can reduce your config to:

kafka-lag-exporter {
  port = 9107
  clusters = [
    {
       name = "kafkatst"
       bootstrap-brokers = "kafkatst01:9092,kafkatst02:9092,kafkatst03:9092"
    }
  ]
}

[seglo]

Documented in #29