A Dashboard for Kafka Streams Applications running in Kubernetes.

Setup

Option A:

1. Deploy a Kafka Streams App with Kubernetes. Make sure that the App exports JMX metrics (example is from sbt):

javaOptions in Universal += Seq(
  "-Dcom.sun.management.jmxremote.port=9186",
  "-Dcom.sun.management.jmxremote.rmi.port=9186",
  "-Dcom.sun.management.jmxremote.ssl=false",
  "-Dcom.sun.management.jmxremote.local.only=false",
  "-Dcom.sun.management.jmxremote.authenticate=false"
).mkString(" ")

1. Start a sidecar prometheus exporter. You can use sscalling/jmx-prometheus-exporter.
2. Check if prometheus is scraping your application. You can test this by checking the query result of kafka_streams_kafka_metrics_count_count.
3. If prometheus is scraping correctly, the dashboard should work.

Option B:

1. Deploy your application with the prometheus-jmx-exporter as java agent (see here
2. No need for an additional sidecar container, your app exports prometheus metrics directly.
3. Check if prometheus is scraping your application. You can test this by checking the query result of kafka_streams_kafka_metrics_count_count.
4. If prometheus is scraping correctly, the dashboard should work.

Kubernetes Config

To automatically add your application to prometheus, use the following annotations in a kubernetes service:

annotations:
  prometheus.io/scrape: 'true'
  prometheus.io/port: '5556'
  prometheus.io/path: "/metrics"