Home Software Development Kafka Kafka Connect: JMX JConsole (Docker)

Kafka Connect: JMX JConsole (Docker)

1572

Kafka Connect: JMX JConsole (Docker)

Yes, you absolutely can connect JConsole to Kafka Connect and extract JMX metrics. If you are experiencing problems getting it to work this blog post will get you through it. There are 2 “gotcha’s” with getting this to work with Docker/Docker Compose. The KAFKA_JMX_HOSTNAME variable must be set and the correct Java JDK must be used. Recently, there was a pull request to include the KAFKA_JMX_HOSTNAME variable in the Docker Compose. I had issues with this because it wasn’t very clear. The other issue I experienced was an incompatibility issue with JConsole.

Git Repo

Kafka Connect: JMX Console

Docker Environmental Variables

ComponentHostnamePort
KafkaKAFKA_JMX_HOSTNAMEKAFKA_JMX_PORT
Kafka ConnectKAFKA_JMX_HOSTNAMEKAFKA_JMX_PORT
ZookeeperKAFKA_JMX_HOSTNAMEKAFKA_JMX_PORT
KSQLKSQL_JMX_HOSTNAMEKSQL_JMX_PORT
REST ProxyKAFKAREST_JMX_HOSTNAMEKAFKAREST_JMX_PORT
Schema RegistrySCHEMA_REGISTRY_JMX_HOSTNAME

Docker Compose Example

Digging through the Confluent Kafka Docker Images

In general, there are settings that are set in Java to allow for JMX metrics. I recommend reading this blog post to become more familiar. I found Piotr Filipowicz’s post (How to configure JMX in Spring Boot inside Docker?) very useful in understanding this process.

https://github.com/confluentinc/kafka-images/search?q=KAFKA_JMX_OPTS

JConsole Failing (Disconnecting)

This is a common problem that can exist for one of two reasons. If the parameters are not configured correctly for the Docker Compose script that can cause failures but a more subtle reason it can fail is if you are using an incompatible JConsole from the wrong JDK.

Opening JConsole in Debug Mode

jconsole -debug

JDK Version Compatibility Issues

It’s important to use JDK version 11 and not the latest when using JConsole to connect to JMX with Kafka.

Further Reading

These articles helped me.

https://blog.inspeerity.com/devops/Spring-Boot-and-JMX-inside-Docker-container/
https://rmoff.net/2018/09/17/accessing-kafka-docker-containers-jmx-from-host/