Kafka Connect: JMX JConsole (Docker)
Yes, you absolutely can connect JConsole to Kafka Connect and extract JMX metrics. If you are experiencing problems getting it to work this blog post will get you through it. There are 2 “gotcha’s” with getting this to work with Docker/Docker Compose. The KAFKA_JMX_HOSTNAME variable must be set and the correct Java JDK must be used. Recently, there was a pull request to include the KAFKA_JMX_HOSTNAME variable in the Docker Compose. I had issues with this because it wasn’t very clear. The other issue I experienced was an incompatibility issue with JConsole.
Git Repo
Docker Environmental Variables
| Component | Hostname | Port |
|---|---|---|
| Kafka | KAFKA_JMX_HOSTNAME | KAFKA_JMX_PORT |
| Kafka Connect | KAFKA_JMX_HOSTNAME | KAFKA_JMX_PORT |
| Zookeeper | KAFKA_JMX_HOSTNAME | KAFKA_JMX_PORT |
| KSQL | KSQL_JMX_HOSTNAME | KSQL_JMX_PORT |
| REST Proxy | KAFKAREST_JMX_HOSTNAME | KAFKAREST_JMX_PORT |
| Schema Registry | SCHEMA_REGISTRY_JMX_HOSTNAME |
Docker Compose Example
Digging through the Confluent Kafka Docker Images
In general, there are settings that are set in Java to allow for JMX metrics. I recommend reading this blog post to become more familiar. I found Piotr Filipowicz’s post (How to configure JMX in Spring Boot inside Docker?) very useful in understanding this process.
https://github.com/confluentinc/kafka-images/search?q=KAFKA_JMX_OPTS
JConsole Failing (Disconnecting)
This is a common problem that can exist for one of two reasons. If the parameters are not configured correctly for the Docker Compose script that can cause failures but a more subtle reason it can fail is if you are using an incompatible JConsole from the wrong JDK.
Opening JConsole in Debug Mode
jconsole -debug
JDK Version Compatibility Issues
It’s important to use JDK version 11 and not the latest when using JConsole to connect to JMX with Kafka.
Further Reading
These articles helped me.
https://blog.inspeerity.com/devops/Spring-Boot-and-JMX-inside-Docker-container/
https://rmoff.net/2018/09/17/accessing-kafka-docker-containers-jmx-from-host/
