Debugging Kafka Connect Connectors/SMTs in Java
Are you developing a custom Kafka Connect Connector or Single Message Transform (SMT) and need to debug this? This blog post will give you ideas on how to output information to standard output and how to attach a remote debugger to a custom SMT running in a Docker container. This article can save you enormous amounts of time while developing a custom Connector or SMT.
This is part of a blog series on Kafka Connect.
GitHub: Kafka Connect Custom SMT
In order for this blog post to make sense make sure you fit the expectations below. Otherwise you may want to brush up on some of these technologies.
- Capable of programming in Java
- Comfortable using Docker and creating Dockerfiles
- Familiarity with IntelliJ
- Kafka Connect
Debugging Using Standard Out
Debugging the custom Connector or SMT can be done using standard out. Within the Kafka Connect library, is a class that supports logging. The
LoggerFactory has a getLogger() method for retrieving a logger that you can use.
Enabling Tracing in Kafka Connect
For this to work you will also have to enable
TRACE in the log4j properties file in the Docker Container. It is located at
/etc/kafka/docker/connect-log4j.properties. The prefered way is to use Docker Compose but I’ll demonstrate how to do this with a manual update which will allow you to inspect the values which might be useful.
To access the Docker container, run these commands against your container. In the sample below the container is named
cc-connect. Yours may be something else.
# get access to running container docker exec -it --user root cc-connect /bin/bash # once in container cd /etc/kafka/docker/ vim connect-log4j.properties # verify changes cat connect-log4j.properties
Make change in configuration file. Should be last line.
docker restart cc-connect`
Another alternative is if you’re using Docker Compose to update the variable that sets this property.
# was ERROR changed to TRACEs CONNECT_LOG4J_LOGGERS: org.reflections=TRACE
This usually goes as a property on the SMT/Connector class.
private static final Logger log = LoggerFactory.getLogger(SmtEftDb.class);
Outputting data can be done using either
log.info("[+] Configuring Custom SMT");
Debugging Kafka Connect with Docker & Java
I’ll demonstrate how to debug a Kafka Connect Single Message Transform (SMT) running in a Docker container. For me, the easiest way to develop an SMT was to create a custom Docker image that extended the Confluent Cloud’s Kafka Connect Docker image. By doing so, I could also copy in my SMT Plugin through the Docker build process and set environmental variables that are necessary for debugging. The key thing here is to bind the debugger to listen on all adapters by using “*:”.
JAVA_TOOL_OPTIONS environmental variable is all you have to do to enable debugging within the Kafka Connect Docker container.
ARG CONNECTOR_SQLSERVER_VERSION=latest FROM confluentinc/cp-kafka-connect-base:$CONNECTOR_SQLSERVER_VERSION # java debugging ENV JAVA_TOOL_OPTIONS -agentlib:jdwp=transport=dt_socket,address=*:5005,server=y,suspend=n
I can’t stress how important this small detail is but in order for the debugger in IntelliJ to connect to the running Docker container you absolutely must bind all adapters by using “*:”.
This follows basic remote debugging configuration.