Connecting Kafka Tool to Azure Event Hubs

I find connecting the Kafka Tool or any tool to Kafka at times to be a cumbersome process. It seems like each service has some nuance for either security, ports, or connecting that makes it tricky or confusing. I’ll show you how to easily connect your Kafka Tool to Azure Event Hubs so you can see data in your Event Hubs (topics).

Make Sure Kafka Services is Enabled

If you are using a Standard version or higher of Azure Event Hubs the “Kafka Surface” will be enabled. This is not supported by the “Basic” tier of Azure Event Hubs.

Get Configuration Values from Azure Event Hubs

Under the Azure Event Hubs Namespace there is a menu item for “Shared access policies”, here you will be able to create a policy or use the RotoManageSharedAccessKey to connect to the Kafka Service. You will only need the connection string so copy that.

Kafka Tool Configuration for Azure Event Hubs

I’ll post pictures of the screens and explain what the values should be here.

Properties

Broker Security

Advanced

Be sure to set the port to the correct port. Azure Event Hubs uses port 9093 and not the default 9092 port of Kafka.

JAAS Config

You can read more on Microsoft’s site here: https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-for-kafka-ecosystem-overview#shared-access-signature-sas

org.apache.kafka.common.security.plain.PlainLoginModule required 
username="$ConnectionString" 
password="Endpoint=sb://{SERVICE}.servicebus.windows.net/;SharedAccessKeyName={POLICY_NAME};SharedAccessKey={KEY}";

Connect

It make take a few seconds to connect but once it does you’re good.

Further Reading

https://docs.microsoft.com/en-us/azure/event-hubs/apache-kafka-developer-guide

LEAVE A REPLY

Please enter your comment!
Please enter your name here