Connecting Kafka Tool to Azure Event Hubs
I find connecting the Kafka Tool or any tool to Kafka at times to be a cumbersome process. It seems like each service has some nuance for either security, ports, or connecting that makes it tricky or confusing. I’ll show you how to easily connect your Kafka Tool to Azure Event Hubs so you can see data in your Event Hubs (topics).
Make Sure Kafka Services is Enabled
If you are using a Standard version or higher of Azure Event Hubs the “Kafka Surface” will be enabled. This is not supported by the “Basic” tier of Azure Event Hubs.
Get Configuration Values from Azure Event Hubs
Under the Azure Event Hubs Namespace there is a menu item for “Shared access policies”, here you will be able to create a policy or use the RotoManageSharedAccessKey to connect to the Kafka Service. You will only need the connection string so copy that.
Kafka Tool Configuration for Azure Event Hubs
I’ll post pictures of the screens and explain what the values should be here.
Properties
Broker Security
Advanced
Be sure to set the port to the correct port. Azure Event Hubs uses port 9093 and not the default 9092 port of Kafka.
JAAS Config
You can read more on Microsoft’s site here: https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-for-kafka-ecosystem-overview#shared-access-signature-sas
|
1 2 3 |
org.apache.kafka.common.security.plain.PlainLoginModule required username="$ConnectionString" password="Endpoint=sb://{SERVICE}.servicebus.windows.net/;SharedAccessKeyName={POLICY_NAME};SharedAccessKey={KEY}"; |
Connect
It make take a few seconds to connect but once it does you’re good.
Further Reading
https://docs.microsoft.com/en-us/azure/event-hubs/apache-kafka-developer-guide
