The Kafka node allows you to automate work in Kafka, and integrate Kafka with other applications. n8n has built-in support for a wide range of Kafka features, including sending messages.

On this page, you'll find a list of operations the Kafka node supports and links to more resources.

Basic Operations#

• Send message

Example Usage#

This workflow allows you to send updates about the position of the ISS every minute to a topic in Kafka using the Kafka node. You can also find the workflow on n8n.io. This example usage workflow uses the following nodes. - Cron - HTTP Request - Set- Kafka

The final workflow should look like the following image.

1. Cron node#

The Cron node will trigger the workflow every minute.

1. Click on Add Cron Time.
2. Select 'Every Minute' from the Mode dropdown list.
3. Click on Execute Node to run the node.

In the screenshot below, you will notice that the Cron node is configured to trigger the workflow every minute.

2. HTTP Request node (GET)#

This node will make a GET request to the API https://api.wheretheiss.at/v1/satellites/25544/positions to fetch the position of the ISS. This information gets passed on to the next node in the workflow.

1. Enter https://api.wheretheiss.at/v1/satellites/25544/positions in the URL field.
2. Click on the Add Parameter button in the Query Parameters section.
3. Enter timestamps in the Name field.
4. Click on the gears icon next to the Value field and click on Add Expression.
5. Enter the following expression: {{Date.now()}}. This expression will return the current timestamp.
6. Click on Execute Node to run the node.

In the screenshot below, you will notice that the node makes a GET request to the API and returns the information about the location of the ISS.

3. Set node#

We will use the Set node to ensure that only the data that we set in this node gets passed on to the next nodes in the workflow.

1. Click on Add Value and select 'String' from the dropdown list.
2. Enter Name in the Name field.
3. Click on the gears icon next to the Value field and click on Add Expression.
4. Select the following in the Variable Selector section: Nodes > HTTP Request > Output Data > JSON > 0 > name. You can also add the following expression: {{$node["HTTP Request"].json["0"]["name"]}}.
5. Click on Add Value and select 'String' from the dropdown list.
6. Enter Latitude in the Name field.
7. Click on the gears icon next to the Value field and click on Add Expression.
8. Select the following in the Variable Selector section: Nodes > HTTP Request > Output Data > JSON > 0 > latitude. You can also add the following expression: {{$node["HTTP Request"].json["0"]["latitude"]}}.
9. Click on Add Value and select 'String' from the dropdown list.
10. Enter Longitude in the Name field.
11. Click on the gears icon next to the Value field and click on Add Expression.
12. Select the following in the Variable Selector section: Nodes > HTTP Request > Output Data > JSON > 0 > longitude. You can also add the following expression: {{$node["HTTP Request"].json["0"]["longitude"]}}.
13. Click on Add Value and select 'String' from the dropdown list.
14. Enter Timestamp in the Name field.
15. Click on the gears icon next to the Value field and click on Add Expression.
16. Select the following in the Variable Selector section: Nodes > HTTP Request > Output Data > JSON > 0 > timpestamp. You can also add the following expression: {{$node["HTTP Request"].json["0"]["timestamp"]}}.
17. Toggle Keep Only Set to true. We set this option to true to ensure that only the data that we have set in this node get passed on to the next nodes in the workflow.
18. Click on Execute Node to run the node.

In the screenshot below, you will notice that the node uses the data from the previous node and returns the data that we set for the workflow.

4. Kafka node#

This node will send the data from the previous node to the iss-position topic in Kafka. If you have created a topic with a different name in Kafka, you can use that topic name instead.

1. First of all, you'll have to enter credentials for the Kafka node. You can find out how to do that here.
2. Enter the name of the topic in the Topic field.
3. Click on Execute Node to run the node.

In the screenshot below, you will notice that the node sends the data from the previous node to the iss-position topic in Kafka.

Kafka is an open-source distributed event streaming platform that one can use for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications.

Example Usage#

This workflow allows you to receive messages from a topic and send an SMS if the value of the temperature is greater than 50. You can also find the workflow on n8n.io. This example usage workflow would use the following nodes.

• Kafka Trigger
• IF
• Vonage
• No Operation, do nothing

The final workflow should look like the following image.

1. Kafka Trigger node#

This node will trigger the workflow when a new message is sent to the topic test_topic. If you're using a different topic, use that instead. We are sending the message { "id": 1, "temp": 100 } to test_topic.

1. First of all, you'll have to enter credentials for the Kafka Trigger node. You can find out how to do that here.
2. Enter the name of the topic in the Topic field.
3. Enter the group ID in the Group ID field.
4. Click on Add Option and select 'JSON Parse Message'.
5. Toggle JSON Parse Message to true.
6. Click on Execute Node to run the node.

In the screenshot below, you will notice that the node gets triggered when a new message is sent to the topic_test topic in Kafka.

2. IF node#

This node will compare the value of temp that we received in the message from the previous node. If the value is greater than 50, it will return true otherwise false.

1. Click on Add Condition and select 'Number' from the dropdown list.
2. Click on the gears icon next to the Value 1 field and click on Add Expression.
3. Select the following in the Variable Selector section: Nodes > Kafka Trigger > Output Data > JSON > message > temp. You can also add the following expression: {{$node["Kafka Trigger"].json["message"]["temp"]}}.
4. Select 'Larger' from the Operation dropdown list.
5. Set Value 2 to 50.
6. Click on Execute Node to run the node.

In the screenshot below, you will notice that the node returns output for true when the temperature is greater than 50.

3. Vonage node (send: sms)#

This node sends an SMS to the specified phone number when the value of temp is greater than 50.

1. Create a Vonage node connected to the 'true' output of the IF node.
2. You'll have to enter credentials for the Vonage node. You can find out how to do that here.
3. Enter the name of the sender in the From field. If you're using a number, enter the number instead.
4. Enter the receiver's phone number in the To field.

Click on the gears icon next to the Message field and click on Add Expression.

Enter the following in the Expression field.

1
2

Alert!
The value of temp is {{$node["Kafka Trigger"].json["message"]["temp"]}}.

1. Click on Execute Node to run the node.

In the screenshot below, you will notice that the node sends an SMS with the value of temp that we received from the Kafka Trigger node.

4. NoOp node#

Adding this node here is optional, as the absence of this node won't make a difference to the functioning of the workflow.

1. Create a NoOp node connected to the 'false' output of the IF node.
2. Click on Execute Node to run the node.

You can use these credentials to authenticate the following nodes with Kafka.

• Kafka
• Kafka Trigger

Prerequisites#

Have Kafka up and running. You can refer to the official documentation to learn how to do that.

Using Kafka#

1. Enter a client ID in the Client ID field.
2. Retrieve the Kafka Brokers and credentials (if any).
3. Use the Client ID, Brokers (comma separated), Username, and Passwordwith your Kafka node credentials in n8n.