Apache Kafka is a distributed event streaming platform capable of handling millions of messages per second, making it a popular choice for real-time data pipelines, log aggregation, and event-driven microservices. Kafka is distributed as a self-contained binary archive and does not require a package manager installation, so the setup process involves downloading, configuring, and wrapping Kafka in systemd services. In this tutorial you will install Java 11, download and configure Kafka on RHEL 8, start ZooKeeper and the Kafka broker, create a test topic, and produce and consume messages from the command line.

Prerequisites

  • RHEL 8 server with at least 4 GB RAM and 20 GB free disk space for logs
  • Root or sudo access
  • Active internet connection to download the Kafka binary
  • firewalld running

Step 1 — Install Java 11

Kafka requires Java 8 or later. Java 11 LTS is the recommended version on RHEL 8:

sudo dnf install -y java-11-openjdk-headless
java -version

The output should confirm openjdk version "11".

Step 2 — Download and Install Kafka

Download the latest stable Kafka binary from the official Apache mirror. Adjust the version number to the latest release listed at kafka.apache.org/downloads:

cd /opt
sudo curl -LO https://downloads.apache.org/kafka/3.7.0/kafka_2.13-3.7.0.tgz
sudo tar xzf kafka_2.13-3.7.0.tgz
sudo mv kafka_2.13-3.7.0 kafka
sudo rm kafka_2.13-3.7.0.tgz

Create a dedicated system user to run Kafka services:

sudo useradd --no-create-home --shell /bin/false kafka
sudo chown -R kafka:kafka /opt/kafka

Step 3 — Configure the Kafka Broker

Edit the main broker configuration file. The key properties to set for a single-node installation are the broker ID, log directory, and ZooKeeper connection string:

sudo vi /opt/kafka/config/server.properties

Verify or update these lines:

broker.id=0
log.dirs=/opt/kafka/kafka-logs
zookeeper.connect=localhost:2181
num.partitions=3
default.replication.factor=1
log.retention.hours=168
log.segment.bytes=1073741824
auto.create.topics.enable=false

Create the log directory and set ownership:

sudo mkdir -p /opt/kafka/kafka-logs
sudo chown kafka:kafka /opt/kafka/kafka-logs

Step 4 — Create systemd Services for ZooKeeper and Kafka

Create the ZooKeeper service unit:

sudo vi /etc/systemd/system/zookeeper.service
[Unit]
Description=Apache ZooKeeper
After=network.target

[Service]
User=kafka
Group=kafka
ExecStart=/opt/kafka/bin/zookeeper-server-start.sh /opt/kafka/config/zookeeper.properties
ExecStop=/opt/kafka/bin/zookeeper-server-stop.sh
Restart=on-failure
RestartSec=10s
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

Create the Kafka broker service unit:

sudo vi /etc/systemd/system/kafka.service
[Unit]
Description=Apache Kafka Broker
After=zookeeper.service
Requires=zookeeper.service

[Service]
User=kafka
Group=kafka
Environment="JAVA_HOME=/usr/lib/jvm/jre-11"
ExecStart=/opt/kafka/bin/kafka-server-start.sh /opt/kafka/config/server.properties
ExecStop=/opt/kafka/bin/kafka-server-stop.sh
Restart=on-failure
RestartSec=10s
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target
sudo systemctl daemon-reload
sudo systemctl enable --now zookeeper
sudo systemctl enable --now kafka
sudo systemctl status zookeeper kafka

Step 5 — Create a Topic and Test Message Flow

Create a test topic with three partitions and a replication factor of one (suitable for a single-node cluster):

/opt/kafka/bin/kafka-topics.sh --create 
  --topic test-events 
  --partitions 3 
  --replication-factor 1 
  --bootstrap-server localhost:9092

/opt/kafka/bin/kafka-topics.sh --list --bootstrap-server localhost:9092

In one terminal session, start a console producer and type a few messages:

/opt/kafka/bin/kafka-console-producer.sh 
  --topic test-events 
  --bootstrap-server localhost:9092

In a second terminal session, start a consumer from the beginning of the topic:

/opt/kafka/bin/kafka-console-consumer.sh 
  --topic test-events 
  --from-beginning 
  --bootstrap-server localhost:9092

Messages typed in the producer terminal should appear immediately in the consumer terminal, confirming end-to-end message flow.

Step 6 — Open Firewall Ports

Allow only trusted application hosts to connect to the Kafka broker on port 9092:

sudo firewall-cmd --permanent --add-rich-rule='rule family="ipv4" source address="192.168.1.0/24" port port="9092" protocol="tcp" accept'
sudo firewall-cmd --reload

Conclusion

You have installed Java 11 on RHEL 8, downloaded and configured a Kafka broker, created systemd unit files for both ZooKeeper and Kafka, verified message production and consumption from the command line, and restricted broker access with firewalld. This single-node setup is suitable for development and testing; production environments should use a three-node cluster with replication.

Next steps: Configuring a Multi-Broker Kafka Cluster on RHEL 8, Securing Kafka with TLS and SASL Authentication, and Monitoring Kafka with Prometheus JMX Exporter.