Prometheus AlertManager handles alert routing, grouping, and silencing so that your monitoring stack delivers the right notifications to the right people at the right time. On RHEL 8, deploying AlertManager as a systemd service gives you a production-ready setup that starts automatically on boot and integrates cleanly with an existing Prometheus instance. This tutorial walks through downloading the AlertManager binary, writing a routing configuration with Slack and email receivers, wiring it into Prometheus, and testing alerts with the amtool CLI. By the end you will have a fully functional alerting pipeline running on your RHEL 8 server.

Prerequisites

  • RHEL 8 server with a running Prometheus instance (port 9090)
  • A Slack workspace with an incoming webhook URL, or an SMTP server for email alerts
  • A non-root user with sudo privileges
  • Outbound internet access to download binaries from github.com

Step 1 — Download and Install AlertManager

Download the latest AlertManager release from the Prometheus GitHub releases page, extract it, and install the binaries into /usr/local/bin.

cd /tmp
curl -LO https://github.com/prometheus/alertmanager/releases/download/v0.27.0/alertmanager-0.27.0.linux-amd64.tar.gz
tar xzf alertmanager-0.27.0.linux-amd64.tar.gz
sudo cp alertmanager-0.27.0.linux-amd64/alertmanager /usr/local/bin/
sudo cp alertmanager-0.27.0.linux-amd64/amtool /usr/local/bin/
sudo chmod +x /usr/local/bin/alertmanager /usr/local/bin/amtool
sudo useradd --no-create-home --shell /bin/false alertmanager
sudo mkdir -p /etc/alertmanager /var/lib/alertmanager
sudo chown alertmanager:alertmanager /etc/alertmanager /var/lib/alertmanager

Step 2 — Write the AlertManager Configuration

Create /etc/alertmanager/alertmanager.yml defining routes, receivers, and grouping behaviour. The example below routes all critical alerts to Slack and all warnings to email.

sudo tee /etc/alertmanager/alertmanager.yml > /dev/null <<'EOF'
global:
  resolve_timeout: 5m
  smtp_smarthost: 'smtp.example.com:587'
  smtp_from: '[email protected]'
  smtp_auth_username: '[email protected]'
  smtp_auth_password: 'your-smtp-password'

route:
  group_by: ['alertname', 'cluster', 'service']
  group_wait: 30s
  group_interval: 5m
  repeat_interval: 4h
  receiver: 'slack-critical'
  routes:
    - match:
        severity: warning
      receiver: 'email-warning'

receivers:
  - name: 'slack-critical'
    slack_configs:
      - api_url: 'https://hooks.slack.com/services/YOUR/WEBHOOK/URL'
        channel: '#alerts'
        title: '{{ template "slack.default.title" . }}'
        text: '{{ template "slack.default.text" . }}'

  - name: 'email-warning'
    email_configs:
      - to: '[email protected]'
        require_tls: true

inhibit_rules:
  - source_match:
      severity: critical
    target_match:
      severity: warning
    equal: ['alertname', 'cluster', 'service']
EOF
sudo chown alertmanager:alertmanager /etc/alertmanager/alertmanager.yml

Step 3 — Create the systemd Service

Register AlertManager as a systemd service so it starts on boot and restarts automatically on failure.

sudo tee /etc/systemd/system/alertmanager.service > /dev/null <<'EOF'
[Unit]
Description=Prometheus AlertManager
Wants=network-online.target
After=network-online.target

[Service]
User=alertmanager
Group=alertmanager
Type=simple
ExecStart=/usr/local/bin/alertmanager 
  --config.file=/etc/alertmanager/alertmanager.yml 
  --storage.path=/var/lib/alertmanager 
  --web.listen-address=:9093 
  --log.level=info
Restart=always

[Install]
WantedBy=multi-user.target
EOF

sudo systemctl daemon-reload
sudo systemctl enable --now alertmanager
sudo systemctl status alertmanager

Step 4 — Connect AlertManager to Prometheus

Edit your prometheus.yml to point the alerting block at the AlertManager instance, then reload Prometheus.

# Add or update the alerting section in /etc/prometheus/prometheus.yml
alerting:
  alertmanagers:
    - static_configs:
        - targets:
            - 'localhost:9093'

# Also add a rule_files entry if you have alert rules:
rule_files:
  - "/etc/prometheus/rules/*.yml"

# Reload Prometheus to apply changes (no restart needed if --web.enable-lifecycle is set)
curl -X POST http://localhost:9090/-/reload

Step 5 — Create a Sample Alert Rule

Write a simple alert rule file so Prometheus fires an alert when a target is down, giving you something to test with.

sudo mkdir -p /etc/prometheus/rules
sudo tee /etc/prometheus/rules/node.yml > /dev/null < 80
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: "High CPU load on {{ $labels.instance }}"
          description: "CPU load is above 80% for 5 minutes."
EOF
sudo chown prometheus:prometheus /etc/prometheus/rules/node.yml
curl -X POST http://localhost:9090/-/reload

Step 6 — Test Alerts with amtool and Silences

Use amtool to fire a test alert manually, verify it appears in the AlertManager UI, then create a silence to suppress notifications during maintenance.

# Configure amtool to point at local AlertManager
mkdir -p ~/.config/amtool
cat > ~/.config/amtool/config.yml <<'EOF'
alertmanager.url: http://localhost:9093
EOF

# Fire a test alert
amtool alert add alertname=TestAlert severity=critical instance=localhost 
  --annotation=summary="Test alert fired via amtool" 
  --annotation=description="This is a test from amtool."

# List active alerts
amtool alert query

# Create a 2-hour silence for all alerts on localhost
amtool silence add instance=localhost 
  --duration=2h 
  --comment="Scheduled maintenance window"

# List active silences
amtool silence query

# Expire a silence by its ID
amtool silence expire 

# Open AlertManager web UI
echo "AlertManager UI: http://$(hostname -I | awk '{print $1}'):9093"

Conclusion

You now have Prometheus AlertManager running as a systemd service on RHEL 8, with a routing tree that directs critical alerts to Slack and warnings to email. AlertManager groups related alerts to reduce noise, and amtool makes it straightforward to fire test alerts and manage silences from the command line. From here you can extend the configuration with additional receivers, more granular routing rules, and inhibition logic to build a mature alerting workflow.

Next steps: Setting Up Prometheus Recording Rules on RHEL 8, Integrating PagerDuty with Prometheus AlertManager, and Building a Grafana Dashboard for Kubernetes Alerts.