How to Configure SSL/TLS with OpenSSL and Self-Signed Certificates on RHEL 7

Encrypting web traffic with SSL/TLS is no longer optional for any server that handles user data, logins, or forms. Even on internal or development servers, a self-signed certificate provides encryption in transit and is far preferable to plain HTTP. On RHEL 7, OpenSSL is installed by default and provides all the tools needed to generate private keys, create Certificate Signing Requests (CSRs), and issue self-signed certificates. This tutorial covers generating a private key and CSR, creating a self-signed certificate, configuring Apache and Nginx to use it, verifying the connection with openssl s_client, and understanding the practical differences between self-signed and CA-signed certificates.

Prerequisites

  • A RHEL 7 server with root or sudo access.
  • Apache (httpd) or Nginx installed and running.
  • OpenSSL installed (verify with openssl version).
  • The mod_ssl package for Apache, or the ngx_http_ssl_module compiled into Nginx.
  • Port 443 open in the firewall.

Step 1: Verify OpenSSL Installation

RHEL 7 ships with OpenSSL 1.0.2 in the base repositories. Confirm it is installed and note the version:

openssl version -a

You should see output similar to OpenSSL 1.0.2k-fips 26 Jan 2017. If OpenSSL is not installed for some reason:

sudo yum -y install openssl openssl-libs

Step 2: Create a Directory for Your Certificate Files

Keeping certificate material in a dedicated, permissions-restricted directory is good practice:

sudo mkdir -p /etc/ssl/private
sudo chmod 700 /etc/ssl/private
sudo mkdir -p /etc/ssl/certs

Step 3: Generate a Private Key

Generate a 2048-bit RSA private key. A 4096-bit key provides stronger security at the cost of slightly more CPU during the TLS handshake — 2048-bit is broadly acceptable for most workloads:

sudo openssl genrsa -out /etc/ssl/private/server.key 2048

Restrict permissions so only root can read the private key:

sudo chmod 600 /etc/ssl/private/server.key

To generate a 4096-bit key instead:

sudo openssl genrsa -out /etc/ssl/private/server.key 4096

If you want to protect the private key with a passphrase (required when using with a production CA), add the -aes256 flag. Be aware that Apache and Nginx will prompt for the passphrase at each startup if the key is encrypted, so most operators leave private keys unencrypted on the server and rely on filesystem permissions instead.

Step 4: Generate a Certificate Signing Request (CSR)

The CSR contains the public key and the Subject Distinguished Name fields that identify your server. When submitting to a CA, you send the CSR; the CA returns a signed certificate. For self-signed certs, you sign the CSR yourself in the next step.

sudo openssl req -new 
  -key /etc/ssl/private/server.key 
  -out /etc/ssl/certs/server.csr

OpenSSL will prompt for the following fields. Fill them in accurately — for self-signed certs these values are informational only, but for CA-signed certs the Common Name must exactly match the fully qualified domain name (FQDN) your users will use:

Country Name (2 letter code) [XX]: US
State or Province Name (full name) []: California
Locality Name (eg, city) [Default City]: San Francisco
Organization Name (eg, company) [Default Company Ltd]: Example Corp
Organizational Unit Name (eg, section) []: IT
Common Name (eg, your name or your server hostname) []: myserver.example.com
Email Address []: [email protected]

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:

Leave the challenge password blank unless specifically required.

You can also pass all fields non-interactively using the -subj flag:

sudo openssl req -new 
  -key /etc/ssl/private/server.key 
  -out /etc/ssl/certs/server.csr 
  -subj "/C=US/ST=California/L=San Francisco/O=Example Corp/CN=myserver.example.com"

Step 5: Create a Self-Signed Certificate

Sign the CSR with the private key to produce a self-signed X.509 certificate valid for 365 days:

sudo openssl x509 -req 
  -days 365 
  -in /etc/ssl/certs/server.csr 
  -signkey /etc/ssl/private/server.key 
  -out /etc/ssl/certs/server.crt

Verify the certificate contents:

sudo openssl x509 -in /etc/ssl/certs/server.crt -noout -text

Confirm the issuer, subject, validity period, and public key in the output. For self-signed certificates, the Issuer and Subject fields will be identical.

Step 6: Configure Apache with SSL

Install mod_ssl if it is not already present:

sudo yum -y install mod_ssl

Installing mod_ssl creates a default configuration file at /etc/httpd/conf.d/ssl.conf. Edit it to point to your certificate and key:

sudo vi /etc/httpd/conf.d/ssl.conf

Locate and update these directives:

SSLCertificateFile      /etc/ssl/certs/server.crt
SSLCertificateKeyFile   /etc/ssl/private/server.key

If you have a CA bundle (intermediate certificates), also set:

SSLCACertificateFile    /etc/ssl/certs/ca-bundle.crt

Ensure the VirtualHost listens on port 443:

<VirtualHost _default_:443>
    ServerName myserver.example.com
    DocumentRoot /var/www/html
    SSLEngine on
    SSLCertificateFile      /etc/ssl/certs/server.crt
    SSLCertificateKeyFile   /etc/ssl/private/server.key
    SSLProtocol all -SSLv2 -SSLv3 -TLSv1 -TLSv1.1
    SSLCipherSuite HIGH:!aNULL:!MD5
</VirtualHost>

Restart Apache:

sudo systemctl restart httpd

Step 7: Configure Nginx with SSL

If you are running Nginx instead of Apache, edit your server block configuration (typically in /etc/nginx/conf.d/):

server {
    listen 443 ssl;
    server_name myserver.example.com;

    ssl_certificate     /etc/ssl/certs/server.crt;
    ssl_certificate_key /etc/ssl/private/server.key;

    ssl_protocols       TLSv1.2 TLSv1.3;
    ssl_ciphers         HIGH:!aNULL:!MD5;
    ssl_prefer_server_ciphers on;

    root /usr/share/nginx/html;
    index index.html;
}

Test the Nginx configuration and reload:

sudo nginx -t
sudo systemctl reload nginx

Step 8: Open Port 443 in the Firewall

sudo firewall-cmd --permanent --add-service=https
sudo firewall-cmd --reload
sudo firewall-cmd --list-all

Step 9: Test with openssl s_client

The openssl s_client command is an invaluable diagnostic tool for TLS connections. Test your server’s certificate and handshake from the command line:

openssl s_client -connect myserver.example.com:443 -showcerts

The output will include the full certificate chain, cipher negotiated, TLS protocol version, and any verification errors. For a self-signed certificate you will see:

Verify return code: 18 (self signed certificate)

This is expected. To test without hostname verification (useful when using an IP address):

openssl s_client -connect 192.168.1.100:443 -verify_return_error 0

To check the certificate expiry date:

echo | openssl s_client -connect myserver.example.com:443 2>/dev/null 
  | openssl x509 -noout -dates

Step 10: Self-Signed vs. CA-Signed Certificates

Understanding the difference is important for choosing the right certificate type for each use case:

  • Self-signed certificates: Signed by the same private key whose public key is in the certificate. Browsers and operating systems do not trust them by default, so users see a warning. Appropriate for internal tools, development environments, and service-to-service communication where you control the trust store.
  • CA-signed certificates: Signed by a Certificate Authority whose root certificate is pre-trusted in major operating systems and browsers. Users see no warning. Required for any public-facing website. Can be obtained for free from Let’s Encrypt (via certbot) or purchased from commercial CAs such as DigiCert or Sectigo.
  • EV certificates: Extended Validation certificates involve a rigorous identity verification process and are typically used by financial institutions.

To add your self-signed certificate to the RHEL 7 system trust store so that local tools like curl stop complaining:

sudo cp /etc/ssl/certs/server.crt /etc/pki/ca-trust/source/anchors/
sudo update-ca-trust extract

Conclusion

You have generated a 2048-bit RSA private key, created a Certificate Signing Request, issued a self-signed X.509 certificate, and configured both Apache and Nginx to serve HTTPS traffic on RHEL 7. The openssl s_client command gives you a powerful way to verify TLS connections and diagnose certificate problems without needing a browser. For production workloads exposed to the public internet, replace the self-signed certificate with one issued by a trusted CA — Let’s Encrypt’s certbot tool can automate this process and handles automatic renewals, making it the preferred choice for most modern deployments.