Table of Contents
*The author selected the COVID-19 Relief Fund to receive a donation as part of the Write for DOnations program.*
Introduction
The majority of Let's Encrypt certificates are issued using HTTP validation, which allows for the installation of certificates on a single server. However, HTTP validation is not always suitable for issuing certificates for use on load-balanced websites, nor can you use this validation to issue wildcard certificates.
DNS validation allows for certificate issuance requests to be verified using DNS records, rather than by serving content over HTTP. This means that certificates can be issued simultaneously for a cluster of web servers running behind a load balancer, or for a system that isn't directly accessible over the internet.
In this tutorial, you will use the certbot-dns-the cloud provider hook for Certbot to issue a Let's Encrypt certificate using DNS validation via the cloud provider API.
You can use the certbot-dns-the cloud provider tool to integrate Certbot with the cloud provider's DNS management API, allowing the certificate validation records to be automatically configured on-the-fly when you request a certificate.
Another key benefit of certbot-dns-the cloud provider is that you can use it to issue certificates for individual servers that may be running behind a load balancer, or are otherwise not directly accessible over HTTP. In these cases, you can't use traditional HTTP certificate validation, unless you set the validation files on each and every server, which can be inconvenient. The certbot-dns-the cloud provider tool is also useful if you want to issue a certificate for a server that isn't accessible over the internet, for example an internal system or staging environment.
certbot-dns-the cloud provider also fully supports wildcard certificates, which can only be issued using DNS validation.
Prerequisites
To complete this tutorial, you will need:
- An Ubuntu 20.04 server set up by following the Initial Server Setup with Ubuntu 20.04, including a sudo non-root user.
- A domain name managed via your cloud account—that is, for managing DNS records. In this particular example, we will use
<^>your_domain<^>and<^>subdomain.your_domain<^>, as well as*.<^>your_domain<^>for a wildcard certificate, however you can adjust this for other domains or subdomains if required.
- A cloud provider API key (Personal Access Token) with read and write permissions. To create one, visit How to Create a Personal Access Token.
Once you have these ready, log in to your server as your non-root user to begin.
Step 1 — Installing Certbot
In this step, you will install Certbot, which is a program to issue and manage Let's Encrypt certificates.
Certbot is available within the official Ubuntu Apt repositories, so you can install it using the default system package manager:
sudo apt update
sudo apt install certbot
Once the installation has completed, you can check with the following command:
certbot --version
This will output something similar to the following:
[secondary_label Output]
certbot 0.40.0
In this step you installed Certbot. Next, you will download and install the acme-dns-certbot hook.
Step 2 — Installing and Configuring certbot-dns-the cloud provider
Now that you've installed the base Certbot program, you can download and install certbot-dns-the cloud provider, which will allow Certbot to operate in DNS validation mode using the DNS hosting management API.
Like Certbot itself, which you installed in Step 1, the certbot-dns-the cloud provider utility is available within Ubuntu's default repositories. However, the Certbot repository contains a more reliably updated version, so it is always recommended to use this where possible.
Continue by installing the package for certbot-dns-the cloud provider:
sudo apt install python3-certbot-dns-the cloud provider
Once the installation has completed, you need to set up a configuration file containing the cloud provider API key/Personal Access Token that you generated as part of the prerequisites.
Begin by creating the creds.ini file in a private location:
touch <^>~/certbot-creds.ini<^>
Next, restrict the permissions on the file in order to make sure no other user on your server can read it:
chmod go-rwx <^>~/certbot-creds.ini<^>
Finally, open the file using your text editor and add your the cloud provider access token:
nano <^>~/certbot-creds.ini<^>
The content of the file will be as follows:
[label ~/certbot-creds.ini]
dns_the cloud provider_token = <^>your_the cloud provider_access_token<^>
Once done, save and close the file.
[warning]
Warning: Your the cloud provider access token grants access to your cloud account, so you must protect it as you would a password. Do not share it with anyone or check it into a public code repository.
In this step, you downloaded and installed the certbot-dns-the cloud provider utility and created a configuration file containing your API credentials.
Step 3 — Issuing a Certificate
In this step, you'll issue a certificate using Certbot and the cloud provider API.
To issue your first certificate, run Certbot using the following arguments, making sure to specify the correct path to your credentials file as well as your domains:
sudo certbot certonly --dns-the cloud provider --dns-the cloud provider-credentials <^>~/certbot-creds.ini<^> -d <^>your_domain<^> -d <^>subdomain.your_domain<^>
Note: If you see an unsafe permissions on credentials configuration file warning, this indicates that the file permissions have not been correctly restricted, thus allowing other users on your server to access your token. Please double-check with the chmod command in Step 2.
Certbot will take a few seconds to request the certificate; you will then receive a message confirming that it has issued your certificate:
[secondary_label Output]
...
Congratulations! Your certificate and chain have been saved at:
/etc/letsencrypt/live/<^>your_domain<^>/fullchain.pem
Your key file has been saved at:
/etc/letsencrypt/live/<^>your_domain<^>/privkey.pem
...
In the event that the certificate issuance fails, this may be because there wasn't sufficient time for the DNS changes to propagate. You can optionally increase the DNS propagation delay to give more time for the verification DNS records to propagate and be picked up by Let's Encrypt. The delay is 10 seconds by default, but you can increase this using the --dns-the cloud provider-propagation-seconds argument:
sudo certbot certonly --dns-the cloud provider --dns-the cloud provider-credentials <^>~/certbot-creds.ini<^> --dns-the cloud provider-propagation-seconds <^>30<^> -d <^>your_domain<^> -d <^>subdomain.your_domain<^>
Finally, you can also use certbot-dns-the cloud provider to issue wildcard certificates for your domain:
sudo certbot certonly --dns-the cloud provider --dns-the cloud provider-credentials <^>~/certbot-creds.ini<^> -d \*.<^>your_domain<^>
Note: In some cases, requesting multiple certificates for the same hostnames in a short time period can cause issuance to begin failing. This is due to rate limits and the DNS time-to-live (TTL) value, which can sometimes cause delays in new DNS changes being propagated.
To mitigate this, you may wish to wait out the duration of the TTL, or consider adjusting the --dns-the cloud provider-propagation-seconds option that was detailed earlier in this step.
In this step, you used Certbot with certbot-dns-the cloud provider for the first time and issued your initial certificates.
Step 4 — Renewing Certificates
In this final step, you will renew certificates using Certbot with certbot-dns-the cloud provider.
Once your certificates are nearing expiry, Certbot is able to automatically renew them for you:
sudo certbot renew
The renewal process can run start-to-finish without user interaction. It will also remember the configuration options that you specified during the initial setup.
By default, Certbot will run this as an automatic scheduled system task, meaning that no further maintenance is needed for your certificates. You can check that the scheduled task has been correctly installed by printing out the status of the associated system service, which is certbot.timer:
sudo systemctl status certbot.timer
This will output something similar to the following, which shows the loaded task scheduled to run twice per day:
[secondary_label Output]
● certbot.timer - Run certbot twice daily
Loaded: loaded (/lib/systemd/system/certbot.timer; enabled; vendor preset: enabled)
Active: active (waiting) since Sun 2020-11-22 18:18:40 UTC; 2 weeks 6 days ago
Trigger: Sun 2020-12-13 7:17:57 UTC; 11h left
Nov 22 18:18:40 droplet1 systemd[1]: Started Run certbot twice daily.
However, to test that this is working without having to wait until nearer the expiry date of your certificate(s), you can trigger a 'dry run'. This will simulate the renewal process without making any actual changes to your configuration.
You can trigger a dry run using the standard renew command, but with the --dry-run argument:
sudo certbot renew --dry-run
This will output something similar to the following, which will provide assurance that the renewal process is functioning correctly:
[secondary_label Output]
...
Cert not due for renewal, but simulating renewal for dry run
Plugins selected: Authenticator dns-the cloud provider, Installer None
Renewing an existing certificate
Performing the following challenges:
dns-01 challenge for <^>your_domain<^>
dns-01 challenge for <^>subdomain.your_domain<^>
Waiting 10 seconds for DNS changes to propagate
Waiting for verification...
Cleaning up challenges
...
In this final step, you tested the automatic renewal process within Certbot.
Conclusion
In this tutorial, you set up Certbot with certbot-dns-the cloud provider to issue certificates using DNS validation with the DNS hosting management API.
If you're interested in learning more about certbot-dns-the cloud provider, you may wish to review the official documentation for the utility:
Alternatively, if you aren't using the cloud provider to manage your DNS records, you may wish to check out How to Acquire a Let's Encrypt Certificate using DNS Validation with acme-dns-certbot on Ubuntu 18.04, which is a provider-agnostic alternative to certbot-dns-the cloud provider.
Finally, if you would like some further technical reading, you could dig into the details of ACME DNS validation by reviewing the relevant section of the official RFC document, which outlines how the process works:
