Table of Contents

*The author selected the Free and Open Source Fund to receive a donation as part of the Write for DOnations program.*

Introduction

Ansible is a configuration management tool that executes *playbooks*, which are lists of customizable actions written in YAML on specified target servers. It can perform all bootstrapping operations, like installing and updating software, creating and removing users, and configuring system services. As such, it is suitable for bringing up servers you deploy using Terraform, which are created blank by default.

Ansible and Terraform are not competing solutions, because they resolve different phases of infrastructure and software deployment. Terraform allows you to define and create the infrastructure of your system, encompassing the hardware that your applications will run on. Conversely, Ansible configures and deploys software by executing its playbooks on the provided server instances. Running Ansible on the resources Terraform provisioned directly after their creation allows you to make the resources usable for your use case much faster. It also enables easier maintenance and troubleshooting, because all deployed servers will have the same actions applied to them.

In this tutorial, you’ll deploy Droplets using Terraform, and then immediately after their creation, you’ll bootstrap the Droplets using Ansible. You’ll invoke Ansible directly from Terraform when a resource deploys. You’ll also avoid introducing race conditions using Terraform’s remote-exec and local-exec provisioners in your configuration, which will ensure that the Droplet deployment is fully complete before further setup commences.

Prerequisites

ansible illustration for: Prerequisites
  • A the cloud provider Personal Access Token, which you can create via the the cloud provider Control Panel. You can find instructions in the the cloud provider product documents, How to Create a Personal Access Token.
  • Terraform installed on your local machine and a project set up with the the cloud provider provider. Complete Step 1 and Step 2 of the How To Use Terraform with the cloud provider tutorial and be sure to name the project folder terraform-ansible, instead of loadbalance.

Note: This tutorial has specifically been tested with Terraform 1.0.2.

Step 1 — Defining Droplets

In this step, you’ll define the Droplets on which you’ll later run an Ansible playbook, which will set up the Apache web server.

Assuming you are in the terraform-ansible directory, which you created as part of the prerequisites, you’ll define a Droplet resource, create three copies of it by specifying count, and output their IP addresses. You’ll store the definitions in a file named droplets.tf. Create and open it for editing by running:

				
					
nano droplets.tf

Add the following lines:

				
					
[label ~/terraform-ansible/droplets.tf]

resource "the cloud provider_droplet" "web" {

 count = 3

 image = "ubuntu-18-04-x64"

 name = "web-${count.index}"

 region = "fra1"

 size = "s-1vcpu-1gb"



 ssh_keys = [

 data.the cloud provider_ssh_key.<^>terraform<^>.id

 ]

}



output "droplet_ip_addresses" {

 value = {

 for droplet in the cloud provider_droplet.web:

 droplet.name => droplet.ipv4_address

 }

}

Here you define a Droplet resource running Ubuntu 18.04 with 1GB RAM on a CPU core in the region fra1. Terraform will pull the SSH key you defined in the prerequisites from your account and add it to the provisioned Droplet with the specified unique ID list element passed into ssh_keys. Terraform will deploy the Droplet three times because the count parameter is set. The output block following it will show the IP addresses of the three Droplets. The loop traverses the list of Droplets, and for each instance, pairs its name with its IP address and appends it to the resulting map.

Save and close the file when you’re done.

You have now defined the Droplets that Terraform will deploy. In the next step, you’ll write an Ansible playbook that will execute on each of the three deployed Droplets and will deploy the Apache web server. You’ll later go back to the Terraform code and add in the integration with Ansible.

Step 2 — Writing an Ansible Playbook

You’ll now create an Ansible playbook that performs the initial server setup tasks, such as creating a new user and upgrading the installed packages. You’ll instruct Ansible on what to do by writing *tasks*, which are units of action that are executed on target hosts. Tasks can use built-in functions, or specify custom commands to be run. Besides the tasks for the initial setup, you’ll also install the Apache web server and enable its mod_rewrite module.

Before writing the playbook, ensure that your public and private SSH keys, which correspond to the one in your cloud account, are available and accessible on the machine from which you’re running Terraform and Ansible. A typical location for storing them on Linux would be ~/.ssh (although you can store them in other places).

Note: On Linux, you'll need to ensure that the private key file has appropriate permissions. You can set them by running:

				
					
chmod 600 <^>your_private_key_location<^>

You already have a variable for the private key defined, so you'll only need to add one for the public key location.

Open provider.tf for editing by running:

				
					
nano provider.tf

Add the following line:

				
					
[label ~/terraform-ansible/provider.tf]

terraform {

 required_providers {

 the cloud provider = {

 source = "the cloud provider/the cloud provider"

 version = "~> 2.0"

 }

 }

}



variable "do_token" {}

variable "pvt_key" {}

<^>variable "pub_key" {}<^>



provider "the cloud provider" {

 token = var.do_token

}



data "the cloud provider_ssh_key" "terraform" {

 name = "terraform"

}

When you're done, save and close the file.

With the pub_key variable now defined, you’ll start writing the Ansible playbook. You’ll store it in a file called apache-install.yml. Create and open it for editing:

				
					
nano apache-install.yml

You’ll be building the playbook gradually. First, you’ll need to define on which hosts the playbook will run, its name, and if the tasks should be run as root. Add the following lines:

				
					
[label ~/terraform-ansible/apache-install.yml]

- become: yes

 hosts: all

 name: apache-install

By setting become to yes, you instruct Ansible to run commands as the superuser, and by specifying all for hosts, you allow Ansible to run the tasks on any given server—even the ones passed in through the command line, as Terraform does.

The first task that you’ll add will create a new, non-root user. Append the following task definition to your playbook:

				
					
[label ~/terraform-ansible/apache-install.yml]

 tasks:

 - name: Add the user 'sammy' and add it to 'sudo'

 user:

 name: sammy

 group: sudo

You first define a list of tasks and then add a task to it. It will create a user named sammy and grant them superuser access using sudo by adding them to the appropriate group.

The next task will add your public SSH key to the user, so you’ll be able to connect to it later on:

				
					
[label ~/terraform-ansible/apache-install.yml]

 - name: Add SSH key to 'sammy'

 authorized_key:

 user: sammy

 state: present

 key: "{{ lookup('file', pub_key) }}"

This task will ensure that the public SSH key, which is looked up from a local file, is present on the target. You’ll supply the value for the pub_key variable from Terraform in the next step.

You can now order the installation of Apache and the mod_rewrite module by appending the following tasks:

				
					
[label ~/terraform-ansible/apache-install.yml]

 - name: Wait for apt to unlock

 become: yes

 shell: while sudo fuser /var/lib/dpkg/lock >/dev/null 2>&1; do sleep 5; done;

 

 - name: Install apache2

 apt: 

 name: apache2

 update_cache: yes

 state: latest

 

 - name: Enable mod_rewrite

 apache2_module:

 name: rewrite

 state: present

 notify:

 - Restart apache2



 handlers:

 - name: Restart apache2

 service:

 name: apache2

 state: restarted

The first task will wait until any previous package installation using the apt package manager is complete. The second task will run apt to install Apache. Then, the third one will ensure that the mod_rewrite module is present. After it’s enabled, you need to ensure that you restart Apache, which you can’t configure from the task itself. To resolve that, you call a handler to issue the restart.

At this point, your playbook will look like the following:

				
					
[label ~/terraform-ansible/apache-install.yml]

- become: yes

 hosts: all

 name: apache-install

 tasks:

 - name: Add the user 'sammy' and add it to 'sudo'

 user:

 name: sammy

 group: sudo

 

 - name: Add SSH key to 'sammy'

 authorized_key:

 user: sammy

 state: present

 key: "{{ lookup('file', pub_key) }}"

 

 - name: Wait for apt to unlock

 become: yes

 shell: while sudo fuser /var/lib/dpkg/lock >/dev/null 2>&1; do sleep 5; done;

 

 - name: Install apache2

 apt:

 name: apache2

 update_cache: yes

 state: latest

 

 - name: Enable mod_rewrite

 apache2_module:

 name: rewrite 

 state: present

 notify:

 - Restart apache2



 handlers:

 - name: Restart apache2

 service:

 name: apache2

 state: restarted

When you're done, check that indentations of all YAML elements are correct and match the ones shown above. This is all you need to define on the Ansible side, so save and close the playbook. You’ll now modify the Droplet deployment code to execute this playbook when the Droplets have finished provisioning.

Step 3 — Running Ansible on Deployed Droplets

Now that you have defined the actions Ansible will take on the target servers, you’ll modify the Terraform configuration to run it upon Droplet creation.

Terraform offers two provisioners that execute commands: local-exec and remote-exec, which run commands locally or remotely (on the target), respectively. remote-exec requires connection data, such as type and access keys, while local-exec does everything on the machine Terraform is executing on, and so does not require connection information. It’s important to note that local-exec runs immediately after the resource you have defined it for has finished provisioning; therefore, it does not wait for the resource to actually boot up. It runs after the cloud platform acknowledges its presence in the system.

You’ll now add provisioner definitions to your Droplet to run Ansible after deployment. Open droplets.tf for editing:

				
					
nano droplets.tf

Add the highlighted lines:

				
					
[label ~/terraform-ansible/droplets.tf]

resource "the cloud provider_droplet" "web" {

 count = 3

 image = "ubuntu-18-04-x64"

 name = "web-${count.index}"

 region = "fra1"

 size = "s-1vcpu-1gb"



 ssh_keys = [

 data.the cloud provider_ssh_key.terraform.id

 ]



 <^>provisioner "remote-exec" {<^>

 <^>inline = ["sudo apt update", "sudo apt install python3 -y", "echo Done!"]<^>



 <^>connection {<^>

 <^>host = self.ipv4_address<^>

 <^>type = "ssh"<^>

 <^>user = "root"<^>

 <^>private_key = file(var.pvt_key)<^>

 <^>}<^>

 <^>}<^>



 <^>provisioner "local-exec" {<^>

 <^>command = "ANSIBLE_HOST_KEY_CHECKING=False ansible-playbook -u root -i '${self.ipv4_address},' --private-key ${var.pvt_key} -e 'pub_key=${var.pub_key}' apache-install.yml"<^>

 <^>}<^>

}



output "droplet_ip_addresses" {

 value = {

 for droplet in the cloud provider_droplet.web:

 droplet.name => droplet.ipv4_address

 }

}

Like Terraform, Ansible runs locally and connects to the target servers via SSH. To run it, you define a local-exec provisioner in the Droplet definition that runs the ansible-playbook command. This passes in the username (root), the IP of the current Droplet (retrieved with ${self.ipv4_address}), the SSH public and private keys, and specifies the playbook file to run (apache-install.yml). By setting the ANSIBLE_HOST_KEY_CHECKING environment variable to False, you skip checking if the server was connected to beforehand.

As was noted, the local-exec provisioner runs without waiting for the Droplet to become available, so the execution of the playbook may precede the actual availability of the Droplet. To remedy this, you define the remote-exec provisioner to contain commands to execute on the target server. For remote-exec to execute, the target server must be available. Since remote-exec runs before local-exec, the server will be fully initialized by the time Ansible is invoked. python3 comes preinstalled on Ubuntu 18.04, so you can comment out or remove the command as necessary.

When you’re done making changes, save and close the file.

Then, deploy the Droplets by running the following command. Remember to replace <^>private_key_location<^> and <^>public_key_location<^> with the locations of your private and public keys respectively:

				
					
terraform apply -var "do_token=${DO_PAT}" -var "pvt_key=&lt;^&gt;private_key_location&lt;^&gt;" -var "pub_key=&lt;^&gt;public_key_location&lt;^&gt;"

The output will be long. Your Droplets will provision and then a connection will establish with each. Next the remote-exec provisioner will execute and install python3:

				
					
[secondary_label Output]

...

the cloud provider_droplet.web[1] (remote-exec): Connecting to remote host via SSH...

the cloud provider_droplet.web[1] (remote-exec): Host: ...

the cloud provider_droplet.web[1] (remote-exec): User: root

the cloud provider_droplet.web[1] (remote-exec): Password: false

the cloud provider_droplet.web[1] (remote-exec): Private key: true

the cloud provider_droplet.web[1] (remote-exec): Certificate: false

the cloud provider_droplet.web[1] (remote-exec): SSH Agent: false

the cloud provider_droplet.web[1] (remote-exec): Checking Host Key: false

the cloud provider_droplet.web[1] (remote-exec): Connected!

...

After that, Terraform will run the local-exec provisioner for each of the Droplets, which executes Ansible. The following output shows this for one of the Droplets:

				
					
[secondary_label Output]

...

the cloud provider_droplet.web[2] (local-exec): Executing: ["/bin/sh" "-c" "ANSIBLE_HOST_KEY_CHECKING=False ansible-playbook -u root -i 'ip_address,' --private-key private_key_location -e 'pub_key=public_key_location' apache-install.yml"]



the cloud provider_droplet.web[2] (local-exec): PLAY [apache-install] **********************************************************



the cloud provider_droplet.web[2] (local-exec): TASK [Gathering Facts] *********************************************************

the cloud provider_droplet.web[2] (local-exec): ok: [ip_address]



the cloud provider_droplet.web[2] (local-exec): TASK [Add the user 'sammy' and add it to 'sudo'] *******************************

the cloud provider_droplet.web[2] (local-exec): changed: [ip_address]



the cloud provider_droplet.web[2] (local-exec): TASK [Add SSH key to 'sammy''] *******************************

the cloud provider_droplet.web[2] (local-exec): changed: [ip_address]



the cloud provider_droplet.web[2] (local-exec): TASK [Update all packages] *****************************************************

the cloud provider_droplet.web[2] (local-exec): changed: [ip_address]



the cloud provider_droplet.web[2] (local-exec): TASK [Install apache2] *********************************************************

the cloud provider_droplet.web[2] (local-exec): changed: [ip_address]



the cloud provider_droplet.web[2] (local-exec): TASK [Enable mod_rewrite] ******************************************************

the cloud provider_droplet.web[2] (local-exec): changed: [ip_address]



the cloud provider_droplet.web[2] (local-exec): RUNNING HANDLER [Restart apache2] **********************************************

the cloud provider_droplet.web[2] (local-exec): changed: [ip_address]



the cloud provider_droplet.web[2] (local-exec): PLAY RECAP *********************************************************************

the cloud provider_droplet.web[2] (local-exec): [ip_address] : ok=7 changed=6 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0



...

At the end of the output, you’ll receive a list of the three Droplets and their IP addresses:

				
					
[secondary_label Output]

droplet_ip_addresses = {

 "web-0" = "..."

 "web-1" = "..."

 "web-2" = "..."

}

You can now navigate to one of the IP addresses in your browser. You will reach the default Apache welcome page, signifying the successful installation of the web server.

This means that Terraform provisioned your servers and your Ansible playbook executed on it successfully.

To check that the SSH key was correctly added to sammy on the provisioned Droplets, connect to one of them with the following command:

				
					
ssh -i &lt;^&gt;private_key_location&lt;^&gt; &lt;^&gt;sammy@droplet_ip_address&lt;^&gt;

Remember to put in the private key location and the IP address of one of the provisioned Droplets, which you can find in your Terraform output.

The output will look similar to the following:

				
					
[secondary_label Output]

Welcome to Ubuntu 18.04.5 LTS (GNU/Linux 4.15.0-121-generic x86_64)



 * Documentation: https://help.ubuntu.com

 * Management: https://landscape.canonical.com

 * Support: https://ubuntu.com/advantage



 System information as of ...



 System load: 0.0 Processes: 88

 Usage of /: 6.4% of 24.06GB Users logged in: 0

 Memory usage: 20% IP address for eth0: &lt;^&gt;ip_address&lt;^&gt;

 Swap usage: 0% IP address for eth1: &lt;^&gt;ip_address&lt;^&gt;



0 packages can be updated.

0 updates are security updates.



New release '20.04.1 LTS' available.

Run 'do-release-upgrade' to upgrade to it.





*** System restart required ***

Last login: ...

...

You’ve successfully connected to the target and obtained shell access for the sammy user, which confirms that the SSH key was correctly configured for that user.

You can destroy the deployed Droplets by running the following command, entering yes when prompted:

				
					
terraform destroy -var "do_token=${DO_PAT}" -var "pvt_key=&lt;^&gt;private_key_location&lt;^&gt;" -var "pub_key=&lt;^&gt;public_key_location&lt;^&gt;"

In this step, you have added in Ansible playbook execution as a local-exec provisioner to your Droplet definition. To ensure that the server is available for connections, you’ve included the remote-exec provisioner, which can serve to install the python3 prerequisite, after which Ansible will run.

Conclusion

Terraform and Ansible together form a flexible workflow for spinning up servers with the needed software and hardware configurations. Running Ansible directly as part of the Terraform deployment process allows you to have the servers up and bootstrapped with dependencies for your development work and applications much faster.

This tutorial is part of the How To Manage Infrastructure with Terraform series. The series covers a number of Terraform topics, from installing Terraform for the first time to managing complex projects.

You can also find additional Ansible content resources on our Ansible topic page.