Table of Contents

Introduction

Vault, by Hashicorp, is an open-source tool for securely storing secrets and sensitive data in dynamic cloud environments. Packer and Terraform, also developed by Hashicorp, can be used together to create and deploy images of Vault.

In this tutorial, you'll use Packer to create an immutable snapshot of the system with Vault installed, and orchestrate its deployment using Terraform.

For a more detailed version of this tutorial, please refer to How To Build a Hashicorp Vault Server Using Packer and Terraform on the cloud provider.

Prerequisites

cloud provider illustration for: Prerequisites
  • An SSH key you'll use to authenticate with the deployed Vault Droplets, available on your local machine and added to your cloud account. You'll also need its fingerprint, which you can copy from the Security page of your account once you've added it. See the the cloud provider documentation for detailed instructions or the How To Set Up SSH Keys tutorial.

Step 1 — Creating a Packer Template

Create and move into the ~/vault-orchestration directory to store your Vault files:

				
					
mkdir ~/vault-orchestration

cd ~/vault-orchestration

Create separate directories for Packer and Terraform configuration by running:

				
					
mkdir packer terraform

Navigate to the Packer directory:

				
					
cd packer

Using Template Variables

Create a variables.json in your packer subdirectory to store your private variable data:

				
					
nano variables.json

Add the following lines:

				
					
[label ~/vault-orchestration/packer/variables.json]

{

  "do_token": "<^>your_do_api_key<^>",

  "base_system_image": "ubuntu-18-04-x64",

  "region": "nyc3",

  "size": "s-1vcpu-1gb"

}

You'll use these variables in the template you are about to create. You can edit the base image, region, and Droplet size values according to the developer docs.

Replace <^>your_do_api_key<^> with your API key, then save and close the file.

Creating Builders and Provisioners

Create your Packer template for Vault in a file named template.json:

				
					
nano template.json

Add the following lines:

				
					
[label ~/vault-orchestration/packer/template.json]

{

   "builders": [{

       "type": "the cloud provider",

       "api_token": "{{user `do_token`}}",

       "image": "{{user `base_system_image`}}",

       "region": "{{user `region`}}",

       "size": "{{user `size`}}",

       "ssh_username": "root"

   }],

   "provisioners": [{

       "type": "shell",

       "inline": [

           "sleep 30",

           "sudo apt-get update",

           "sudo apt-get install unzip -y",

           "curl -L &lt;^&gt;https://releases.hashicorp.com/vault/1.3.2/vault_1.3.2_linux_amd64.zip&lt;^&gt; -o vault.zip",

           "unzip vault.zip",

           "sudo chown root:root vault",

           "mv vault /usr/local/bin/",

           "rm -f vault.zip"

       ]

}]

}

You define a single the cloud provider builder. Packer will create a temporary Droplet of the defined size, image, and region using the provided API key.

The provisioner will connect to it using SSH with the specified username and will sequentially execute all defined provisioners before creating a the cloud provider Snapshot from the Droplet and deleting it.

It's of type shell, which will execute given commands on the target. The commands in the template will wait 30 seconds for the system to boot up, and will then download and unpack Vault <^>1.3.2<^>. Check the official Vault download page for the most up-to-date version for Linux.

Save and close the file.

Verify the validity of your template:

				
					
packer validate -var-file=variables.json template.json

You'll see the following output:

				
					
[secondary_label Output]

Template validated successfully.

Step 2 — Building the Snapshot

Build your snapshot with the Packer build command:

				
					
packer build -var-file=variables.json template.json

You'll see a lot of output, which will look like this:

				
					
[secondary_label Output]

the cloud provider: output will be in this color.



==&gt; the cloud provider: Creating temporary ssh key for droplet...

==&gt; the cloud provider: Creating droplet...

==&gt; the cloud provider: Waiting for droplet to become active...

==&gt; the cloud provider: Using ssh communicator to connect: ...

==&gt; the cloud provider: Waiting for SSH to become available...

==&gt; the cloud provider: Connected to SSH!

==&gt; the cloud provider: Provisioning with shell script: /tmp/packer-shell035430322

...

==&gt; the cloud provider:   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current

==&gt; the cloud provider:                                  Dload  Upload   Total   Spent    Left  Speed

  the cloud provider: Archive:  vault.zip

==&gt; the cloud provider: 100 45.5M  100 45.5M    0     0   154M      0 --:--:-- --:--:-- --:--:--  153M

  the cloud provider:   inflating: vault

==&gt; the cloud provider: Gracefully shutting down droplet...

==&gt; the cloud provider: Creating snapshot: packer-1581537927

==&gt; the cloud provider: Waiting for snapshot to complete...

==&gt; the cloud provider: Destroying droplet...

==&gt; the cloud provider: Deleting temporary ssh key...

Build 'the cloud provider' finished.



==&gt; Builds finished. The artifacts of successful builds are:

--&gt; the cloud provider: A snapshot was created: 'packer-1581537927' &lt;^&gt;(ID: 58230938)&lt;^&gt; in regions '...'

The last line contains the name of the snapshot (such as packer-1581537927) and its ID in parentheses, highlighted here. Note your ID of the snapshot, because you'll need it in the next step.

If the build process fails due to API errors, wait a few minutes and then retry.

Step 3 — Writing Terraform Configuration

Navigate to the terraform subdirectory:

				
					
cd ~/vault-orchestration/terraform

Create a file named do-provider.tf to store the provider:

				
					
nano do-provider.tf

Add the following lines:

				
					
[label ~/vault-orchestration/terraform/do-provider.tf]

variable "do_token" {

}



variable "ssh_fingerprint" {

}



variable "instance_count" {

default = "1"

}



variable "do_snapshot_id" {

}



variable "do_name" {

default = "vault"

}



variable "do_region" {

}



variable "do_size" {

}



variable "do_private_networking" {

default = true

}



provider "the cloud provider" {

token = var.do_token

}

This file provides the the cloud provider provider with an API key. To specify the values of these variables you'll create a _variable definitions file_ similarly to Packer. The filename must end in either .tfvars or .tfvars.json.

Save and close the file.

Create a variable definitions file:

				
					
nano definitions.tfvars

Add the following lines:

				
					
[label ~/vault-orchestration/terraform/definitions.tf]

do_token         = "&lt;^&gt;your_do_api_key&lt;^&gt;"

ssh_fingerprint  = "&lt;^&gt;your_ssh_key_fingerprint&lt;^&gt;"

do_snapshot_id   = &lt;^&gt;your_do_snapshot_id&lt;^&gt;

do_name          = "vault"

do_region        = "nyc3"

do_size          = "s-1vcpu-1gb"

instance_count   = 1

Replace <^>your_do_api_key<^>, <^>your_ssh_key_fingerprint<^>, and <^>your_do_snapshot_id<^> (the snapshot ID you noted from the previous step). The do_region and do_size parameters must have the same values as in the Packer variables file.

Save and close the file.

Create the following file to store the Vault snapshot deployment configuration:

				
					
nano deployment.tf

Add the following lines:

				
					
[label ~/vault-orchestration/terraform/deployment.tf]

resource "the cloud provider_droplet" "vault" {

count              = var.instance_count

image              = var.do_snapshot_id

name               = var.do_name

region             = var.do_region

size               = var.do_size

private_networking = var.do_private_networking

ssh_keys = [

  var.ssh_fingerprint

]

}



output "instance_ip_addr" {

value = {

  for instance in the cloud provider_droplet.vault:

  instance.id =&gt; instance.ipv4_address

}

description = "The IP addresses of the deployed instances, paired with their IDs."

}

You define a single _resource_ of the type the cloud provider_droplet named vault. You set its parameters according to the variable values and add an SSH key (using its fingerprint) from your cloud account to the Droplet resource. You output the IP addresses of all newly deployed instances to the console.

Save and close the file.

Initialize the directory as a Terraform project:

				
					
terraform init

You'll see the following output:

				
					
[secondary_label Output]



Initializing the backend...



Initializing provider plugins...



The following providers do not have any version constraints in configuration,

so the latest version was installed.



To prevent automatic upgrades to new major versions that may contain breaking

changes, it is recommended to add version = "..." constraints to the

corresponding provider blocks in configuration, with the constraint strings

suggested below.



* provider.the cloud provider: version = "~&gt; 1.14"



Terraform has been successfully initialized!



You may now begin working with Terraform. Try running "terraform plan" to see

any changes that are required for your infrastructure. All Terraform commands

should now work.



If you ever set or change modules or backend configuration for Terraform,

rerun this command to reinitialize your working directory. If you forget, other

commands will detect it and remind you to do so if necessary.

Step 4 — Deploying Vault Using Terraform

Test the validity of your configuration:

				
					
terraform validate

You'll see the following output:

				
					
[secondary_label Output]

Success! The configuration is valid.

Run the plan command to see what Terraform will attempt when it comes to provision the infrastructure:

				
					
terraform plan -var-file="definitions.tfvars"

The output will look similar to:

				
					
[secondary_label Output]

Refreshing Terraform state in-memory prior to plan...

The refreshed state will be used to calculate this plan, but will not be

persisted to local or remote state storage.





------------------------------------------------------------------------



An execution plan has been generated and is shown below.

Resource actions are indicated with the following symbols:

+ create



Terraform will perform the following actions:



# the cloud provider_droplet.vault[0] will be created

+ resource "the cloud provider_droplet" "vault" {

  ...

  }



Plan: 1 to add, 0 to change, 0 to destroy.



------------------------------------------------------------------------



Note: You didn't specify an "-out" parameter to save this plan, so Terraform

can't guarantee that exactly these actions will be performed if

"terraform apply" is subsequently run.

Execute the plan:

				
					
terraform apply -var-file="definitions.tfvars"

The Droplet will finish provisioning and you'll see output similar to this:

				
					
[secondary_label Output]

An execution plan has been generated and is shown below.

Resource actions are indicated with the following symbols:

+ create



Terraform will perform the following actions:



+ the cloud provider_droplet.vault-droplet



...



Plan: 1 to add, 0 to change, 0 to destroy.



...



the cloud provider_droplet.vault-droplet: Creating...



...



Apply complete! Resources: 1 added, 0 changed, 0 destroyed.



Outputs:



instance_ip_addr = {

"181254240" = "&lt;^&gt;your_new_server_ip&lt;^&gt;"

}

Step 5 — Verifying Your Deployed Droplet

Run the following to connect to your new Droplet:

				
					
ssh root@&lt;^&gt;your_server_ip&lt;^&gt;

Once you are logged in, run Vault with:

				
					
vault

You'll see its "help" output:

				
					
[secondary_label Output]

Usage: vault &lt;command&gt; [args]



Common commands:

  read        Read data and retrieves secrets

  write       Write data, configuration, and secrets

  delete      Delete secrets and configuration

  list        List data or secrets

  login       Authenticate locally

  agent       Start a Vault agent

  server      Start a Vault server

  status      Print seal and HA status

  unwrap      Unwrap a wrapped secret



Other commands:

  audit          Interact with audit devices

  auth           Interact with auth methods

  debug          Runs the debug command

  kv             Interact with Vault's Key-Value storage

  lease          Interact with leases

  namespace      Interact with namespaces

  operator       Perform operator-specific tasks

  path-help      Retrieve API help for paths

  plugin         Interact with Vault plugins and catalog

  policy         Interact with policies

  print          Prints runtime configurations

  secrets        Interact with secrets engines

  ssh            Initiate an SSH session

  token          Interact with tokens

Conclusion

You now have an automated system for deploying Hashicorp Vault on cloud servers using Terraform and Packer. To start using Vault, you'll need to initialize it and further configure it. For instructions on how to do that, visit the official docs.

For more tutorials using Terraform, check out our Terraform content page.