URL: https://www.progressiverobot.com/webinar-series-deploying-stateful-services-in-kubernetes/

This article supplements a webinar series on deploying and managing containerized workloads in the cloud. The series covers the essentials of containers, including managing container lifecycles, deploying multi-container applications, scaling workloads, and working with Kubernetes. It also highlights best practices for running stateful applications.

This tutorial includes the concepts and commands in the fifth session of the series, Deploying Stateful Services in Kubernetes.

[youtube _zPuVGh2z9c 480 854]

###Introduction

Kubernetes is an open-source container orchestration tool for managing containerized applications. In the previous parts of this series, you learned the building blocks of Kubernetes and packaged containers as Kubernetes ReplicaSets. While ReplicaSets ensure the availability of stateless Pods, they cannot be used with stateful workloads such as database clusters.

While it may be easy to package, deploy, manage, and scale contemporary cloud-native applications in Kubernetes, deploying and managing traditional workloads such as databases and content management systems in a containerized environment calls for a different approach. StatefulSets bring the flexibility of Kubernetes ReplicaSet to stateful workloads.

In the final installment of this tutorial series, you will deploy a highly available MongoDB ReplicaSet in Kubernetes as a StatefulSet using Helm, a popular open source package manager for Kubernetes.

Prerequisites

kubernetes illustration for: Prerequisites

To complete this tutorial, you will need:

  • An active Docker Hub account to store the Docker image for the application.
  • A local machine with Git installed
  • If you're using macOS, ensure you have Homebrew installed.

Step 1 – Installing the Helm Client on the Development Machine

With Helm, administrators can deploy complex Kubernetes applications with a single command. Applications are packaged as Charts that define, install, and upgrade Kubernetes applications. Charts provide an abstraction over Kubernetes objects such as Pods, Deployments, and Services.

Helm has two components–the server and the client. The server-side of Helm runs in Kubernetes as a Service called Tiller. The client is a command line tool that interacts with Tiller.

Since you are going to deploy a MongoDB ReplicaSet Helm Chart, you need the CLI that talks to Tiller, the server-side component of Helm. StackPointCloud, which you have used to set up Kubernetes on the cloud provider, comes with Tiller preinstalled.

Note: These instructions are for macOS. If you are using other operating systems, please refer to the Helm installation guide.

Assuming you have Homebrew installed and configured on your Mac, run the following command to install Helm:

				
					
brew install kubernetes-helm

				
			
				
					
[secondary_label Output]

==> Downloading https://homebrew.bintray.com/bottles/kubernetes-helm-2.8.2.high_sierra.bottle.tar.gz

...

==> Summary

🍺 /usr/local/Cellar/kubernetes-helm/2.8.2: 50 files, 121.7MB

				
			

Once Helm is installed, verify that you can run it by checking its current version.

				
					
helm version

				
			
				
					
[secondary_label Output]

Client: &version.Version{SemVer:"v2.7.2", GitCommit:"8478fb4fc723885b155c924d1c8c410b7a9444e6", GitTreeState:"clean"}

Server: &version.Version{SemVer:"v2.8.2", GitCommit:"a80231648a1473929271764b920a8e346f6de844", GitTreeState:"clean"}

				
			

This confirms that the client is installed properly and is able to talk to Tiller.

In the next step, we will use Helm to deploy the MongoDB ReplicaSet in Kubernetes.

Step 2 – Deploying the MongoDB ReplicaSet in Kubernetes

A StorageClass in Kubernetes provides a way for administrators to describe the “classes” of storage they offer. For example, when users request a storage volume, the StorageClass will determine what class of storage backend is provisioned from them. The classes may include standard HDD and a faster SSD. Behind the scenes, the StorageClass interacts with the underlying infrastructure such as a cloud provider’s API, to provision storage.

Since you need persistent storage to store MongoDB data, you may want to attach a the cloud provider Block Storage volume to a worker node, and point the MongoDB Pod to use the storage volume for persistence.

In this case, the StorageClass acts as the interface between the Pod and the the cloud provider block storage service. When you request a volume of block storage, the StorageClass talks to the preconfigured driver that knows how to allocate a block storage volume.

StackPointCloud installs the the cloud provider storage driver and registers the StorageClass with Kubernetes during the setup. This saves us from the steps involved in installing and configuring the driver and the StorageClass.

Before we deploy the MongoDB cluster, let’s ensure that the StorageClass for the cloud provider volumes is configured:

				
					
kubectl get storageclass

				
			

The output confirms that StorageClass is configured and ready.

				
					
[secondary_label Output

NAME PROVISIONER AGE

the cloud provider (default) the cloud provider/flex-volume-provisioner 1d

				
			

Next, you will configure and deploy the MongoDB ReplicaSet based on the the cloud provider StorageClass.

Create a new directory for your project and switch to the new directory:

				
					
mkdir ~/mongo-rs

cd ~/mongo-rs

				
			

Clone the Helm Chart repository from GitHub:

				
					
git clone https://github.com/kubernetes/charts.git

				
			

Navigate to the MongoDB ReplicaSet directory (charts/stable/mongodb-replicaset/) and verify that the file values.yaml exists.

				
					
cd charts/stable/mongodb-replicaset/

ls values.yaml

				
			
				
					
[secondary_label Output]

values.yaml

				
			

This file contains the parameters and configuration for the chart. You need to modify this file to configure the MongoDB ReplicaSet to use the the cloud provider StorageClass.

Edit values.yaml:

				
					
nano values.yaml

				
			

Find and uncomment the following section:

				
					
[label values.yaml]

...

# storageClass: "-" 

...

				
			

Replace `"-" with "the cloud provider"`, like this:

				
					
[label values.yaml]

...

storageClass: <^>"the cloud provider"<^>

...

				
			

Save the file and exit your editor.

Now navigate to the ~/mongo-rs folder.

				
					
cd ~/mongo-rs

				
			

You're now ready to deploy the MongoDB ReplicaSet to your Kubernetes cluster, powered by the cloud provider’s block storage. Run the following command to launch the database cluster.

				
					
helm install --name=<^>todo<^> -f charts/stable/mongodb-replicaset/values.yaml stable/mongodb-replicaset

				
			

In the preceding command, --name refers to the name of the Helm chart. The switch -f points to the configuration settings stored in values.yaml.

You will immediately see the output confirming that the chart creation has started.

				
					
[secondary_label Output]

NAME: todo

LAST DEPLOYED: Sat Mar 31 10:37:06 2018

NAMESPACE: default

STATUS: <^>DEPLOYED<^>



RESOURCES:

==> v1/Service

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE

<^>todo-mongodb-replicaset<^> ClusterIP None <none> 27017/TCP 1s



==> v1beta1/StatefulSet

NAME DESIRED CURRENT AGE

<^>todo-mongodb-replicaset<^> 3 1 0s



==> v1/Pod(related)

NAME READY STATUS RESTARTS AGE

<^>todo-mongodb-replicaset-0<^> 0/1 Init:0/2 0 0s



==> v1/ConfigMap

NAME DATA AGE

<^>todo-mongodb-replicaset<^> 1 1s

<^>todo-mongodb-replicaset-tests<^> 1 1s





NOTES:

1. After the statefulset is created completely, one can check which instance is primary by running:



 $ for ((i = 0; i < 3; ++i)); do kubectl exec --namespace default todo-mongodb-replicaset-$i -- sh -c 'mongo --eval="printjson(rs.isMaster())"'; done



2. One can insert a key into the primary instance of the mongodb replica set by running the following:

 MASTER_POD_NAME must be replaced with the name of the master found from the previous step.



 $ kubectl exec --namespace default MASTER_POD_NAME -- mongo --eval="printjson(db.test.insert({key1: 'value1'}))"



3. One can fetch the keys stored in the primary or any of the slave nodes in the following manner.

 POD_NAME must be replaced by the name of the pod being queried.



 $ kubectl exec --namespace default POD_NAME -- mongo --eval="rs.slaveOk(); db.test.find().forEach(printjson)"

				
			

Let’s now run a series of commands to track the status of the cluster.

First, look at the StatefulSet:

				
					
kubectl get statefulset

				
			

This command confirms that the MongoDB ReplicaSet was created as a Kubernetes StatefulSet.

				
					
[secondary_label Output]

NAME DESIRED CURRENT AGE

<^>todo-mongodb-replicaset<^> 3 2 2m

				
			

Now explore the Pods:

				
					
kubectl get pods

				
			

The number of Pods and their naming convention indicates that the MongoDB ReplicaSet is successfully configured:

				
					
[secondary_label Output]

NAME READY STATUS RESTARTS AGE

<^>todo-mongodb-replicaset-0<^> 1/1 Running 0 3m

<^>todo-mongodb-replicaset-1<^> 1/1 Running 0 1m

<^>todo-mongodb-replicaset-2<^> 1/1 Running 0 54s

				
			

Notice that each Pod has a suffix that ends with a sequential number, which is a distinctive feature of a StatefulSet.

Let’s now check if the MongoDB instances are communicating with each other. We will do this by running a command in the MongoDB shell within one of the Pods.

Use kubectl to launch the mongo console on one of the hosts:

				
					
kubectl exec -it <^>todo-mongodb-replicaset-0<^> mongo

				
			

After connecting, you'll find yourself in the MongoDB shell:

				
					
[secondary_label Output]

MongoDB shell version v3.6.3

connecting to: mongodb://127.0.0.1:27017

MongoDB server version: 3.6.3

Welcome to the MongoDB shell.

For interactive help, type "help".

...



2018-03-31T05:08:20.239+0000 I CONTROL [initandlisten]

				
			

Check the ReplicaSet’s configuration with the following command:

				
					
rs.conf()

				
			

The output confirms that there are three instances of MongoDB running as a ReplicaSet.

				
					
[secondary_label Output]

{

	"_id" : "rs0",

	"version" : 3,

	"protocolVersion" : NumberLong(1),

	"members" : [

		{

			"_id" : 0,

			"host" : "<^>todo-mongodb-replicaset-0.todo-mongodb-replicaset.default.svc.cluster.local:27017"<^>,

			"arbiterOnly" : false,

			"buildIndexes" : true,

			"hidden" : false,

			"priority" : 1,

			"tags" : {



			},

			"slaveDelay" : NumberLong(0),

			"votes" : 1

		},

		{

			"_id" : 1,

			"host" : "<^>todo-mongodb-replicaset-1.todo-mongodb-replicaset.default.svc.cluster.local:27017<^>",

			"arbiterOnly" : false,

			"buildIndexes" : true,

			"hidden" : false,

			"priority" : 1,

			"tags" : {



			},

			"slaveDelay" : NumberLong(0),

			"votes" : 1

		},

		{

			"_id" : 2,

			"host" : "<^>todo-mongodb-replicaset-2.todo-mongodb-replicaset.default.svc.cluster.local:27017<^>",

			"arbiterOnly" : false,

			"buildIndexes" : true,

			"hidden" : false,

			"priority" : 1,

			"tags" : {



			},

			"slaveDelay" : NumberLong(0),

			"votes" : 1

		}

	],

	"settings" : {

		"chainingAllowed" : true,

		"heartbeatIntervalMillis" : 2000,

		"heartbeatTimeoutSecs" : 10,

		"electionTimeoutMillis" : 10000,

		"catchUpTimeoutMillis" : -1,

		"catchUpTakeoverDelayMillis" : 30000,

		"getLastErrorModes" : {



		},

		"getLastErrorDefaults" : {

			"w" : 1,

			"wtimeout" : 0

		},

		"replicaSetId" : ObjectId("5abdb4f61d952afc4b0b8218")

	}

}

				
			

Exit the MongoDB console:

				
					
exit

				
			

This will disconnect you from your remote host as well.

Let’s switch gears and check the the cloud provider control panel for the block storage volumes associated with the cluster. Log in to your cloud account and select the Volumes tab:

You can see that three volumes of 10GB each are attached to Kubernetes worker nodes. Each Pod of the MongoDB StatefulSet is storing the data in one of the block storage volumes. The size of 10GB is defined in values.yaml under the persistentVolume section.

				
					
[label values.yaml]

persistentVolume:

 enabled: true

 ## mongodb-replicaset data Persistent Volume Storage Class

 ## If defined, storageClassName: <storageClass>

 ## If set to "-", storageClassName: "", which disables dynamic provisioning

 ## If undefined (the default) or set to null, no storageClassName spec is

 ## set, choosing the default provisioner. (gp2 on AWS, standard on

 ## GKE, AWS & OpenStack)

 ##

 storageClass: the cloud provider

 accessModes:

 - ReadWriteOnce

 <^>size: 10Gi<^>

 annotations: {}

				
			

You have successfully configured a highly available MongoDB ReplicaSet running in Kubernetes.

Now let's deploy the web application that talks to the MongoDB cluster.

Step 3 – Deploying and Scaling the Web Application in Kubernetes

Let's extend the ToDo Node.js application we used in previous parts of this tutorial series to take advantage of the MongoDB cluster.

Note: You can also build the container image from the source code or use the YAML files in the Kubernetes files directly. Refer to the tutorial Deploying and Scaling Microservices in Kubernetes for steps on building the image and deploying the application to Kubernetes.

Start by creating a new working directory:

				
					
mkdir ~/web-app

cd ~/web-app

				
			

Then clone the ToDo application's repository that contains the code and Kubernetes artifacts.

				
					
git clone https://github.com/janakiramm/todo.git

				
			

Switch to the todo-app/kubernetes directory which contains the Kubernetes configuration files.

				
					
cd todo-app/kubernetes

				
			

Open the file web-rs-ss.yaml in your editor.

				
					
nano web-rs-ss.yaml

				
			

Notice the env section on the YAML file.

				
					
[label web-rs-ss.yaml]

 containers:

 - name: web 

 image: janakiramm/todo

 <^>env:<^>

 <^>- name: "DBHOST"<^>

 <^>value: "mongodb://todo-mongodb-replicaset-0.todo-mongodb-replicaset,todo-mongodb-replicaset-1.todo-mongodb-replicaset,todo-mongodb-replicaset-2.todo-mongodb-replicaset:27017"<^>

 ports:

 - containerPort: 3000

				
			

This passes the database connection string to the application at runtime as an environment variable. Instead of pointing the application to a simple MongoDB Pod, this version of the app uses the StatefulSet you created. Each entry in the value section refers to one of the Pods of the MongoDB StatefulSet.

Use kubectl to deploy the web ReplicaSet along with the web Service

				
					
kubectl create -f web-rs-ss.yaml -f web-service.yaml

				
			

You'll see that both are created:

				
					
[secondary_label Output]

replicaset "web" created

service "web" created

				
			

List the pods again:

				
					
kubectl get pods

				
			

You now see all the Pods belonging to MongoDB and the web app.

				
					
[secondary_label Output]

NAME READY STATUS RESTARTS AGE

todo-mongodb-replicaset-0 1/1 Running 0 26m

todo-mongodb-replicaset-1 1/1 Running 0 24m

todo-mongodb-replicaset-2 1/1 Running 0 23m

<^>web-t5zzk 1/1 Running 0 17s<^>

<^>web-x6dh8 1/1 Running 0 17s<^>



Let’s check out the Kubernetes services



​```command

kubectl get svc

				
			
				
					
[secondary_label Output]

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE

kubernetes ClusterIP 10.3.0.1 <none> 443/TCP 1d

todo-mongodb-replicaset ClusterIP None <none> 27017/TCP 27m

<^>web NodePort 10.3.0.167 <none> 3000:31201/TCP 14s<^>

				
			

The web Pods talk to the MongoDB cluster through the todo-mongodb-replicaset Service. The web application is available through the web Service on NodePort 31201.

Accessing this port on any of the worker nodes shows the web application.

You can scale the web application by increasing the number of Pods in the ReplicaSet.

				
					
kubectl scale rs/web --replicas=10

				
			
				
					
[secondary_label Output]

replicaset "web" scaled

				
			

You can then scale the application back to two Pods.

				
					
kubectl scale rs/web --replicas=2

				
			
				
					
[secondary_label Output]

replicaset "web" scaled

				
			

Now let's run some tests for availability.

Step 4 – Testing the MongoDB ReplicaSet for High Availability

One of the advantages of running a StatefulSet is the high availability of workloads. Let’s test this by deleting one of the Pods in the MongoDB StatefulSet.

				
					
kubectl delete pod todo-mongodb-replicaset-2

				
			
				
					
[secondary_label Output]

pod "todo-mongodb-replicaset-2" deleted

				
			

Check the number of Pods:

				
					
kubectl get pods

				
			

You'll see that todo-mongodb-replicaset-2 is terminating:

				
					
[secondary_label Output]

NAME READY STATUS RESTARTS AGE

todo-mongodb-replicaset-0 1/1 Running 0 33m

todo-mongodb-replicaset-1 1/1 Running 0 32m

<^>todo-mongodb-replicaset-2 0/1 Terminating 0 31m<^>

web-t5zzk 1/1 Running 0 8m

web-x6dh8 1/1 Running 0 8m

				
			

Within a few minutes, you will see that Kubernetes initializes another Pod to replace the deleted one.

				
					
kubectl get pods

				
			

You'll see todo-mongodb-replicaset-2 is initializing:

				
					
NAME READY STATUS RESTARTS AGE

todo-mongodb-replicaset-0 1/1 Running 0 34m

todo-mongodb-replicaset-1 1/1 Running 0 33m

<^>todo-mongodb-replicaset-2 0/1 Init:0/2 0 29s<^>

web-t5zzk 1/1 Running 0 8m

web-x6dh8 1/1 Running 0 8m

				
			

Now that you know everything works, you can clean things up.

Delete all the objects created during this tutorial with the following commands:

				
					
helm delete --purge todo

				
			
				
					
kubectl delete -f web-rs-ss.yaml -f web-service.yaml

				
			
				
					
[secondary_label Output]

replicaset "web" deleted

service "web" deleted

				
			

To delete the Kubernetes cluster itself, visit StackPointCloud and do so through their control panel.

Conclusion

In this tutorial, you deployed a durable, persistent, highly available, MonogDB ReplicaSet as a Kubernetes StatefulSet. You also learned how to access the StatefulSet from other applications deployed in the same Kubernetes cluster.