Welcome to the first part of our Kubernetes tutorial. We would like it to cover some day-to-day challenges that every DevOps engineer working with Kubernetes might encounter. That’s why we will assume that you, dear reader, are already familiar with Kubernetes and its basic concepts.

In this particular part of tutorial we will focus on managing high availability Kubernetes cluster on AWS infrastructure. We will use kops - Production Grade K8s Installation, Upgrades, and Management tool.


You will need to setup a few things before we can proceed:

Setup your environment

First of all, let’s start by creating dedicated IAM user for kops. Here are the AWS CLI commands to achieve that:

… this can be done through AWS console as well, just remember to create IAM user with these permissions assigned:

Now we can use newly created SecretAccessKey and AccessKeyID:

Additionally kops requires dedicated S3 bucket to store the state of the cluster. This is how it can be created:

Two notes regarding state store bucket:

  • --create-bucket-configuration LocationConstraint=$REGION option is required only for regions other than us-east-1.
  • S3 bucket name must be unique, it’s a good practice to include some prefix or domain name in it

Configure DNS

There’s only one thing left that has to be done before we can spin our Kubernetes cluster up - to configure proper DNS entries. We will assume that K8s cluster will be available under the subdomain of our main domain, e.g. I will show how to achieve that in the case when both domain and subdomain are hosted by AWS.

First step would be to create new hosted zone that will handle the subdomain:

When command completes, file called subdomain-hosted-zone.json should contain details of newly created hosted zone. We need to retrieve details of its NS servers:

Once we have addresses of nameservers extracted we need to add them as a new record set of the main domain’s hosted zone. Let’s prepare a subdomain.json file that we will use to create new record sets:

Note: use nameservers extracted in previous step!

Now subdomain’s NS records can be applied to main domain’s hosted zone:

If everything went fine - after view minutes - DNS entries should be reachable:

Create the cluster

Finally! We have everything ready to create and start the cluster. Let’s setup few environment variables to avoid some additional dull typing:

… and now brace yourself, the deployment is coming!

Brace yourself, the deployment is coming...

Naah… Relax, it should actually be quite seamless and handled by kops:

As a result we should obtain a brand new K8s cluster spread across 3 availability zones, with multi-master setup. Additionally all nodes and masters will be launched in a private subnet in the VPC (thanks to --topology private flag), but we will be able to reach them with SSH through the dedicated bastion host (--bastion flag).

It is worth to mention that all instances created by kops will be spawned within Auto Scaling Groups, so each instance will be automatically monitored and rebuilt by AWS in case of any failure.

Output of the last command should look similar to this:

kops should print quite long and verbose list of resources that will be created to setup the cluster. Yes, you read it right - will be created. As for now kops has not started to actually create cluster on AWS yet. It has just shown what will be done and gave you chance to review it.

You can update cluster details for example:

… or change details of node instances (instance type, number of nodes, etc.)

Once you are sure cluster config is ready you can actually build the cluster on AWS:

When the command succeeds, you should see a message like this one:

After view minutes you can validate the cluster:

NOTE: you might experience interim error when calling kops validate cluster command when cluster is still starting:

In such case just wait few more minutes for cluster to boot up.

Cloud Solutions Architects/DevOps Engineers Available

Are looking for cloud solutions architects? Or maybe you need to support of experienced DevOps engineers? Then you are in the right place!

Schedule a call with our expert

Interact with the cluster

Once the cluster is up and running it’s time to use it. In the Prerequisites section I’ve asked you to install kubectl tool. We can now use it to interact with the new cluster as kops has generated required configuration and wrote it down into ~/.kube/config. For example, let’s check status of cluster nodes:

… or list pods running inside:

And when it’s time to say goodbye to the cluster and tear it down you can use kops to delete it along with all its resources:

Going multi-tier

Ok, we’ve built fresh and shiny K8s cluster and played with it using kops and kubectl. It’s been informative and fun. But in most real-life scenarios one cluster is not enough…


What if you need multiple tiers like for example dev, qa, stage and prod? With our current setup is quite easy. Let’s just clean up our prototype cluster:

… and adjust our previous exports a little bit:

As you may notice the only thing that changed is the name of the cluster (we have added dev. subdomain). And that’s it! We can use the same commands as before to create new cluster for dev tier:

Need new qa environment? Here you go:

Note that the same state bucket has been used for both clusters - no worries, kops will handle that, state of dev cluster will not get overridden.

Run kops create cluster ... and kops update cluster ... commands once again and you are ready to go (although you might want ot tweak some cluster settings accordingly to tier purpose, e.g. use different EC2 instance types or update labels with tier name).

Additionally you can still manage both clusters with help of kubectl:


We have successfully created high availability Kubernetes cluster, on multiple environments. I want to give a shout-out to kops for making this almost effortless and as smooth as possible. Now, as we know how to spin up and manage clusters easily, we can proceed to other kubernetes-related challenges that will be covered in upcoming blog posts.