For the past couple of weeks, our Prometheus cluster has been quietly polling this blog’s web server for metrics. Now that we’re collecting the data, our next job is make the data provide value. Our data provides value when it assists us in understanding our application’s past and current SLO adherence, and when it improves our actual SLO adherence. In this blog post, we’ll focus on the first of the two aforementioned value propositions. Specifically, we will create visualizations of metrics pertaining to our SLO using Grafana.

For all of you just itching to deploy another application to your Kubernetes cluster, this post is for you.

via GIPHY

In it, I’ll be discussing deploying Prometheus, the foundation of our planned monitoring and alerting, to our Kubernetes cluster. This post will only discuss getting the Prometheus cluster running on our Kubernetes cluster. I’ll leave setting up monitoring, alerting, and useful visualizations for a later blog post in the series.

The Problem

So far, my ideas for experimenting with my personal Kubernetes cluster have been spread out across discrete blog posts. As a result, its difficult for me, and I imagine y’all as the readers, to track a prioritized list of projects.

via GIPHY

I also think that, in the future, it will be useful for us to be able to easily see which projects have been completed and which have not. Particularly as I plan to accompany all projects with blog posts, having a discoverable enumeration of my personal cluster’s features will give readers an easy way to know on which subjects they can expect guidance.

In the blog post overviewing our SLO implementation, I listed configuring our blog to expose the metrics for Prometheus to scrape as the first step. To fulfill that promise, this post examines the necessary steps for taking our static website and serving it via a production web server which exposes the latency and success metrics our SLO needs.

A brief examination of Prometheus metrics

Application monitoring has two fundamental components: instrumentation and exposition. Instrumentation refers to measuring and recording different quantities and states. Exposition refers to making metrics available to some consumer, which for us is Prometheus. We’ll discuss both in the context of our static website.

My last two blog posts enumerated this blog’s SLO and error budget. Our next logical step is adding the monitoring and alerting infrastructure which will transform our SLO usage from theoretical to practical. Like creating a Kubernetes of One’s Own, this project contains multiple steps which we’ll explore over multiple blog posts. While this series focuses on achieving this goal for this blog’s specific SLO, the techniques are applicable to many scenarios.

Our goal for this project is to create the monitoring infrastructure which will allow us to monitor metrics pertaining to our SLO, and alert me when this blog is violating its SLO (i.e. spent its error budget).

In my last blog post, I publicized an SLO for this blog. I also mentioned that, in the future, I’d couple the SLO with an error budget and error budget policy. Well, the future is today, because this post will define error budgets and error budget policies and their benefits, before proposing a specific error budget and error budget policy to accompany our previously defined SLO.

What are Error Budget and Error Budget Policies?

via GIPHY

Background

I recently started reading The Site Reliability Workbook, which is the companion book to the excellent Site Reliability Engineering: How Google Runs Production Systems.

via GIPHY

These books devote considerable attention to Service Level Ojectives (SLOs), which are a way of defining a given level of service that users can expect. More technically, a SLO is a collection of Service Level Indicators (SLIs), metrics that measure whether our service is providing value, and their acceptable ranges. For example, our SLO for a web service could be that 95% of requests are successful and 99% of requests return in less than 500ms.

In my last three blog posts, we focused on creating a Kubernetes cluster you can use for your own personal computing needs. But what good is a Kubernetes cluster if we’re not using it to run applications? Spoiler alert, not much.

Let’s make your Kubernetes cluster worth the cash you’re paying and get some applications running on it. In this post, we’ll walk through deploying your first application to Kubernetes: a static blog.

In my last blog post, we outlined the different methods of creating and maintaining a Kubernetes cluster, before deciding on Kops. In this blog post, we’ll actually create the cluster using Kops. I’ll provide source code and instructions, so by the end of this post, you can have your own Kubernetes cluster!

This tutorial is strongly based on Kops AWS tutorial, although its even simplifier because I’ve written some generic terraform configurations which simplify initial AWS configuration.

In my last blog post, I hope I convinced you why you should be creating your own Kubernetes cluster for personal usage. Now, we can tackle the fun part of creating the cluster.

What is a Kubernetes cluster?

via GIPHY

We can start by answering the most important question: what is a Kubernetes cluster? A Kubernetes cluster is a collection of physical resources on which we run the Kubernetes container management software. Once we have a Kubernetes cluster, we can use it to host applications, run batch workloads, etc.