Standardize observability

The Keptn Lifecycle Toolkit (KLT) makes any Kubernetes deployment observable. In other words, it creates a distributed, end-to-end trace of everything Kubernetes does in the context of a Deployment. It provides this information for all applications running in your cluster, and includes information about everything Kubernetes does in the context of a deployment. To do this, Keptn introduces the concept of an application, which is an abstraction that connects multiple Workloads that logically belong together, even if they use different deployment strategies.

This means that:

• You can readily see why a deployment takes so long or why it fails, even when using multiple deployment strategies.
• KLT can capture DORA metrics and expose them as OpenTelemetry metrics

The observability data is an amalgamation of the following:

• DORA metrics are collected out of the box when the Lifecycle Toolkit is enabled
• OpenTelemetry runs traces that show everything that happens in the Kubernetes cluster
• Custom Keptn metrics that you can use to monitor information from all the data providers configured in your cluster

All this information can be displayed with dashboard tools such as Grafana.

Using this exercise

This exercise shows how to standardize access to the observability data for your cluster. It is based on the simplenode-dev example.

This is the second of three exercises in the Introducing the Keptn Lifecycle Toolkit series:

• In the Getting started with Keptn metrics exercise, you learn how to define and use Keptn metrics. You may want to complete that exercise before doing this exercise although that is not required.
• In Manage release lifecycle, you learn how to implement pre- and post-deployment tasks and evaluations to orchestrate the flow of all the workloads that are part of your application.

This exercise shows how to standardize access to the observability data for your cluster.

If you are installing the Keptn Lifecycle Toolkit on an existing cluster or on a local cluster you are creating for this exercise, you need to do the following:

1. Follow the instructions in Install and update to install and enable KLT on your cluster.

2. Follow the instructions in Integrate KLT with your applications to integrate KLT with your Kubernetes cluster. This requires the following:

   • Follow the instructions in Annotate workload to integrate the Lifecycle Toolkit into your Kubernetes cluster by applying basic annotations to your Deployment resource.
   • Follow the instructions in Define a Keptn application to create a Keptn application that aggragates all the workloads for your deployment into a single KeptnApp resource. For this exercise, we recommend that you use Keptn automatic app discovery to automatically generate a Keptn Application.

DORA metrics

DORA metrics are an industry-standard set of measurements; see the following for a description:

• What are DORA Metrics and Why Do They Matter?
• Are you an Elite DevOps Performer? Find out with the Four Keys Project

DORA metrics provide information such as:

• How many deployments happened in the last six hours?
• Time between deployments
• Deployment time between versions
• Average time between versions.

The Keptn Lifecycle Toolkit starts collecting these metrics as soon as you annotate the Deployment resource. Metrics are collected only for the Deployment resources that are annotated.

To view DORA metrics, run the following command:

kubectl port-forward -n keptn-lifecycle-toolkit-system \
   svc/lifecycle-operator-metrics-service 2222

Then view the metrics at:

http://localhost:2222/metrics

DORA metrics are also displayed on Grafana or whatever dashboard application you choose. For example:

Using OpenTelemetry

The Keptn Lifecycle Toolkit extends the Kubernetes primitives to create OpenTelemetry data that connects all your deployment and observability tools without worrying about where it is stored and where it is managed. OpenTelemetry traces collect data as Kubernetes is deploying the changes, which allows you to trace everything done in the context of that deployment.

• You must have an OpenTelemetry collector installed on your cluster. See OpenTelemetry Collector for more information.
• Follow the instructions in OpenTelemetry observability to configure where your OpenTelemetry data is sent. This requires you to define a KeptnConfig resource that defines the URL and port of the OpenTelemetry collector. For our example, this is in the keptnconfig.yaml file.

Keptn metrics

You can supplement the DORA Metrics and OpenTelemetry information with information you explicitly define using Keptn metrics. The Getting started with Keptn metrics exercise discusses how to define Keptn metrics.

View the results

To start feeding observability data for your deployments onto a dashboard of your choice:

1. Modify either your Deployment or KeptnApp resource yaml file to increment the version number
2. Commit that change to your repository.

Note that, from the KeptnApp YAML file, you can either increment the version number of the application (which causes all workloads to be rerun and produce observability data) or you can increment the version number of a single workload, (which causes just that workload to be rerun and produce observability data).

The videos that go with this exercise show how the DORA, OpenTelemetry, and Keptn metrics information appears on a Grafana dashboard with Jaeger.

If you also have the Jaeger extension for Grafana installed on your cluster, you can view the full end-to-end trace for everything that happens in your deployment. For more information, see Monitoring Jaeger.