Building a Continuous Deployment Pipeline Using LKE (Part 10): Installing metrics-server

Slide deck: Cloud Native Continuous Deployment with GitLab, Helm, and Linode Kubernetes Engine: Installing metrics-server (Slide #152)

Now that there is an application running on our Kubernetes cluster, the next step is to collect metrics on the resources being used. This part covers installing and using metrics-server as a basic data collection tool.

• Main guide: Building a Continuous Deployment Pipeline Using LKE
• Previous section: Part 9: Installing Traefik
• Next section: Part 11: Prometheus and Grafana

Here’s a copy of the text contained within this section of the presentation. A link to the source file can be found within each slide of the presentation. Some formatting may have been changed.

• We’ve installed a few things on our cluster so far
• How much resources (CPU, RAM) are we using?
• We need metrics!
• If metrics-server is installed, we can get Nodes metrics like this: kubectl top nodes
• At the moment, this should show us error: Metrics API not available
• How do we fix this?

• We could use a SAAS like Datadog, New Relic…
• We could use a self-hosted solution like Prometheus
• Or we could use metrics-server
• What’s special about metrics-server?

• Cons:
  • no data retention (no history data, just instant numbers)
  • only CPU and RAM of nodes and pods (no disk or network usage or I/O…)
• Pros:
  • very lightweight
  • doesn’t require storage
  • used by Kubernetes autoscaling

• We may install something fancier later (think: Prometheus with Grafana)
• But metrics-server will work in minutes
• It will barely use resources on our cluster
• It’s required for autoscaling anyway

• It runs a single Pod
• That Pod will fetch metrics from all our Nodes
• It will expose them through the Kubernetes API aggregation layer (we won’t say much more about that aggregation layer; that’s fairly advanced stuff!)

• In a lot of places, this is done with a little bit of custom YAML (derived from the official installation instructions)

• We’re going to use Helm one more time:

  helm upgrade --install metrics-server bitnami/metrics-server \
    --create-namespace --namespace metrics-server \
    --set apiService.create=true \
    --set extraArgs.kubelet-insecure-tls=true \
    --set extraArgs.kubelet-preferred-address-types=InternalIP
  

• What are these options for?

• apiService.create=true

  register metrics-server with the Kubernetes aggregation layer

  (create an entry that will show up in kubectl get apiservices)

• extraArgs.kubelet-insecure-tls=true

  when connecting to nodes to collect their metrics, don’t check kubelet TLS certs

  (because most kubelet certs include the node name, but not its IP address)

• extraArgs.kubelet-preferred-address-types=InternalIP

  when connecting to nodes, use their internal IP address instead of node name

  (because the latter requires an internal DNS, which is rarely configured)

• After a minute or two, metrics-server should be up
• We should now be able to check Nodes resource usage: kubectl top nodes
• And Pods resource usage, too: kubectl top pods --all-namespaces

• The RAM usage that we see should correspond more or less to the Resident Set Size
• Our pods also need some extra space for buffers, caches…
• Do not aim for 100% memory usage!
• Some more realistic targets:
  • 50% (for workloads with disk I/O and leveraging caching)
  • 90% (on very big nodes with mostly CPU-bound workloads)
  • 75% (anywhere in between!)