Workload Resource Recommendations Based on Usage
303,406

Created 10/26/2021
Updated 12/13/2021
Revision 2
Grafana Version >=7.5.4
Datasources
Prometheus

This dashboard shows historical usage for Kubernetes Workloads and based on that, gives a recommendation for Requests and Limits settings. It has variables for the percentile and overhead amount used to calculate the recommendations, so you can account for workloads with variable load and also adjust limits based on your own appetite to tradeoff reliability against efficiency.

The dashboard uses:

  • kubelet cadvisor metrics which would be normally collected via the metrics-server, for container resource usage data
  • kube-state-metrics, for details of current requests & limits set It also relies on a recording rule node_namespace_pod_container:container_cpu_usage_seconds_total:sum_rate from kube-prometheus, which is defined as:
sum by(cluster, namespace, pod, container) (rate(container_cpu_usage_seconds_total{image!="",job="kubelet",metrics_path="/metrics/cadvisor"}[5m])) * on(cluster, namespace, pod) group_left(node) topk by(cluster, namespace, pod) (1, max by(cluster, namespace, pod, node) (kube_pod_info{node!=""}))
Export Dashboard
Download
Copy to Clipboard

Used Metrics 12

  • node_namespace_pod_container:container_cpu_usage_seconds_total:sum_rate

  • kube_pod_container_resource_requests

  • kube_pod_container_resource_limits

  • cpu_overhead_percent

  • quantile_over_time

  • cpu_percentile

  • __range:

  • owner

  • kube_pod_owner

  • container_memory_working_set_bytes

  • memory_overhead_percent

  • memory_percentile