OpenTelemetry Collector 2,394,1262,394,126
Visualisation of OpenTelemetry collector metrics from Prometheus.
Part of opentelemetry-collector-monitoring repo.
This dashboard can also be used partially for Alloy monitoring (mainly for OTEL components).
Metric name/label name conventions
OpenTelemetry collector metrics and label names can vary significantly. These variations depend on several factors:
- OpenTelemetry Collector version
- Specific receivers/exporters in use
- Receiver/exporter configuration settings
- Metric storage backend configuration
For example uptime metric can be saved as:
otelcol_process_uptime{service_instance_id="123456", ...}
otelcol_process_uptime_total{service_instance_id="123456", ...}
otelcol_process_uptime_seconds_total{service_instance_id="123456", ...}
otelcol_process_uptime_seconds_total{service.instance.id="123456", ...}
otelcol_process_uptime_seconds_total_total{service.instance.id="123456", ...}
This dashboard attempts to detect all these variations through hidden dashboard variables to ensure compatibility across different configurations.
Collector configuration, where the collector collects their own metrics and posts them to Prometheus:
extensions:
sigv4auth/aws:
receivers:
prometheus:
config:
scrape_configs:
- job_name: otel-collector-metrics
scrape_interval: 10s
static_configs:
- targets: ['localhost:8888']
exporters:
# AWS Managed Prometheus example
prometheusremotewrite/aws:
endpoint: https://aps-workspaces.<region>.amazonaws.com/workspaces/<workspace-id>/api/v1/remote_write
auth:
authenticator: sigv4auth/aws
# Grafana Cloud example
prometheusremotewrite/grafanacloud:
# check your Grafana Cloud admin console for correct hostname
endpoint: https://prometheus-prod-11-prod-us-central-0.grafana.net/api/prom
headers:
# hardcoded base64 value in auth header or use basicauth extension
"Authorization": "Basic <base64(username:password)>"
# Prometheus server example
prometheusremotewrite/my-prometheus:
# my-prometheus server must have '--enable-feature=remote-write-receiver'
endpoint: https://my-prometheus/api/v1/write
service:
pipelines:
metrics:
receivers: [prometheus]
processors: []
# use configured prometheusremotewrite exporter
exporters: [prometheusremotewrite]
telemetry:
metrics:
address: 0.0.0.0:8888
level: detailed
See: collector monitoring doc and collector troubleshooting - metrics.
Used Metrics 6969
otelcol_receiver_accepted_spans
otelcol_receiver_refused_spans
otelcol_receiver_accepted_metric_points
otelcol_receiver_refused_metric_points
otelcol_receiver_accepted_log_records
otelcol_receiver_refused_log_records
otelcol_processor_incoming_items
signal
traces
otelcol_processor_outgoing_items
metrics
logs
otelcol_processor_accepted_spans
otelcol_processor_refused_spans
otelcol_processor_dropped_spans
otelcol_processor_accepted_metric_points
otelcol_processor_refused_metric_points
otelcol_processor_dropped_metric_points
otelcol_processor_accepted_log_records
otelcol_processor_refused_log_records
otelcol_processor_dropped_log_records
otelcol_processor_batch_batch_send_size_bucket
otelcol_processor_batch_batch_send_size_count
otelcol_processor_batch_batch_send_size_sum
otelcol_processor_batch_batch_size_trigger_send
otelcol_processor_batch_timeout_trigger_send
otelcol_exporter_sent_spans
otelcol_exporter_enqueue_failed_spans
otelcol_exporter_send_failed_spans
otelcol_exporter_sent_metric_points
otelcol_exporter_enqueue_failed_metric_points
otelcol_exporter_send_failed_metric_points
otelcol_exporter_sent_log_records
otelcol_exporter_enqueue_failed_log_records
otelcol_exporter_send_failed_log_records
otelcol_exporter_queue_size
min
otelcol_exporter_queue_capacity
otelcol_process_memory_rss
otelcol_process_runtime_total_sys_memory_bytes
otelcol_process_runtime_heap_alloc_bytes
otelcol_process_cpu_seconds
otelcol_process_uptime
instance
id
job
service
version
target_info
receivers
rcv
transport
receiver
title
icon
arrow
to
right
dummy
processor
Processor
s
random
exporters
exporter
from
source
target
edg