Grafana's Alloy embeds several Prometheus Exporters. In a previous article I wrote how metrics can be scraped from Alloy's embedded Node Exporter. This time it's about another widely used Prometheus Exporter: The Process Exporter.
The official documentation of the Alloy embedded Process Exporter (component "prometheus.exporter.process") is straight-forward and easy to understand. At least for the enabling part.
The following snippet enables the embedded Process Exporter:
prometheus.exporter.unix "example" {
include_exporter_metrics = true
}
After a restart of Alloy, Alloy now starts to collect (scrape) process metrics from its embedded Process Exporter.
However when checking out the metrics (using Alloy's local component API), only a few metrics show up:
root@linux ~ # curl -s localhost:12345/api/v0/component/prometheus.exporter.process.localhost/metrics
# HELP namedprocess_scrape_errors general scrape errors: no proc metrics collected during a cycle
# TYPE namedprocess_scrape_errors counter
namedprocess_scrape_errors 0
# HELP namedprocess_scrape_partial_errors incremented each time a tracked proc's metrics collection fails partially, e.g. unreadable I/O stats
# TYPE namedprocess_scrape_partial_errors counter
namedprocess_scrape_partial_errors 2003
# HELP namedprocess_scrape_procread_errors incremented each time a proc's metrics collection fails
# TYPE namedprocess_scrape_procread_errors counter
namedprocess_scrape_procread_errors 0
# HELP process_exporter_build_info A metric with a constant '1' value labeled by version, revision, branch, goversion from which process_exporter was built, and the goos and goarch for the build.
# TYPE process_exporter_build_info gauge
process_exporter_build_info{branch="HEAD",goarch="amd64",goos="linux",goversion="go1.22.7",revision="dc8a365f8",tags="netgo,builtinassets,promtail_journal_enabled",version="v1.5.1"} 1
That's all there is. I expected hundreds of lines including details about all processes running on this server.
Looking at the output it seems that the metrics show no processes at all, only internal metrics from Alloy's embedded process exporter.
An interesting information can be found on the official GitHub repository of Process Exporter. The example configuration (packaging/conf/all.yaml) shows the following snippet:
process_names:
- name: "{{.Comm}}"
cmdline:
- '.+'
To my understanding, this "filter" matches all running processes - hence Process Exporter is told to retrieve metrics from all matching processes (all).
According to the Alloy documentation, the Process Exporter filters can be added to the configuration using the "matcher" context. Let's use the same example from above inside the Alloy configuration:
root@linux ~ # vi /etc/alloy/config.alloy
[...]
prometheus.exporter.process "example" {
matcher {
name = "{{.Comm}}"
cmdline = [".+"]
}
}
[...]
Another Alloy restart and let's see if this works.
root@linux ~ # systemctl restart alloy
And after quickly checking the internal component API again, the output now shows a ton of metrics (over 2K):
root@linux ~ # curl -s localhost:12345/api/v0/component/prometheus.exporter.process.example/metrics | wc -l
2015
What a difference!
All the processes (able to be read by Alloy) are now included in the metrics. Success!
By the way, a quick comparison of Alloy's embedded Process Exporter vs. a standalone Prometheus Process Exporter reveals a slight difference in the number of metrics:
root@linux ~ # curl -s localhost:9256/metrics | wc -l
2134
root@linux ~ # curl -s localhost:12345/api/v0/component/prometheus.exporter.process.example/metrics | wc -l
2038
But from what I can tell, all the important metrics (for me anyway) show up on both.
AWS Android Ansible Apache Apple Atlassian BSD Backup Bash Bluecoat CMS Chef Cloud Coding Consul Containers CouchDB DB DNS Database Databases Docker ELK Elasticsearch Filebeat FreeBSD Galera Git GlusterFS Grafana Graphics HAProxy HTML Hacks Hardware Icinga Influx Internet Java KVM Kibana Kodi Kubernetes LVM LXC Linux Logstash Mac Macintosh Mail MariaDB Minio MongoDB Monitoring Multimedia MySQL NFS Nagios Network Nginx OSSEC OTRS Observability Office OpenSearch PGSQL PHP Perl Personal PostgreSQL Postgres PowerDNS Proxmox Proxy Python Rancher Rant Redis Roundcube SSL Samba Seafile Security Shell SmartOS Solaris Surveillance Systemd TLS Tomcat Ubuntu Unix VMWare VMware Varnish Virtualization Windows Wireless Wordpress Wyse ZFS Zoneminder