Prometheus Edge Exporter for Gateway Metrics

This page solves one concrete problem: exposing a handful of gauges — die temperature, process RSS, sync-queue depth — over HTTP for a same-LAN scraper, using the prometheus_client Python library, without turning the exporter into a second load-bearing process on a 256 MB–2 GB Linux gateway. Within the Edge Operations & Observability guide, and as a companion to monitoring and observability, this is the wire-format specific half of that guide’s metric surface: where the parent page hand-rolls a minimal registry, this page builds the same handful of values on the maintained client library for sites where a standard scrape target is worth the extra dependency.

The deployment context matters here more than it does on a server: a field gateway serving /metrics to a scraper is not competing for rack space or a shared network — it is competing with its own spatial pipeline for the same core and the same RAM, and the scraper itself is usually a Raspberry Pi or industrial PC sitting on the same LAN segment as a small fleet of gateways, not a cloud-hosted Prometheus server pulling over a metered WAN link. Every choice below is shaped by that: keep cardinality low, cache expensive reads, and never let the scrape path touch sysfs directly.

Exporter design selection rationale #

prometheus_client gives three ways to expose metrics, and the choice between them is a real trade-off on constrained hardware rather than a style preference.

start_http_server() spins up a background thread and a full HTTP server for you — the least code, but it means a second thread is always resident, always listening, whether or not anything is scraping. On a gateway that duty-cycles most of its work, an always-on thread is a small but real tax on the power budget.

Building the endpoint yourself with generate_latest() and a lightweight WSGI-free HTTP handler gives up the convenience but keeps the exporter’s cost visible: the handler only runs when a request actually arrives, and the values it serves come from gauges that were already updated by the sampling loop, not recomputed on the request path. That is the shape used below, because it matches the guide’s standing rule that a scrape must never trigger a live sysfs read.

Either way, the metric surface itself has to be small and fixed. A Gauge per label combination multiplies memory and scrape-response size; on a node serving four or five values, labels should be reserved for genuinely low-cardinality dimensions (a zone name, a queue name) and never for anything that varies per request, per device, or per timestamp.

It is worth being explicit about what this exporter deliberately does not do. It does not aggregate across a fleet — that is the collector’s job, running on the same-LAN scraper or further upstream, not the gateway’s. It does not retain history beyond the current gauge value — a scrape that misses a transient spike has genuinely lost that data point, and if that loss matters, the answer is a shorter scrape interval or the health-snapshot ring buffer described in the parent guide, not an in-process time series on the gateway itself. Keeping the exporter’s responsibility this narrow is what keeps its resource cost predictable regardless of how many gauges a later revision adds.

from prometheus_client import Gauge, CollectorRegistry, generate_latest, CONTENT_TYPE_LATEST

registry = CollectorRegistry()

GATEWAY_TEMP_C = Gauge(
    "gateway_temp_celsius", "SoC die temperature", registry=registry
)
GATEWAY_RSS_BYTES = Gauge(
    "gateway_rss_bytes", "Process resident set size", registry=registry
)
GATEWAY_QUEUE_DEPTH = Gauge(
    "gateway_sync_queue_depth", "Pending items in the store-and-forward queue", registry=registry
)
GATEWAY_CPU_FREQ_HZ = Gauge(
    "gateway_cpu_freq_hz", "Current CPU clock frequency", registry=registry
)

A dedicated CollectorRegistry() rather than the global default registry keeps this exporter’s metric set isolated and auditable — every gauge that exists on this device is declared in one place, and nothing from an imported library’s default collectors (which often includes process and platform metrics you did not ask for) leaks into the scrape response.

Caching reads instead of sampling on request #

The gauges above are cheap to read on a scrape, but they must never be cheap to populate on a scrape — populating them means touching sysfs, and a scraper hitting /metrics every 15 seconds must not become a second, uncoordinated sysfs polling loop competing with the one described in sysfs thermal polling for throttle-aware pipelines. The sampling loop owns the read; the exporter only ever calls .set() on values already in hand:

import asyncio
import resource

async def update_gauges_loop(read_sample, get_queue_depth):
    """Runs on the same cadence as the sampler; never invoked by the HTTP handler."""
    while True:
        sample = read_sample()  # temp_c, freq_khz, rss_kb from the sysfs sampler
        if sample.get("temp_c") is not None:
            GATEWAY_TEMP_C.set(sample["temp_c"])
        if sample.get("freq_khz") is not None:
            GATEWAY_CPU_FREQ_HZ.set(sample["freq_khz"] * 1000)
        rss_kb = sample.get("rss_kb")
        if rss_kb is None:
            rss_kb = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
        GATEWAY_RSS_BYTES.set(rss_kb * 1024)
        GATEWAY_QUEUE_DEPTH.set(get_queue_depth())
        await asyncio.sleep(2.0)

The HTTP handler that serves the scrape is then a pure read of already-set gauge state, with generate_latest() doing the text-format rendering:

from http.server import BaseHTTPRequestHandler, HTTPServer

class PrometheusHandler(BaseHTTPRequestHandler):
    def do_GET(self):
        if self.path != "/metrics":
            self.send_response(404)
            self.end_headers()
            return
        output = generate_latest(registry)
        self.send_response(200)
        self.send_header("Content-Type", CONTENT_TYPE_LATEST)
        self.send_header("Content-Length", str(len(output)))
        self.end_headers()
        self.wfile.write(output)

    def log_message(self, fmt, *args):
        pass  # request logging to flash storage is not free; skip it

def run_exporter(port=9100):
    HTTPServer(("0.0.0.0", port), PrometheusHandler).serve_forever()

Running run_exporter in its own thread — started once at boot, not per-scrape — is the one background cost this design accepts; it is a listening socket, not a polling loop, so it is idle between scrapes rather than burning cycles.

Constraint validation table #

Gotchas and edge cases #

• The default registry accumulates surprises. Importing a library that happens to also import prometheus_client can register its own collectors against the global default registry, which then show up in your scrape response uninvited. Always create and pass an explicit CollectorRegistry() as done above.
• Metric name changes break dashboards silently. prometheus_client will happily let you rename a gauge between deployments; nothing in the library warns a downstream dashboard that the series changed. Treat metric names as part of the deployment’s stable interface once a fleet is scraping them.
• Gauge units must be explicit in the name. gateway_cpu_freq_hz versus a raw kHz reading is a common source of an order-of-magnitude bug; the Prometheus naming convention expects base units (Hz, bytes, seconds), so convert at the .set() call site, not in a dashboard query.
• A crashed exporter thread does not crash the pipeline — and that can hide a problem. Because the HTTP server runs in a background thread, an unhandled exception in the request handler can silently kill the scrape endpoint while the spatial pipeline keeps running. Wrap run_exporter in a supervisor that restarts the thread and logs the failure rather than letting the endpoint go dark unnoticed.
• generate_latest() cost scales with gauge count, not scrape frequency. It is cheap at four gauges; it stops being cheap if a later change adds label combinations without revisiting the cardinality budget. Re-check this page’s constraint table before adding a fifth metric.
• Port choice matters more on a gateway than on a server. Binding to a well-known port (9100, the conventional node-exporter port) on a device that also runs other services risks a collision that only surfaces after deployment. Confirm the port is free on the target image, and prefer a private or firewalled interface over 0.0.0.0 unless the LAN segment scraping it is already isolated from the wider network the gateway’s modem connects to.

Integrating with the gateway pipeline #

Start the exporter thread once at process boot, alongside — not inside — the async ingestion loop, and feed it the same queue-depth accessor the sync layer already exposes:

import threading

def bootstrap(read_sample, sync_queue):
    threading.Thread(target=run_exporter, daemon=True).start()
    asyncio.ensure_future(
        update_gauges_loop(read_sample, lambda: sync_queue.qsize())
    )

For a systemd-managed deployment, expose the exporter on a private interface or loopback-only address unless the LAN segment is already trusted, and pair MemoryLimit= on the unit with the fixed-gauge design above so a leak in an unrelated part of the process is caught by cgroups before it silently changes the exporter’s own memory footprint. The queue-depth gauge populated here is the same signal that drives the alerting thresholds in queue-depth alerting thresholds for edge sync — the exporter and the alerting logic should read from one shared accessor, never two independent queue inspections.

A fleet of these gateways scraped by one collector on the same LAN segment (a site gateway, a local Prometheus instance, or a lightweight collector agent on a field laptop during a maintenance visit) is the intended shape of this pattern. It is not designed for a central, cloud-hosted Prometheus server pulling directly over the cellular uplink — that path belongs to the compressed, opportunistic health-snapshot reporting the parent guide describes, since a WAN-facing scrape target would either need to tolerate long, unpredictable latency or force the gateway to hold a persistent outbound tunnel open, both of which cost more than the metrics are worth on a metered link.

Related #

• Monitoring & Observability for Spatial Gateways — the hand-rolled registry this exporter replaces when a standard client library is worth the dependency.
• Sysfs Thermal Polling for Throttle-Aware Pipelines — the sampling loop that populates the temperature gauge without a request-time sysfs read.
• Queue-Depth Alerting Thresholds for Edge Sync — the alerting logic built on the same queue-depth signal exported here.