Spatial Data Precision Standards

Within the Core Edge GIS Fundamentals framework, this guide defines how much coordinate precision to keep — and how to enforce it — when telemetry flows from GNSS hardware through a constrained gateway to the cloud.

Geospatial telemetry at the network edge operates under strict computational and bandwidth ceilings. When IoT sensors, RTK receivers, and gateway processors stream coordinate data to downstream analytics or a backhaul link, unmanaged floating-point precision becomes a silent failure vector: it inflates payloads, fragments the heap, and disguises sensor noise as signal. Treat precision as an operational contract that fixes storage footprint, sync reconciliation latency, and field-grade positional reliability at the bill-of-materials stage — not a cosmetic rounding decision made at the end. This guide details constraint-aware precision management, edge-optimized Python and C implementations, and diagnostic workflows for IoT engineers, field GIS technicians, and embedded gateway teams.

The precision contract set here is the input to everything downstream: a coordinate quantized once at ingestion feeds on-device geometry filtering, survives the delta sync for spatial datasets reconciler when the link returns, and bounds the work the coordinate reference system transform has to do. Over-keep precision and every later stage pays for it in RAM and bytes; under-keep it and you discard survey accuracy you cannot recover.

Precision vs. Accuracy in Edge Telemetry #

Precision defines the repeatability and decimal resolution of a coordinate value; accuracy measures its deviation from ground truth. The two are independent axes — the failure that dominates edge telemetry is the top-left quadrant below: a GNSS module reports many decimal digits (high precision) while its true position is capped by the hardware noise floor (low accuracy), so the extra digits encode noise as if it were survey data.

At the edge, precision is routinely inflated by default IEEE 754 double-precision outputs from GNSS modules or IMU fusion stacks. A sensor reporting lat: 45.12345678901234 consumes 8 bytes per float64, but the underlying hardware noise floor rarely beats ±0.00001° (about 1.1 m at mid-latitudes) without RTK correction. Transmitting 15 significant digits encodes thermal noise as if it were survey data — it wastes RAM, bloats MQTT payloads, and enlarges every spatial index on the gateway.

The decimal resolution you keep maps directly to a ground distance. For a latitude change the relationship is linear in Earth radius R; for longitude it shrinks with the cosine of latitude:

$$\Delta_{\text{lat}} \approx R \, \Delta\varphi, \qquad \Delta_{\text{lon}} \approx R \, \cos\varphi \; \Delta\lambda$$

with R ≈ 6 371 000 m and angles in radians. One unit in the n-th decimal place of a degree is therefore worth roughly 111 320 / 10^n metres of latitude. That single conversion is what lets you pick a decimal budget from a tolerance instead of guessing:

Pin the precision standard to the operational use case, not to whatever the module emits:

• Asset tracking and fleet routing: 5 decimal places is sufficient for road snapping and geofencing.
• Utility mapping and cadastral sync: 6–7 places for boundary alignment and pipe/cable routing.
• Survey-grade IoT and structural monitoring: 8+ places held in fixed-point or decimal arithmetic, paired with hardware error budgets and continuous RTK validation.

The Precision Pipeline #

The core architectural rule is clamp once at ingestion, carry integers across the device, and restore scale only at the cloud. Quantize the raw float64 to the use-case decimal budget the moment it arrives, pack it as a scaled integer for all on-device transport and storage, and apply the inverse scale only at the visualization or ingestion boundary.

Clamp at ingestion, pack as integers, and restore scale only at the cloud — the device never carries spurious precision.

Constraint Mapping: What the Hardware Forces #

Four hardware limits decide which precision pattern is viable. Map your target against them before choosing a representation — the right answer on a 512 MB Cortex-A53 gateway is the wrong answer on a Cortex-M7. These envelopes are the same ones catalogued in Device Constraints & Resource Limits; precision is one of the cheapest line items to get right and one of the most expensive to get wrong.

Implementation 1: Fixed-Point Scaling and Binary Packing #

The primary technique converts each coordinate to a scaled signed integer and serializes it with struct, avoiding floating-point text conversion entirely. Quantize with Python’s decimal module first to get deterministic, banker’s-free rounding, then multiply by the decimal scale and pack. At 5 decimals the range is ±9 000 000 for latitude and ±18 000 000 for longitude — comfortably inside signed int32 (±2 147 483 647), so two coordinates fit in 8 bytes instead of the 16 a float64 pair would need.

This runs on the asyncio ingestion path. The packing function is pure and allocation-light (no per-call object churn beyond the two Decimal temporaries, which are short-lived and GC-cheap); only the publish coroutine touches the event loop.

import struct
import asyncio
from decimal import Decimal, ROUND_HALF_UP

# Fixed-point scaling + binary packing for constrained transport.
# Pure function, no I/O — safe to call from the asyncio ingestion path.
def clamp_and_pack(lat: float, lon: float, decimals: int = 5) -> bytes:
    scale = 10 ** decimals
    quantum = Decimal(1).scaleb(-decimals)  # e.g. Decimal('0.00001')
    # Quantize to exact decimal places (deterministic rounding), then scale.
    lat_int = int(Decimal(str(lat)).quantize(quantum, rounding=ROUND_HALF_UP) * scale)
    lon_int = int(Decimal(str(lon)).quantize(quantum, rounding=ROUND_HALF_UP) * scale)
    # Two 32-bit signed ints, little-endian, explicit standard alignment ('<').
    # At 5 decimals: lat ±9_000_000, lon ±18_000_000 — both within int32.
    return struct.pack("<ii", lat_int, lon_int)

def unpack(payload: bytes, decimals: int = 5) -> tuple[float, float]:
    lat_int, lon_int = struct.unpack("<ii", payload)
    scale = 10 ** decimals
    return lat_int / scale, lon_int / scale

# Async MQTT gateway handler with a local fallback on link loss.
async def publish_telemetry(mqtt_client, topic: str, payload: bytes, spool: asyncio.Queue):
    try:
        await mqtt_client.publish(topic, payload, qos=1)
    except (ConnectionError, asyncio.TimeoutError):
        # Backhaul down — spool the *already-packed* bytes; never re-encode.
        if spool.full():
            spool.get_nowait()        # drop oldest, bounded memory
        spool.put_nowait(payload)

Two rules make this safe in the field. First, always pass an explicit byte-order/alignment prefix to struct (< for little-endian standard size); the native @ default re-introduces platform padding and will mis-align an embedded C buffer on a cross-compiled target. Second, spool the packed bytes, not the float — re-encoding on the retry path is where rounding drift sneaks back in.

Implementation 2: Holding High Precision Without float64 #

Survey-grade and FFI-bound workloads need a complementary representation. Two variants cover almost every case.

Fixed-point at the C boundary. When bridging Python to C/C++ via ctypes, cffi, or pybind11 for hot-path geometry, cast coordinates to int32 (or int64 for sub-millimetre) before crossing the boundary. Python’s native float is 64-bit; handing it to a struct that expects a 32-bit field silently truncates or shifts every coordinate. Keep the integer contract end to end:

// coord.h — header-only, integer-domain coordinate packing for the FFI boundary.
// Compile the consumer with: -O2 -fno-exceptions -fno-rtti
#include <stdint.h>

// Fixed-point degrees: value = degrees * 10^5  (5 decimal places, ~1.1 m).
typedef struct __attribute__((packed)) {
    int32_t lat_e5;   // matches Python struct '<ii'
    int32_t lon_e5;
} coord_e5_t;

// Integer geofence test — no floating point, deterministic on any MCU.
static inline int inside_bbox_e5(coord_e5_t p,
                                 int32_t min_lat, int32_t min_lon,
                                 int32_t max_lat, int32_t max_lon) {
    return (p.lat_e5 >= min_lat) & (p.lat_e5 <= max_lat) &
           (p.lon_e5 >= min_lon) & (p.lon_e5 <= max_lon);
}

Because both sides agree on int32 at 1e5 scale, the geofence test, containment checks, and the polygon containment routines run entirely in integer arithmetic — no FFI float conversion, no rounding ambiguity, and the same result on a Cortex-M as on the gateway.

Decimal-degree serialization for JSON/GeoJSON. When the wire format must stay human-readable, clamp before stringifying. For GeoJSON output, hold to the RFC 7946 recommendation of 6 decimal places, which caps both client parsing cost and payload size:

import json

def to_geojson_point(lat: float, lon: float, decimals: int = 6) -> str:
    # RFC 7946 §11.2: precision beyond ~6 dp is rarely meaningful and bloats payloads.
    lon_r = round(lon, decimals)
    lat_r = round(lat, decimals)
    return json.dumps({"type": "Point", "coordinates": [lon_r, lat_r]},
                      separators=(",", ":"))  # no whitespace on the wire

Note the coordinate order: GeoJSON is [lon, lat], the opposite of the lat, lon most sensor APIs emit. Getting this backwards is the most common precision-adjacent field bug, and it survives every numeric test because both values are plausible degrees.

Configuration & Tuning #

Precision behaviour is governed mostly by a handful of constants and serialization flags. Lock them into the firmware image so a coordinate cannot silently change scale between build and deploy.

# Precision policy — single source of truth, set per deployment profile.
PRECISION_DECIMALS = 5          # 5 dp ≈ 1.1 m; raise to 6/7 only for RTK fleets
COORD_SCALE = 10 ** PRECISION_DECIMALS
SPOOL_DEPTH = 4096              # bounded ring buffer; * 8 bytes ≈ 32 KB resident
STRUCT_FMT = "<ii"             # explicit little-endian, standard size — no native padding

Knobs that matter on constrained targets:

• decimals is the whole contract. Set it once from the use-case table above; never let two stages disagree, or the inverse scale at the cloud will be wrong by a power of ten.
• struct format prefix: always < or >, never the default @. Verify on the target with struct.calcsize("<ii") returning 8, not a padded value.
• ROUND_HALF_UP in decimal gives predictable half-up rounding; Python’s built-in round() uses banker’s rounding (round-half-to-even), which will disagree at the boundary. Pick one and use it on both ends.
• SQLite spool schema: store coordinates as INTEGER, not REAL. Add PRAGMA journal_mode=WAL; and PRAGMA synchronous=NORMAL; so the spool survives power loss without paying a full fsync per insert. Integer columns also keep the on-disk index small under a read-only-rootfs flash budget.

For the C consumer, compile with -O2 -fno-exceptions -fno-rtti and confirm sizeof(coord_e5_t) == 8; the __attribute__((packed)) removes any padding the compiler would otherwise insert between the two int32 fields.

Verification & Field Diagnostics #

Precision drift rarely crashes — it manifests as coordinate jitter during sync reconciliation, spatial-join misses, or phantom geofence triggers. Diagnosis means isolating the failure vector: sensor hardware, the FFI boundary, or network serialization. Implement these checks in gateway firmware so a scale or datum problem is caught before it corrupts a shift of telemetry.

1. Hardware fix gating. Parse the NMEA GGA sentence and watch the RTK fix-quality field. While the fix is FLOAT rather than FIXED, force decimals down to 4 — emitting 7 decimal places off an unconverged fix transmits noise dressed as survey data. This couples cleanly with the projection discipline in coordinate reference systems at the edge, where the same untrusted fix must not be reprojected at full precision.
2. Alignment self-check on boot. Assert struct.calcsize(STRUCT_FMT) == 8 and round-trip a known coordinate through clamp_and_pack/unpack; a mismatch means the format prefix or the C struct layout drifted. Fail to safe-mode rather than ship shifted coordinates.
3. Scale-consistency probe. Periodically pack-then-unpack a fixed control coordinate and assert the result is within one quantum (1 / COORD_SCALE) of the input. A delta of an exact power of ten means two stages disagree on decimals.
4. Reconciliation variance window. Buffer high-frequency samples and apply median smoothing or a lightweight scalar Kalman filter; flush only when positional variance drops below the target quantum. This keeps the delta sync reconciler from treating jitter as real movement and re-sending unchanged points.

Failure Modes Specific to This Pattern #

Precision bugs are quiet because every intermediate value stays a plausible coordinate. The table maps the failures worth alerting on, how each surfaces on a deployed device, and the safe recovery path.

Deploy the clamp at the ingestion layer, carry integers across the device, gate precision on the hardware fix state, and validate scale on boot. That eliminates floating-point bloat, stabilizes the edge memory footprint, and guarantees field-grade positional reliability across constrained IoT networks.

Related #

• Coordinate Reference Systems at the Edge — the transform contract every quantized coordinate must honour.
• Device Constraints & Resource Limits — the RAM, bandwidth, and flash envelope precision has to fit inside.
• Fallback Routing & Offline Navigation — where integer fixed-point coordinates keep navigation alive offline.
• Delta Sync for Spatial Datasets — how the precision contract bounds what the reconciler re-sends.
• Implementing Polygon Containment Checks in C — integer-domain geometry that consumes the packed int32 coordinates.
• Core Edge GIS Fundamentals — the full field reference this guide belongs to.