Edge Geospatial IoT Gateway Processing

Integer Microdegree Quantization for Coordinate Storage

This guide solves a storage question distinct from how a coordinate travels over the wire: once a fix is confirmed, what should the gateway hold it in — a float64 degree pair, or a signed 32-bit integer count of microdegrees? Within the Core Edge GIS Fundamentals framework, and specifically the decimal-budget contract set out in spatial data precision standards, the microdegree convention — scaling degrees by 10^6 and storing the result as int32 — is the representation MAVLink telemetry, S2 geometry, and OpenStreetMap’s node table all converge on independently, because the range and precision math works out cleanly for exactly the coordinates a gateway needs to hold at rest: not a wire delta, a database column, a struct field, a row in a flat binary log.

Choosing int32 microdegrees over float64: the range and precision math

A float64 coordinate pair costs 16 bytes and carries roughly 15–17 significant decimal digits — far more precision than any GNSS receiver resolves, and more than any spatial index or comparison on the gateway needs to pay for. Scaling to microdegrees converts each coordinate to an integer: value_e6 = round(degrees × 10^6). One microdegree of latitude change converts to ground distance by the same relation used throughout this site’s precision guides:

Δμ=10−6 deg  ⟹  dlat≈111,320×10−6 m≈0.111 m\Delta_{\mu} = 10^{-6}\ \text{deg} \implies d_{\text{lat}} \approx 111{,}320 \times 10^{-6}\ \text{m} \approx 0.111\ \text{m}

That is finer than a consumer GNSS module’s noise floor by roughly an order of magnitude, so the quantization step itself never becomes the limiting factor on positional accuracy — the receiver’s own drift dominates long before microdegree rounding would. The reason int32 — not int16 or int64 — is the natural container is a headroom question: the full longitude range is ±180°, which at 10^6 scale is ±1.8 × 10^8, comfortably inside signed 32-bit range:

∣valuee6∣≤180×106=1.8×108  ≪  231−1≈2.147×109|{\text{value}_{e6}}| \le 180 \times 10^{6} = 1.8 \times 10^{8} \;\ll\; 2^{31} - 1 \approx 2.147 \times 10^{9}

That headroom is worth checking explicitly rather than assuming, because it is not universal across scale choices. A 10^7 (micro-arcsecond-adjacent) scale — the resolution some survey-grade pipelines reach for — pushes the same longitude range to ±1.8 × 10^9, which still fits inside int32 but leaves less than a factor of 1.2 of margin below the type’s ceiling. Any intermediate arithmetic that adds two coordinates, doubles a value during a reprojection step, or accumulates a running sum during delta reconstruction is one careless operation away from signed overflow at 10^7 scale, whereas 10^6 microdegrees leaves roughly twelve times the margin — headroom that absorbs sums, doubling, and off-by-a-bit arithmetic mistakes without needing int64 at all. Reach for int64 only when the scale factor or the arithmetic genuinely demands the extra range; carrying 8 bytes per coordinate when 4 would do doubles storage for no precision benefit.

Storage cost of a float64 coordinate pair versus an int32 microdegree pair A float64 latitude-longitude pair costs sixteen bytes, two eight-byte doubles. An int32 microdegree pair storing the same position at 10 to the minus 6 degree resolution costs eight bytes, two four-byte signed integers, half the storage, while remaining an order of magnitude finer than typical GNSS receiver accuracy. An arrow between the two boxes is labelled with the halving and the fact that boundary comparisons become exact integer equality instead of floating point tolerance checks. float64 pair lat: 8 B, lon: 8 B 16 bytes total exact compares need an epsilon int32 microdegree pair lat_e6: 4 B, lon_e6: 4 B 8 bytes total bit-exact integer boundary compares 50% smaller
Half the storage per coordinate and, because comparisons are now integer equality rather than float tolerance, no epsilon guesswork at geofence and grid boundaries.

A packed on-disk record and exact boundary comparisons

The centerpiece below defines the at-rest layout for one telemetry observation — an id, a microdegree coordinate pair, and a one-byte fix-quality flag — packed with an explicit byte layout rather than left to the compiler’s or Python’s default alignment rules, and a bounding-box test written as exact integer comparison with no epsilon anywhere in it.

# microdegree_record.py
# On-disk/in-memory record layout for a single coordinate observation,
# stored as int32 microdegrees. Designed for a flat binary log or a
# fixed-width database column, not for wire transmission.
import struct

# Explicit little-endian, no native padding: id(i32) + lat_e6(i32) + lon_e6(i32) + quality(u8)
# '<' forces standard sizes with no compiler-inserted alignment padding.
RECORD_FMT = "<iiiB"
RECORD_SIZE = struct.calcsize(RECORD_FMT)  # 13 bytes, not 16 -- no padding with '<'

SCALE = 10 ** 6  # microdegrees

def to_microdegrees(lat: float, lon: float) -> tuple[int, int]:
    """Quantize a float64 coordinate to signed-32-bit microdegrees.
    Range check happens once, at the storage boundary, not on every
    later comparison against it."""
    lat_e6 = round(lat * SCALE)
    lon_e6 = round(lon * SCALE)
    if not (-90 * SCALE <= lat_e6 <= 90 * SCALE):
        raise ValueError(f"latitude {lat} out of range at microdegree scale")
    if not (-180 * SCALE <= lon_e6 <= 180 * SCALE):
        raise ValueError(f"longitude {lon} out of range at microdegree scale")
    return lat_e6, lon_e6

def pack_record(rec_id: int, lat: float, lon: float, quality: int) -> bytes:
    lat_e6, lon_e6 = to_microdegrees(lat, lon)
    return struct.pack(RECORD_FMT, rec_id, lat_e6, lon_e6, quality)

def unpack_record(payload: bytes) -> tuple[int, int, int, int]:
    return struct.unpack(RECORD_FMT, payload)

def in_bbox_e6(lat_e6: int, lon_e6: int,
              min_lat_e6: int, min_lon_e6: int,
              max_lat_e6: int, max_lon_e6: int) -> bool:
    """Bounding-box membership as exact integer comparison. No epsilon:
    two microdegree integers are either equal or they are not, so a
    point exactly on the stored boundary compares correctly every time."""
    return (min_lat_e6 <= lat_e6 <= max_lat_e6) and (min_lon_e6 <= lon_e6 <= max_lon_e6)

The to_microdegrees range check belongs at the storage boundary, run once per observation, not sprinkled through every downstream comparison — every value that makes it past pack_record is guaranteed to fit its 32-bit field, so in_bbox_e6 never has to defend against overflow, only compare. That is the real payoff of quantizing to integers for storage rather than keeping floats at rest: a boundary test becomes two <= comparisons per axis with a deterministic answer, rather than a float comparison that needs a tolerance to account for representation error — the same class of problem the WGS84-vs-UTM guide solves for the CRS layer by moving to planar Euclidean math instead of geodesic trigonometry.

Constraint validation table

Constraint Expected impact Mitigation built into the code
RAM A float64 pair costs 16 bytes; ten thousand cached observations at that rate is 160 KB versus 80 KB int32 microdegree pair halves per-record storage; RECORD_SIZE computed once via struct.calcsize
CPU Float bounding-box comparisons need an epsilon guard to avoid representation-error false negatives at boundaries Integer comparisons in in_bbox_e6 are exact; no epsilon constant to tune or get wrong
Latency Re-validating range on every comparison wastes cycles on data that was already checked once Range check runs once in to_microdegrees at the storage boundary, not on every later read
Power Larger records mean more bytes written per flush to flash storage, more write-amplification cycles 13-byte packed records versus a naive 16+ byte float layout reduce flash writes per observation

Gotchas and edge cases

  • Struct field ordering interacts with alignment even when you think you have avoided it. The RECORD_FMT above uses < (explicit little-endian, standard sizes, no padding), which is what keeps it at 13 bytes. Drop the < and use the platform-native @ format, and struct inserts padding to align the int32 fields to their natural 4-byte boundaries — the same record can silently grow to 16 bytes on some platforms and stay 13 on others, corrupting any file written by one and read by the other.
  • A C-side struct needs the matching attribute, not just matching field types. A naive struct { int32_t id; int32_t lat_e6; int32_t lon_e6; uint8_t quality; }; in C is not automatically 13 bytes — the compiler pads the trailing uint8_t up to the struct’s 4-byte alignment requirement, producing 16 bytes overall. Add __attribute__((packed)) to force the compiler to match Python’s <iiiB layout exactly, and be aware that packed structs can incur an unaligned-access penalty (or a fault, on strict-alignment configurations) when the CPU reads the int32 fields at a non-4-byte-aligned offset — profile on the actual target rather than assuming it is free.
  • Range checks belong at ingestion, not scattered through every comparison. If to_microdegrees is skipped anywhere in the ingestion path — a second code path that packs records directly from a cache, for instance — an out-of-range value can slip through and either raise deep inside an unrelated comparison or, worse, silently wrap if the check is missing entirely. Centralize the check at the one function every record must pass through before it is packed.
  • Microdegrees are not free precision beyond what the receiver can prove. A quantized int32 value is exact once stored, but that exactness describes the stored number, not the ground truth — a value that looks bit-precise to 0.111 m still inherits whatever error the GNSS fix carried before quantization. Do not treat the storage format’s precision as a substitute for the receiver’s actual accuracy budget.
  • Scale factor is a schema decision, not a per-record one. Every reader of a stored file or database column must agree on 10^6 (or whatever scale the deployment picked) with no per-record variation; storing the scale factor once in a file header or column comment, and asserting it on read, catches a schema mismatch before it silently shifts every coordinate by a power of ten.

Integrating with the storage layer

Wire pack_record/unpack_record into whatever holds observations at rest — a SQLite table with INTEGER columns, a flat append-only log, or a shared-memory ring buffer feeding a UTM projection stage:

import sqlite3

def create_observation_table(conn: sqlite3.Connection) -> None:
    # INTEGER columns store the microdegree values directly -- no REAL
    # columns, no float rounding introduced by the database engine itself.
    conn.execute("""
        CREATE TABLE IF NOT EXISTS observations (
            rec_id INTEGER PRIMARY KEY,
            lat_e6 INTEGER NOT NULL,
            lon_e6 INTEGER NOT NULL,
            quality INTEGER NOT NULL
        )
    """)
    conn.execute("PRAGMA journal_mode=WAL")
    conn.execute("PRAGMA synchronous=NORMAL")

def insert_observation(conn: sqlite3.Connection, rec_id: int, lat: float, lon: float, quality: int) -> None:
    lat_e6, lon_e6 = to_microdegrees(lat, lon)
    conn.execute(
        "INSERT INTO observations (rec_id, lat_e6, lon_e6, quality) VALUES (?, ?, ?, ?)",
        (rec_id, lat_e6, lon_e6, quality),
    )

Keeping the column type INTEGER end to end means SQLite’s own query planner can use exact integer range predicates (WHERE lat_e6 BETWEEN ? AND ?) for a bounding-box query without ever touching floating-point comparison inside the database engine — the same exactness the in_bbox_e6 function above provides in application code, now pushed down to the storage layer itself.