Zstd vs Brotli for MVT Payloads on Metered LTE

This guide answers a question that recurs on every gateway serving Mapbox Vector Tiles (MVT) over a metered cellular backhaul: given a protobuf-encoded tile in the tens-of-kilobytes range, does Zstandard or Brotli win once you weigh compression ratio against the CPU budget and decode latency the receiving side has to pay? Within the Bandwidth & Async Sync Optimization practice, and specifically the entropy-coding stage of the compression strategies for geospatial payloads this guide belongs to, MVT tiles are a distinctive workload: small, individually low-entropy protobuf messages with a lot of shared structure — repeated tag bytes, coordinate deltas, a handful of layer names — that neither codec can exploit fully without help.

Selecting a codec for small, structurally repetitive tiles #

Brotli’s static dictionary was built for web text and general binary data, not for MVT’s specific field layout, so its advantage on a single small tile comes almost entirely from its context-modeling entropy coder rather than any tile-aware knowledge. At quality 9–11 Brotli routinely beats Zstandard’s default settings on ratio for an individual tile, but the cost scales badly: quality 11 on a 30 KB tile can take 40–80 ms on a Cortex-A53, which is untenable if a tile server on the gateway is filling a viewport’s worth of requests under load.

Zstandard’s real advantage over Brotli for this workload is not raw ratio at equal CPU cost — it is dictionary training. A single MVT tile is too small for either codec’s built-in windowing to find much repetition; the previous tile in the same layer, at the same zoom level, looks almost identical in its tag structure and property-key strings. zstd --train builds a shared dictionary from a corpus of representative tiles, and that dictionary is loaded once into the compressor and decompressor, letting every subsequent tile compress against patterns pulled from thousands of prior tiles instead of its own few kilobytes. That is the single largest lever available for tile-sized payloads, and it has no equivalent in Brotli’s standard workflow — Brotli does support a custom dictionary parameter, but the tooling and community practice around it is far thinner than Zstandard’s mature --train/ZDICT pipeline.

The selection therefore is not “faster codec wins” but “which codec’s tuning knob matches the shape of a tile stream.” A fleet serving many small, structurally similar tiles from the same schema should train and ship a Zstandard dictionary; a batch job compressing large, one-off archival exports — where a shared dictionary provides no benefit because there is no repeated small-payload corpus — should reach for Brotli’s higher quality levels instead, the same trade-off already worked out for Brotli-compressed shapefile chunks.

Training and applying a tile dictionary #

The centerpiece here is dictionary training, because it is the one technique with no Brotli equivalent in common use and the one that actually changes the shape of the trade-off for a tile-serving gateway. First, build the dictionary offline from a representative sample of tiles — ideally pulled from the same zoom level and layer schema the gateway will serve in production:

# Train a 16 KB dictionary from a sample of same-schema MVT tiles.
# Run this offline during provisioning, not on the gateway itself --
# training scans the whole corpus and is far heavier than compressing one tile.
zstd --train samples/*.mvt -o tile_dict.zstd --maxdict=16384

Then load the trained dictionary at both ends of the link. The gateway compresses against it; the client (or the cloud ingestor, for uplink tiles) decompresses against the identical dictionary bytes shipped once in the firmware image or fetched at provisioning time:

# mvt_zstd_dict.py
# Dictionary-backed Zstandard compression for small, structurally similar
# MVT tiles. The dictionary is loaded once per process; only the per-tile
# compress/decompress calls run in the request hot path.
import zstandard

DICT_PATH = "/etc/edge-gis/tile_dict.zstd"

def load_dictionary(path: str = DICT_PATH) -> zstandard.ZstdCompressionDict:
    with open(path, "rb") as f:
        return zstandard.ZstdCompressionDict(f.read())

# Reused across calls: constructing a compressor per tile re-pays dictionary
# setup cost every time, which erases most of the win for small payloads.
_dict = load_dictionary()
_compressor = zstandard.ZstdCompressor(level=9, dict_data=_dict)
_decompressor = zstandard.ZstdDecompressor(dict_data=_dict)

def compress_tile(mvt_bytes: bytes) -> bytes:
    """Compress one MVT tile against the shared dictionary."""
    return _compressor.compress(mvt_bytes)

def decompress_tile(payload: bytes) -> bytes:
    """Decompress a dictionary-backed tile back to raw MVT protobuf bytes."""
    return _decompressor.decompress(payload)

Level 9 with a trained dictionary on a ~25 KB tile typically lands within a few percent of Brotli quality 10’s ratio while costing a fraction of the CPU time, because the dictionary is doing the work that Brotli’s higher quality levels spend cycles searching for from scratch inside each tile in isolation. Retrain the dictionary whenever the tile schema changes meaningfully — a new layer, a renamed property key — since a stale dictionary still decompresses correctly but stops contributing much ratio.

Constraint validation table #

Gotchas and edge cases #

• A dictionary trained on the wrong schema hurts, it does not just underperform. If the sample corpus used for zstd --train comes from a different zoom level or a schema with different property keys than what the gateway actually serves, the dictionary can add overhead relative to no dictionary at all for badly mismatched tiles — always retrain from a fresh, representative sample after a schema or zoom-tier change.
• Brotli quality is not linear in cost. The jump from quality 5 to quality 9 buys meaningfully better ratio; the jump from 9 to 11 buys only a few more percent for roughly double the CPU time. Reserve quality 11 for genuinely offline batch work, never a request-path tile server.
• Decompression context reuse matters as much as compression context reuse. Constructing a fresh ZstdDecompressor per incoming tile on the client re-pays dictionary attachment cost on every request; cache it exactly like the compressor is cached on the gateway.
• MVT’s own delta-encoded coordinates already remove some redundancy. Tiles that have already had their geometry commands delta-encoded per the MVT spec compress somewhat worse in relative terms than naive coordinate dumps would, because the easy redundancy is already gone — do not expect the same ratio improvement you would see compressing raw, undeltified GeoJSON.
• Dictionary size is a real tuning knob, not a fixed constant. --maxdict values much larger than the corpus can support waste RAM without adding ratio; start at 16 KB for typical tile schemas and measure before growing it.

Integrating the codec choice into the tile server #

In the gateway’s tile-serving path, the dictionary-backed compressor sits directly behind MVT encoding and ahead of the transport layer, with content negotiation deciding which encoding a given client actually receives:

import asyncio

async def serve_tile(request, mvt_bytes: bytes) -> bytes:
    """Negotiate encoding, compress off the event loop, and return the
    compressed body. CPU-bound compression runs in a thread so it never
    stalls other in-flight requests on the same event loop."""
    accepts_zstd = "zstd" in request.headers.get("accept-encoding", "")
    loop = asyncio.get_running_loop()
    if accepts_zstd:
        body = await loop.run_in_executor(None, compress_tile, mvt_bytes)
        content_encoding = "zstd"
    else:
        import brotli
        body = await loop.run_in_executor(None, lambda: brotli.compress(mvt_bytes, quality=5))
        content_encoding = "br"
    return body, content_encoding

Falling back to Brotli quality 5 for clients that do not advertise zstd in Accept-Encoding keeps the path CPU-bounded without a dictionary dependency, since older map clients or intermediate proxies do not always support Zstandard content negotiation yet. Watch compression_cpu_ms per encoding path in production the same way the parent compression guide’s diagnostics recommend, and confirm the dictionary is actually attached rather than silently falling back to non-dictionary compression by asserting _compressor’s configured dictionary ID matches what the client expects.

Related #

• Compression Strategies for Geospatial Payloads — the parent guide’s codec-selection framework this comparison narrows to tile-sized payloads.
• Applying Brotli Compression to Shapefile Chunks — a worked Brotli implementation for the large, one-off archival case this guide contrasts against.
• FlatGeobuf vs GeoArrow for the Edge Wire Format — the structural-layer decision that precedes this entropy-coding stage.