PMTiles vs MBTiles for read-only edge basemaps

Choosing the container for an on-device basemap looks like a format preference and is actually a decision about how many bytes the reader has to touch to answer one tile request. This page compares the two containers a field gateway realistically ships — MBTiles, a SQLite database with a fixed schema, and PMTiles, a single flat file with a hierarchical directory baked into its head — on a 512 MB ARM gateway running a read-only rootfs with the archive on eMMC.

It belongs to the Core Edge GIS Fundamentals guide and to the offline tile and basemap storage topic, which covers the read path, the hot cache and the coverage policy that both formats sit underneath. Here the question is narrower: given that read path, which container costs less to run, and under which conditions does the answer flip?

Selection rationale: what actually differs #

Both formats store the same thing — compressed tile blobs addressed by z/x/y — and both are single files that can be copied, checksummed and swapped atomically. The differences that matter on a constrained device are in the lookup structure and in what the process has to keep resident.

MBTiles resolves a tile through SQLite’s B-tree index on (zoom_level, tile_column, tile_row). The index pages are read through SQLite’s pager, which means the working set is whatever B-tree interior pages the recent queries touched — typically 100–400 KB for a regional archive. SQLite itself costs about 700 KB of resident memory once loaded, and it is already present on every Linux gateway image, so the marginal cost is close to zero.

PMTiles resolves a tile through its own directory structure: a root directory in the header, and leaf directories fetched on demand. The design target was HTTP range requests against object storage, and that constraint produced something that happens to suit flash extremely well — the directory is a few kilobytes, the entries are delta-encoded and run-length compressed, and a lookup is one or two reads with no query planner in between. The reader is a few hundred lines with no dependency at all.

The practical consequence is that MBTiles wins where SQLite is already a dependency and the deployment wants one storage technology for tiles, the spool and the reference layer; PMTiles wins where the reader has to be minimal, where the archive may also be served over HTTP from the same file, or where a language without a good SQLite binding is doing the reading.

A reader that works against either container #

The pragmatic answer for a fleet that has not committed yet is to keep the archive behind an interface with two implementations, so the container becomes a provisioning decision rather than a code decision. The PMTiles side is short enough to write by hand:

# pmtiles_reader.py — minimal read-only PMTiles v3 lookup.
# No third-party dependencies; one file handle, no threads, no writes.
# Call from a single reader task, or give each task its own instance.
import struct
import gzip
from typing import NamedTuple

HEADER_LEN = 127


class Entry(NamedTuple):
    tile_id: int
    offset: int
    length: int
    run: int


def zxy_to_tile_id(z: int, x: int, y: int) -> int:
    """Hilbert-order tile id — the ordering PMTiles stores entries in."""
    acc = 0
    for t in range(z):
        acc += (1 << t) * (1 << t)
    n = 1 << z
    rx = ry = 0
    d = 0
    s = n >> 1
    while s > 0:
        rx = 1 if (x & s) > 0 else 0
        ry = 1 if (y & s) > 0 else 0
        d += s * s * ((3 * rx) ^ ry)
        # rotate
        if ry == 0:
            if rx == 1:
                x = s - 1 - x
                y = s - 1 - y
            x, y = y, x
        s >>= 1
    return acc + d


class PMTiles:
    def __init__(self, path: str):
        self._fh = open(path, "rb", buffering=0)
        head = self._fh.read(HEADER_LEN)
        if head[:7] != b"PMTiles":
            raise ValueError("not a PMTiles archive")
        # v3 header: offsets/lengths are little-endian uint64 at fixed positions.
        (self._root_off, self._root_len,
         self._meta_off, self._meta_len,
         self._leaf_off, self._leaf_len,
         self._data_off, self._data_len) = struct.unpack_from("<8Q", head, 8)
        self._internal_compression = head[97]
        self._root = self._read_dir(self._root_off, self._root_len)

    def _read_dir(self, off: int, length: int) -> list[Entry]:
        self._fh.seek(off)
        raw = self._fh.read(length)
        if self._internal_compression == 2:          # gzip
            raw = gzip.decompress(raw)
        return _decode_directory(raw)

    def get(self, z: int, x: int, y: int) -> bytes | None:
        tid = zxy_to_tile_id(z, x, y)
        entries = self._root
        for _ in range(4):                            # depth is 1–2 in practice
            e = _find(entries, tid)
            if e is None:
                return None
            if e.run == 0:                            # points at a leaf directory
                entries = self._read_dir(self._leaf_off + e.offset, e.length)
                continue
            self._fh.seek(self._data_off + e.offset)
            return self._fh.read(e.length)
        return None


def _find(entries: list[Entry], tid: int) -> Entry | None:
    """Binary search honouring run-length entries."""
    lo, hi = 0, len(entries) - 1
    while lo <= hi:
        mid = (lo + hi) // 2
        e = entries[mid]
        if tid < e.tile_id:
            hi = mid - 1
        elif tid >= e.tile_id + max(e.run, 1):
            lo = mid + 1
        else:
            return e
    # A run-length entry can still cover tid via the preceding entry.
    if hi >= 0 and entries[hi].run == 0:
        return entries[hi]
    return None

The _decode_directory helper (omitted for length) unpacks four varint arrays — tile ids as deltas, run lengths, entry lengths, and offsets with a zero meaning “immediately after the previous entry”. That encoding is why a directory covering tens of thousands of tiles fits in a few kilobytes, and it is the single design decision that makes the format viable on a device that cannot hold a full index.

Constraint validation #

The last row is the one most often decisive in practice and least often considered at design time. A technician with a serial console can run three SQL statements against an MBTiles archive and answer “which zoom levels are present for this area” without any custom tooling. Getting the same answer from a PMTiles file means shipping a command that can read it — which is worth doing regardless, and is a real cost in an image where every binary is scrutinised.

Gotchas and edge cases #

• The y axis. MBTiles stores rows in TMS order (y counted from the south); PMTiles uses Hilbert-ordered tile ids derived from XYZ coordinates. Convert at exactly one boundary and assert it with a fixture: pick a tile containing a recognisable coastline and compare the rendered output against the source. A silent flip renders a plausible map of the wrong place.
• Compression is inside the tile, not the container. Both formats store already-gzipped vector tiles and record that fact in metadata. A reader that gzips again on read wastes cycles; one that fails to decompress passes compressed bytes to the renderer and produces an empty map with no error.
• SQLite’s immutable=1 is a promise, not a hint. It tells SQLite the file will not change while open. Violating it — by swapping the archive under a live reader — produces undefined results rather than an error. Close, swap, reopen.
• PMTiles directory depth varies with archive size. Small archives have a single root directory and no leaves; large ones add a leaf level. A reader that assumes one level works perfectly until the first archive that crosses the threshold.
• Neither format stores an integrity hash. Add one alongside — the same detached signature used for signed tile and config bundles — because a truncated archive is otherwise indistinguishable from one that simply lacks coverage.
• Metadata differs. MBTiles’ metadata table and PMTiles’ JSON metadata block carry the same information under different names. Normalise both into one internal structure at load time so the rest of the code does not branch on container type.

Integration with the tile server #

Both readers slot behind the same interface, which is what keeps the container a deployment decision:

# tile_source.py — one interface, two backends, chosen at start-up.
from pathlib import Path

class TileSource:
    """Resolve (z, x, y) to bytes, whatever the container underneath."""

    def __init__(self, path: str):
        p = Path(path)
        if p.suffix == ".pmtiles":
            from pmtiles_reader import PMTiles
            self._impl = PMTiles(str(p))
            self.kind = "pmtiles"
        else:
            from edge_tiles import TileArchive
            self._impl = TileArchive(str(p))
            self.kind = "mbtiles"

    def get(self, z: int, x: int, y: int) -> bytes | None:
        return self._impl.get(z, x, y)

Wire that into the local HTTP surface described in serving vector tiles from a read-only SQLite cache, and record kind in the health snapshot so a field diagnosis never has to guess which container is mounted.

Related #

• Offline Tile & Basemap Storage — the read path, coverage policy and failure modes both containers share.
• Serving vector tiles from a read-only SQLite cache — the local HTTP layer that sits in front of this reader.
• Pruning tile caches under a flash write budget — what to do about the writable half of the store.