MQTT persistent sessions vs local broker bridging

Two mechanisms let an MQTT client survive a disconnection with its messages intact, and they solve overlapping but different problems. A persistent session asks the remote broker to hold state on the client’s behalf; a local broker with a bridge holds that state on the device itself. Choosing between them decides where the queue lives, who pays for it, and what happens when the outage is longer than anyone planned for. This guide works through both for a field gateway, inside message queue management at the edge and the Bandwidth & Async Sync Optimization guide.

What a persistent session actually promises #

Connecting with cleanStart=false and a non-zero session expiry asks the broker to remember the client: its subscriptions, its unacknowledged QoS 1 and 2 messages in both directions, and its message ordering. On reconnection the broker replays what the client missed and the client replays what the broker did not acknowledge.

Three limits matter on a field device, and all three are properties of the broker rather than the client.

The session expiry interval bounds how long the broker holds state after the client disappears. Set it to an hour and a device that is offline for a day comes back to a clean session and a silently discarded backlog. Brokers also impose a maximum, frequently far below what a field deployment needs.

The inflight window bounds how many unacknowledged messages the client may have outstanding — commonly 20 by default. It is a flow-control mechanism, and on a link with a 700 ms round trip it caps throughput at roughly 28 messages a second regardless of bandwidth.

The broker’s queue limit per session bounds how much it will hold. Exceed it and messages are dropped at the broker, usually silently from the client’s point of view.

Configuring a persistent session honestly #

Where the deployment can rely on the remote broker’s state — short outages, a broker you control, generous session limits — the configuration is short and every parameter matters.

# mqtt_session.py — persistent-session client sized for a field link.
# The client library's inflight window is the real throughput limit, not the
# link's bandwidth. Threading: paho runs its own network thread; publish from
# one task only, or serialise with a lock.
import paho.mqtt.client as mqtt

SESSION_EXPIRY_S = 7 * 24 * 3600      # a week — must not exceed the broker's max
KEEPALIVE_S = 120                     # long enough not to churn a sleepy modem
INFLIGHT = 40                         # raise from the default 20 on a slow link


def build_client(client_id: str, tls_ctx) -> mqtt.Client:
    c = mqtt.Client(client_id=client_id, protocol=mqtt.MQTTv5,
                    clean_session=None)             # v5: use session expiry
    c.tls_set_context(tls_ctx)
    # Inflight is the flow-control window. On a 700 ms RTT link, 20 in flight
    # at QoS 1 caps throughput near 28 msg/s regardless of bandwidth.
    c.max_inflight_messages_set(INFLIGHT)
    # Queue on the client while disconnected. Bounded: unbounded here is an
    # out-of-memory kill during a long outage.
    c.max_queued_messages_set(2000)
    c.reconnect_delay_set(min_delay=1, max_delay=120)
    return c


def connect(c: mqtt.Client, host: str, port: int = 8883):
    props = mqtt.Properties(mqtt.PacketTypes.CONNECT)
    props.SessionExpiryInterval = SESSION_EXPIRY_S
    # cleanStart False asks the broker to resume our session if it still has it.
    c.connect(host, port, keepalive=KEEPALIVE_S,
              clean_start=False, properties=props)


def on_connect(client, userdata, flags, reason_code, properties):
    """`flags.session_present` is the field that matters: False means the
    broker did NOT have our session, so anything it was holding is gone and
    anything we assumed it acknowledged may not have arrived."""
    if not flags.session_present:
        userdata["session_lost"] += 1
        userdata["resync_required"] = True

session_present is the single most important thing this code reads. A client that reconnects and does not check it will silently assume continuity that the broker did not provide — and the messages the broker discarded are exactly the ones from the outage, which are the ones that mattered.

When the local broker wins #

Three conditions push a deployment toward running a broker on the device.

Outages longer than the session expiry. A device offline for a week against a broker with a four-hour session limit has no continuity at all. Local persistence does not care.

Multiple local publishers. A gateway aggregating several sensor processes benefits from a local broker as a local bus, independent of any uplink. The bridge then becomes one concern rather than N.

Backlogs larger than RAM. A week of telemetry is megabytes, and a client-library queue holding it in memory is an out-of-memory kill waiting for the right outage. A local broker writes to flash.

The costs are real. A local broker is another process, another 8–20 MB of resident memory, another thing to configure, secure and update. And the bridge introduces a second hop where messages can be reordered or duplicated — which is fine for telemetry with an idempotency key and not fine for anything assuming exactly-once ordering end to end.

The third option nobody names #

There is an arrangement that beats both for a telemetry gateway and it is worth stating explicitly: publish into your own spool and treat MQTT as a transport rather than a queue.

The spool described in store-and-forward buffering already provides durability, bounded growth, per-class retention policies and an explicit cursor. MQTT then carries batches from that spool with QoS 1, and the acknowledgement advances the cursor. The client needs no persistent session and no local broker; if it reconnects with a clean session, nothing is lost, because the spool never depended on the broker’s memory.

That arrangement gives the local broker’s durability without its resident memory, gives the persistent session’s simplicity without depending on the broker operator’s expiry policy, and puts the retention policy where the deployment can actually reason about it. Its cost is that the device does not get MQTT’s local bus for free, so a gateway with several publishing processes still needs something to aggregate them.

Constraint validation #

Gotchas and edge cases #

• session_present false is an event, not a detail. Count it, export it, and treat a rise as a broker-side configuration change. It is the only signal that the continuity you designed for is not being delivered.
• Keepalive interacts with modem power saving. A short keepalive keeps the radio awake and destroys a battery node’s power budget; a long one delays detection of a dead link. On a duty-cycled node, connect, drain, disconnect cleanly, and let the spool hold everything between windows.
• QoS 1 duplicates are guaranteed, not exceptional. Any reconnection with unacknowledged messages re-delivers them. The idempotency key is not optional at any quality of service below 2, and QoS 2’s cost is rarely worth avoiding a deduplication the platform needs anyway.
• Bridges can loop. A local broker bridged bidirectionally to a remote one, with overlapping topic filters, will happily forward a message back to itself forever. Use directional bridges with explicit topic prefixes, and test with a message that would loop.
• Client-side queueing is not durable. max_queued_messages_set holds messages in memory. A restart discards them, which makes it a burst absorber rather than an outage store — a distinction worth being precise about when choosing a size.

Deciding #

Ask one question first: how long must the device survive without the link? If the answer is under an hour and the broker’s session expiry comfortably exceeds it, a persistent session is the simplest thing that works and adds nothing to the image. If the answer is a day or more, broker-side state is not a plan, and the choice is between a local broker and a spool. Choose the spool unless the device genuinely needs a local message bus for several publishers — because the spool is smaller, its retention policy is yours, and it is the same mechanism the rest of the pipeline already depends on.

Migrating between them #

Fleets change their minds about this, usually after the first outage longer than the session expiry, and the migration is more manageable than it looks because the two arrangements can coexist.

Publish through a small abstraction that hides which is in use — one publish(topic, payload, qos) call — and switch its implementation by configuration. A device running the new arrangement and one running the old produce identical output on the wire, so the platform sees no change and the rollout can proceed ring by ring like any other.

The one migration hazard is the backlog in flight at the moment of the switch. A device that changes from a persistent session to a spool while holding unacknowledged messages will either lose them, if the session state is discarded, or duplicate them, if the broker later replays what the spool has already re-sent. Drain before switching: hold the update until the client reports zero inflight and the spool is empty, which on a healthy link is a matter of seconds and on an unhealthy one is a reason to wait.

Related #

• Message Queue Management at the Edge — the broker selection and queue architecture this sits inside.
• Configuring MQTT QoS levels for telemetry drops — the per-payload quality decision that interacts with the inflight window.
• Store-and-Forward Buffering — the third option, and the one most field gateways should reach for.