MQTT message replay after an offline client reconnects

MQTT can preserve a client session between connections, but a persistent session is not the same thing as an independently replayable event history. The distinction matters when an edge device, gateway, or downstream service can be offline long enough to exhaust a bounded session queue.

Sagüin adds an append channel to ordinary MQTT topics. The channel stores each accepted record once and keeps a position for every durable consumer. A returning consumer resumes from that position instead of requiring a private copy of the complete backlog.

Session recovery and replay solve different problems

A persistent MQTT session remembers subscriptions and unacknowledged QoS 1 or QoS 2 deliveries. That is the right tool for a temporary network interruption. It does not create an application-controlled event log with an independent cursor for every reader.

An append channel is useful when the history itself matters:

• telemetry becomes evidence for billing, maintenance, or safety;
• a consumer may be offline for days rather than seconds;
• a second consumer needs to read the same records at a different pace;
• an operator must deliberately seek a consumer to an earlier point;
• retention should be defined for the stream rather than by each client's private queue.

Sagüin stores a record once, then stores the next unread position for each durable consumer. Ten consumers do not require ten copies of every retained record.

Configure a replayable MQTT topic range

An append channel claims an ordinary MQTT topic filter:

channels:
  fleet-telemetry:
    type: append
    filter: fleet/+/telemetry/#

Publishers continue to use normal MQTT topics such as fleet/truck-17/telemetry/temperature. Subscribers use normal MQTT subscriptions. The storage and replay behaviour comes from the channel that owns that part of the topic space, not from a new wire protocol.

Topics outside configured channels remain ordinary live MQTT broadcast topics.

What happens during an outage

Consider a service consuming fleet/+/telemetry/#:

1. The service connects with a stable client identifier and a session that outlives the connection.
2. It consumes records and acknowledges them.
3. Sagüin advances its stored position as acknowledgements are committed.
4. The service loses its network connection while publishers continue writing records.
5. On reconnect, the service resumes from its stored position.

Delivery is at least once. A crash can cause a record to be delivered again, so the consuming application must be idempotent. Sagüin does not claim exactly-once processing.

Retention still has a boundary

A replayable log is deliberately bounded. Time-based, record-count, or byte-based retention may remove older records. If retention advances past a consumer's stored position, Sagüin does not silently skip the missing interval and pretend the history is complete. The consumer is told that its position is below the retention floor and must make an explicit recovery decision.

That failure mode is important: replay is useful only if a successful read means the history was not quietly shortened.

When not to use an append channel

Use ordinary MQTT broadcast when only currently connected subscribers matter. Use a retained message or Sagüin latest channel when a subscriber needs the current value rather than every transition. Use a queue channel when each record should be processed by one worker rather than by every subscriber.

The four behaviours are complementary:

Verify the contract

The detailed rules for positions, acknowledgements, reconnects, retention floors, and failure handling are in RFC 0003: Delivery semantics. Storage guarantees for memory and SQLite providers are in RFC 0004: Storage.

For a hands-on check, clone the Sagüin repository and run make demo. The guided demo starts its own broker and walks through live broadcast, replay, latest-value state, and queued work.