Practical MQTT patterns

Durable work queues over MQTT

Use MQTT topics as a leased work queue with acknowledgements, retries, timeouts, and dead-letter handling for edge workers.

Publish/subscribe normally delivers a matching MQTT message to every subscriber. Work queues need a different contract: one available worker receives a job, the application reports the outcome, failed work is retried, and repeatedly failing work is moved aside for inspection.

Sagüin's queue channel adds that contract to a configured region of the MQTT topic space while keeping MQTT 5 and MQTT 3.1.1 clients on the wire.

Why a shared subscription is not the whole queue

An MQTT shared subscription distributes live messages across group members. It is useful for load balancing, but application work often needs more than distribution:

  • a job must survive when no worker is connected;
  • accepting the MQTT delivery and completing the business operation are separate events;
  • a worker can disconnect or time out after receiving work;
  • attempts need to be counted;
  • exhausted work needs a dead-letter destination;
  • an operator needs to see why a job stopped retrying.

A durable queue records those states explicitly. Sagüin describes a delivery as a lease, not an execution. If the lease expires, the job can be offered again while the first worker may still be running. Workers therefore have to be idempotent.

Configure a queue-shaped topic range

A queue channel claims the topics matched by its filter:

channels:
  device-work:
    type: queue
    filter: fleet/+/work/+

A publisher can send a job to a normal topic such as fleet/truck-17/work/reindex. Sagüin persists the job according to the channel's storage provider before reporting the corresponding durable acceptance.

The full configuration surface includes bounds, lease timing, retry behaviour, and storage selection. Validate the actual configuration with saguin --check-config before deployment.

The lifecycle of one job

The queue's useful unit is the application outcome:

  1. A publisher writes a job to a queue-owned topic.
  2. The broker persists it and makes it available.
  3. One eligible worker receives a lease.
  4. The worker performs the external operation.
  5. The worker resolves the lease with the application result.
  6. Successful work leaves the queue; returned or expired work becomes available again.
  7. Work that reaches its attempt limit moves to the associated dead-letter channel.

The dead-letter move is part of the queue's storage transaction. A queue and its dead-letter channel therefore live in the same provider.

Design workers for at-least-once execution

No broker can prove that an external side effect completed merely because a connection disappeared. A worker might update a database and then lose its link before reporting success. The lease may later expire and the job may be delivered again.

Use an idempotency key carried in the payload or MQTT properties, then make the downstream operation safe to repeat. Common patterns include:

  • a database uniqueness constraint on the job identifier;
  • an upsert instead of an unconditional insert;
  • recording completed commands before acknowledging them;
  • making the target operation converge on a desired state.

Exactly-once language tends to hide this boundary. Sagüin states it directly: queue processing is at least once.

Backpressure is a feature

A bounded queue is safer than an apparently infinite one. Configure limits that fit the disk and the operational response time. When durable storage cannot accept another promised record, the broker refuses the publish instead of acknowledging data it cannot keep.

Watch queue depth, lease expiry, retries, and dead-letter growth. A rising queue is usually a capacity or downstream-health signal, not something to solve by making every bound unbounded.

When to use another channel

Use append when every durable consumer should see every record. Use latest when only the current value of each topic matters. Use ordinary broadcast or MQTT shared subscriptions when live delivery is enough and an application-level lease lifecycle would add machinery without value.

Read RFC 0003: Delivery semantics for the exact lease and resolution rules, and RFC 0004: Storage for transaction and recovery guarantees. The Sagüin examples provide runnable clients against a real broker.