Your receiver returned too slowly, the connection closed, and the same event arrived again. That is expected under at-least-once delivery. The bug begins only if the second POST sends a second alert, starts a second model run, or repeats another external effect.
Deduplication needs durable state. An in-memory set disappears in exactly the restart that is most likely to trigger a retry.
Use the delivery id, not the package id
RelayLink places the same UUID in:
X-RelayLink-Delivery- The JSON body's
delivery_id
That UUID identifies one queued delivery to one receiver. It stays stable across attempts. Use it as the unique key in your receiver's idempotency store.
Do not deduplicate on package_id. One package may legitimately produce separate deliveries to multiple receivers, and your service may need to record each one. Do not use the signature either: each attempt gets an attempt timestamp, and the signature covers that timestamp, so a retry can carry a different signature.
After verification, require the header delivery id and body delivery id to match.
Separate receipt from processing
A reliable receiver first records that it owns the event, then processes it outside the HTTP request. A durable inbox row might contain:
- Delivery id as a unique key
- Event name
- A digest of the exact body bytes
- Receipt time
- State such as pending, processing, or completed
- Last processing error
The callback path is:
- Verify the signature and timestamp.
- Parse the verified body and require its event and delivery id to match the headers.
- Compute a digest over the exact body bytes.
- In one transaction, insert the inbox row and a work-queue row together.
- Commit and return 2xx.
If the unique insert reports that the delivery already exists, compare the stored digest. The same digest means the event was already accepted: return 2xx without adding another work item. A different digest under the same id violates the wire contract; quarantine it rather than processing or silently merging it.
Putting the inbox row and queue item in one commit is the useful part. A row that says “seen” without any work behind it is only a durable way to lose the event.
The mark-before failure window
The simplest mark-before algorithm is:
insert delivery id
perform work
It blocks duplicates, but a crash between those lines leaves a permanent idempotency record and no completed work. Every retry sees the record and exits successfully. The event is lost from the application's point of view.
A pending inbox plus durable queue closes that window. The queue worker can resume after a process restart. The receipt means “accepted for processing,” not “effect completed.”
The mark-after failure window
The opposite algorithm is:
perform work
insert delivery id
It avoids marking unfinished work as complete, but a crash after the effect and before the insert lets the retry perform the effect again. That is dangerous for sends, charges, deployments, or any action that is not naturally repeatable.
Moving the final insert closer to the effect reduces the window but does not remove it when the effect belongs to another system. Use the delivery id as an idempotency key in that downstream call when possible. Otherwise record an intent first, make a reconciler inspect uncertain outcomes, or design the effect so repeating it reaches the same state.
Byte-identical retries help diagnosis
RelayLink serializes the body once when it creates the delivery. A retry uses the same delivery id and byte-identical body. The timestamp and signature headers may differ because they are generated for each attempt.
Keeping the digest avoids putting sender metadata or a raw body in routine logs while still detecting a contract violation. Do not log the raw body, signing secret, or full receiver URL; a webhook path may itself contain a token.
Always verify each retry's own signature before looking up the idempotency record. Otherwise anyone who learns a delivery id could make your endpoint return success to an unauthenticated request.
Make the MCP fetch part of the worker
A package.received callback contains no briefing. The queued worker authenticates to the supplied mcp_url and calls get_package with package_id.
Keep agent execution after that authenticated fetch, not in the callback. Mark the delivery completed only when your intended workflow has reached its defined completion point. If processing fails, retry it from your own queue without asking RelayLink to repeat the HTTP delivery.
This gives each layer one job: RelayLink retries transport, the receiver deduplicates receipt, and the worker owns processing recovery.