A reliable webhook integration handles repeated, delayed and missing events without repeating the business action. Verify the sender, persist the event durably, process it with duplicate protection and reconcile the resulting records against the source system. Receiving an HTTP request is only the beginning of the workflow; the real acceptance test is whether the correct business state survives interruptions.
Let’s plan your webhook recovery plan.
Bring your workflow, constraints and open questions. We’ll help you define the scope and work through the next decision.
Separate delivery from completion
Consider a hypothetical order system receiving a payment notification. There are at least three outcomes to track: the notification arrived, its processing finished, and the order now reflects the verified payment. A successful delivery response does not prove the last two.
Give the event record a provider identifier, received time, processing state, attempt count and related business record. Store only the payload and personal information needed for the purpose, with an explicit retention policy. Operators should be able to distinguish an event waiting for a worker from an event that failed permanently.
Stripe's webhook documentation describes signature verification and warns that delivery order is not guaranteed and duplicate deliveries can occur. Check the equivalent contract for each provider; do not assume all senders share Stripe's retry behaviour.
Agree a failure and recovery matrix
| Scenario | Expected behaviour | Evidence for acceptance |
|---|---|---|
| Same event delivered twice | One effective business transition | Duplicate recorded without another fulfilment |
| Older event arrives later | No invalid state reversal | Current state and transition rule inspected |
| Worker stops after receipt | Pending work remains recoverable | Restart completes the stored event |
| Downstream request times out | Outcome checked before repeating | Provider reference and reconciliation result |
| Invalid signature | No trusted business action | Rejection visible without exposing secrets |
| Event never arrives | Reconciliation detects the discrepancy | Exception assigned to an owner |
Run these scenarios with authorised test data before release. Keep the provider's delivery identifier separate from the business operation identifier: two different events can sometimes describe the same underlying change.
Put duplicate protection at the business boundary
Checking whether an event was seen is useful, but insufficient if a worker crashes between changing the order and marking the event complete. Agree how the write and its processing record become consistent. Database transactions or a recoverable operation ledger may be appropriate, depending on the system boundary.
For external actions, first determine whether the provider supports an idempotency key and how long it retains the key. See the separate guide to idempotency and repeated operations for that narrower design problem.
Make replay an accountable operation
An operator replay should show the intended target, previous attempts and likely side effects. Require the relevant permission, retain an audit record and route ambiguous outcomes for investigation. A button labelled Retry all can multiply mistakes if processing is not safe to repeat.
Schedule reconciliation according to the business deadline and provider capabilities. Compare eligible source records with destination outcomes, not just event counts. Start with a small sample to verify matching rules before running a broad repair.
Questions buyers should ask
Can webhooks guarantee that nothing is lost?
An endpoint alone cannot establish that. Ask for durable processing, provider-specific delivery limits, reconciliation coverage and a demonstrated recovery procedure.
Should every failed event retry automatically?
No. Temporary connection failures may justify bounded retries. Invalid credentials, incompatible data and uncertain financial outcomes need different handling.
Bring the event types, affected records and failure deadlines to an API integration discussion. Those details define the work more usefully than the number of webhook endpoints.
This article explains what to build, not legal advice. Rules change; confirm against the current official text before relying on it.
