Why automation retries create duplicates and how to prevent them

Your system sends a request to create a task. The destination saves it, but the reply is interrupted. Retrying the same request can create a second task. The practical question is whether the first action happened.

Give the intended action a stable identity

Keep the source record ID and destination record ID together. For a proposed enquiry-to-task workflow, a key could combine the source system, enquiry ID and action type. Use a new version only when your update rules call for a distinct action. An email address alone is usually a poor identity: one customer can submit several legitimate enquiries.

Some APIs support idempotency keys or unique constraints. Others require a lookup and a durable record of completed actions. Check the actual destination’s guarantees; a search followed by a write can still race if two executions run together.

Use a three-way recovery decision

“No response” belongs in the third row until evidence resolves it. A timeout is a transport observation, not a business-state conclusion. For customer-facing actions with weak lookup support, retain an explicit review state instead of blindly repeating.

Read the text version

1. Receive a stable ID. Keep the same ID on every retry.
2. Look up the destination. Does the corresponding record exist?
3. Choose one route. Reuse the record or create it once.
4. Verify and record. Retain the destination ID and outcome.

Decide what a changed record means

A repeat event and a changed source record are different. If a customer corrects a phone number, your policy may update the original task. If they send a new enquiry, your policy may create a new task. Write that distinction before choosing a duplicate-removal feature.

Asana documents webhook event handling. A workflow responding to those events still needs its own policy for repeated delivery, record changes and execution history. Event delivery alone does not provide end-to-end exactly-once execution.

Test interruption and concurrent execution

1. Send the same source event twice.
2. Send a legitimate second enquiry from the same email.
3. Update the first source record.
4. In a controlled environment, run two handlers for the same key.
5. Simulate missing confirmation and inspect the destination before recovery.

Record how many destination records exist and why. Confirmed duplicates should be reconciled with care; do not delete records automatically without an agreed recovery policy.

If your software lacks stable IDs or lookup support, treat that as a design constraint. It may justify a custom adapter, a review queue or a simpler manual process.

Why “check, then create” can still create two records

Consider two workers receiving the same hypothetical event E-27 almost simultaneously. Both look for an existing task. Both see none. Both create one. The lookup was reasonable, but the separate lookup and write did not protect against concurrency. A delay between them does not establish correctness either.

The implementation needs a supported way to claim the intended action atomically, a destination uniqueness rule, or an API idempotency facility with understood scope. Which option is possible depends on the actual systems. A spreadsheet flag checked by two independent runs is not automatically a lock. Ask the implementer to demonstrate the simultaneous-event case rather than only replaying events one at a time.

Keep event identity and action identity separate

A source can send different events about the same record: created, edited and reassigned. Deduplicating only the delivery event prevents one repeated delivery, but may still create a new task for every edit. Conversely, treating every event for one customer as identical may suppress legitimate new enquiries.

For an illustrative design, identify the intended action by source enquiry E-27 plus the operation “create follow-up task”. Later corrections update that task under a separate version rule. Store the destination ID and current state: pending, confirmed, failed before write or unknown. A history of these decisions is more informative than a single “processed” checkbox.

Treat API idempotency as a documented contract

Some APIs accept an idempotency key. Stripe documents one such mechanism, but its retention and retry behaviour are specific to that API and version. Do not assume a header with the same name works on every destination, persists indefinitely or permits changing the original request parameters.

Read the destination’s supported behaviour, keep the key free of personal data and verify its lifetime against the recovery window. If the destination has no such facility, use an alternative supported design and document remaining limitations. After an uncertain write, the recovery question is still whether the intended effect already exists.

Test the same event twice, two concurrent deliveries, an edited source, a lost confirmation and a retry after the key-retention window. A design that passes only the first test can look reliable in a demo while still producing duplicates in production.

Technical reference: Stripe’s API-specific idempotency contract.

Take one practical next step

If this is a problem in your team, explore custom development. Describe one task, the software you use and the result you need. Scope and terms are agreed before work begins.

Sources and further reading

• Asana: webhook documentation

About the author

Martin Iliev

Martin builds Set It Once to help people spend less time on repeated work. His guides explain what you can try, how to check the result and where the limits are.

More about Martin

All articles