Engineering Practice
Before Shipping a New Integration, Test Timeouts, Retries, and Duplicate Requests
An executed local SQLite print-job model loses its response after commit, replays the original job with a stable scoped key, rejects changed payloads, and demonstrates duplicate creation after changed identity or receipt pruning. A virtual three-attempt budget separates deadline arithmetic from untested real transport behavior.
Before shipping a new integration, test what happens when the receiver commits an operation but the caller never receives its response. Keep the logical operation’s identity stable across retries, verify the receiver rejects conflicting reuse, and count side effects rather than successful responses alone. Set separate limits for total attempts and overall elapsed time. A timeout tells the caller it stopped waiting; it does not establish that nothing happened.
This LandOffer.ai guide includes a complete synthetic print-job sequence, executed local SQLite checks, and a virtual retry budget. No HTTP service, payment provider, printer, or production integration was called. Response loss is deliberately injected after a local transaction commits, so the evidence demonstrates an uncertainty boundary rather than a real network timeout or a claim of exactly-once delivery.

Define the operation before choosing a retry policy
Name the user’s intended one-time effect before configuring retries. In the fictional PrintParcel integration, a design studio asks a partner service to create one print job for a synthetic poster artifact. In this fixture, the counted side effect is an inserted job row. Actually printing paper, dispatching a courier, or charging an account is outside the local fixture.
One logical request can produce several network attempts. Preserve a stable operation key across those attempts and retain enough evidence to reconcile an uncertain result later. Generating a new key every time the caller times out turns retries into independently identified operations, undermining the receiver’s ability to recognize them.
RFC 9110’s idempotent-method section defines idempotence in terms of the intended server effect of repeated identical requests. It also cautions against automatically retrying non-idempotent requests without knowing their semantics make that safe or knowing the first request was not applied. Naming an HTTP method or adding a header does not create that contract in an arbitrary service.
Record the receiver’s actual rules: the key’s scope, which request parameters are compared, how concurrent requests behave, how long records are retained, and which outcomes are replayed. Treat those as version-specific API behavior rather than borrowing rules from an unrelated vendor.
Stripe’s current idempotency documentation illustrates why that distinction matters. It describes API v1 saving an executed request’s status and body, including server errors, while API v2 has different retry-completion behavior. Its API v1 discussion also describes parameter comparison and key pruning. This article does not call Stripe or assume PrintParcel implements either API’s behavior.
Worked sequence: the job exists after the response disappears
The local receiver has a jobs table and a receipt table. A receipt is scoped by account, route, and operation key. Its stored request fingerprint covers the fixture’s artifact identifier and copy count; the receipt also records the created job ID. A matching replay returns that ID. A different payload under the same scoped key raises a conflict.
Both the job insertion and receipt insertion occur inside one SQLite write transaction. This is a deliberately small, local atomic boundary. It does not include a separate printing system, queue, email service, or remote database. Keeping that boundary visible prevents a successful database example from becoming a promise about external side effects.
Here is the first completed sequence record, executed with the synthetic account studio-a, key op-k17, artifact SYN-POSTER-17, and one copy. The short readable key is for teaching, not a production key-generation scheme.
| Step | Caller’s view | Receiver’s actual local state | Required interpretation |
|---|---|---|---|
| Submit operation | Caller requests one job with op-k17 |
Receiver creates job 1 and its receipt in one transaction | Intended local effect exists |
| Commit, then lose response | Injected TimeoutError reaches caller |
Job 1 and receipt remain committed | Caller has an unknown outcome, not confirmed failure |
| Repeat same operation | Caller uses op-k17 and unchanged payload |
Receiver finds the matching receipt and returns job 1 | Retry does not add another job row |
| Inspect effects | Caller receives the original job ID | Exactly one job row exists | Count the effect, not merely the retry response |
The unprotected comparison performs two creation attempts without a shared operation receipt and produces two job rows. In the protected model, the post-commit loss followed by the same-key replay produces one. The retained source and state snapshots show the difference: repeating a creation is not the same as replaying its recorded result.
Do not display a definitive “nothing was created” message after an ambiguous timeout. The product needs an uncertainty state and a recovery path consistent with the real receiver’s capabilities. If a user presses a retry button, that action should not silently discard the original operation identity.
If the receiver supports looking up an operation, define how the caller reconciles an unknown result. If no such capability exists and safe replay is unsupported, stop automatic resubmission and escalate with the original reference. A status lookup is a protocol feature to verify, not something an integration can assume every API offers.
Test key reuse, conflicts, persistence, and concurrency
The second completed asset is the executed boundary matrix. Python 3.12.14 runs a disposable SQLite database; each protected submission opens a new connection. The fixture uses a transaction that acquires the write lock before checking the receipt. Temporary databases are removed after the retained results and snapshots are written.
| Test | Executed local result | What the result establishes |
|---|---|---|
| Lost response after commit, then same-key replay | One job; replay returns job 1 | The local receipt survives caller uncertainty |
| Same key with changed copy count | Conflict; job count stays one | The model does not silently reuse a result for different parameters |
| New connection with original key and payload | Original job 1 returned | Receipt state is persisted in SQLite, not only a process dictionary |
| New key with otherwise identical payload | Job 2 created | Changing identity can create a second operation |
| Two local threads, same key and payload | Both return job 1; one job exists | SQLite serialization protects this local transaction boundary |
| Deliberately delete the receipt, then reuse the key | Another job is created | Protection ends when the matching receipt no longer exists |
| Invalid zero-copy request | Validation error; no job inserted | This invalid input has no modeled side effect |
Twenty-one checks cover this matrix and the retry-budget arithmetic below. Each expected result is stated independently of a successful response. Conflict tests inspect the unchanged job count; replay tests inspect both the returned identity and the number of jobs. A test that only checks for a non-error response would miss the changed-key duplicate.
The concurrency receipt covers two local callers serialized by SQLite, not workers in a distributed service. Two local threads sharing one SQLite file are not a distributed deployment under load. A real receiver must coordinate its own workers and durable state. An application-level “look up, then insert” sequence without the necessary transaction or uniqueness protection can race even when sequential tests pass.
The receipt-pruning case is a deliberate deletion in the fixture, not a tested retention scheduler. It demonstrates why a client must understand the receiver’s retention window. A retry arriving after deduplication evidence expires may behave like a new operation. Choose recovery rules for delayed workers and long-lived user actions rather than treating a key as permanent protection.
Bound elapsed time as well as the number of attempts
Amazon’s timeouts, retries, and jitter guidance explains that retrying can increase load on an already struggling dependency and can repeat side effects. Backoff spaces attempts; jitter helps avoid synchronized bursts. Those controls solve a different problem from ensuring repeated requests have the same intended effect.
For the fictional caller, choose an illustrative three-second overall budget, at most three total attempts, and an 800-millisecond allowance per attempt. These values are teaching inputs, not a recommendation for an unmeasured API. The two example waits are injected values within 200- and 400-millisecond backoff caps; no random sampler or network client is exercised.
| Virtual action | Start | End | Running elapsed time |
|---|---|---|---|
| Attempt 1 | 0 ms | 800 ms | 800 ms |
| Example wait 1 | 800 ms | 900 ms | 900 ms |
| Attempt 2 | 900 ms | 1,700 ms | 1,700 ms |
| Example wait 2 | 1,700 ms | 2,000 ms | 2,000 ms |
| Attempt 3 | 2,000 ms | 2,800 ms | 2,800 ms |
| Stop | 2,800 ms | No fourth attempt | Three attempts used; outcome can still be unknown |
The virtual calculation includes waits instead of granting each retry a new overall deadline. It also checks a constrained scenario: if the elapsed pause after attempt 2 is 600 milliseconds, the caller reaches 2,300 milliseconds. Another full 800-millisecond attempt would exceed the budget, so it must not start under this policy. That pause models elapsed delay, not a valid draw from the earlier 400-millisecond jitter cap.
In a real client, use an appropriate elapsed-time clock and verify what its timeout covers. Connection setup, name resolution, TLS, response headers, and body reads may have different controls. Passing this arithmetic test does not establish that a particular library enforces an end-to-end deadline or cancels work already executing at the receiver.
Classify the response before scheduling another attempt. A validation failure needs a corrected request, while a timeout after an unknown commit needs reconciliation or a safe replay contract. A parameter conflict must not trigger a loop that keeps changing keys until something succeeds. For service errors, follow the receiver’s documented rules rather than treating every server error as permission to repeat an operation.
Keep cancellation separate from confirmation. The caller exhausting its budget does not demonstrate that server-side work was cancelled. If a background worker continues, the user-facing state and later reconciliation must account for that possibility. A test should inspect receiver state after the caller stops, not end as soon as a timeout exception appears.
Choose the layer that owns retrying. If a user-facing client, service wrapper, and queue worker each retry independently, downstream traffic can multiply. Record total attempts across those layers and retain the same operation identity where the protocol requires it. Backoff alone does not cap an unbounded retry loop, and an attempt cap alone does not prevent synchronized traffic.

Finish with an evidence-based shipping decision
The fictional integration review records: “The local receiver model replays a committed job after injected response loss and rejects a changed payload under the same key. The virtual caller stops within its stated budget. Do not release the real integration yet: transport timeout behavior, receiver retention rules, production concurrency, and downstream printing effects remain unverified.” This is an authored decision record, not a real deployment approval or message.
Retain a failing control alongside the passing replay case. The unprotected duplicate and the changed-key test show what the proposed protection actually prevents, while the pruning case shows where it ends. That comparison makes the review easier to challenge than a test pack containing only successful responses. Do not discard an inconvenient case because it complicates the shipping summary.
The integration owner must connect those remaining checks to the actual service contract and an authorized test environment. The operations owner needs a way to identify unresolved requests and reconcile them without blind resubmission. Preserve the logical operation ID, attempt number, receiver reference, and result classification while avoiding sensitive payloads in routine logs.
A successful repeat response is only part of the evidence. Look for repeated database rows, queued work, downstream commands, and any external effect that matters to the user. If the job record and external action cannot commit atomically, document the recovery design and test that boundary separately; this local receipt table does not establish recovery for those external effects.
For your next integration, inject one failure after a meaningful commit, retry with stable identity, and inspect the effect count. Then test conflicts and deadline exhaustion against the receiver’s documented replay contract. More practical engineering guides are available from LandOffer.ai.



