Engineering Practice
An API Rate-Limits Your Integration: Budget Requests Across Workers
Allocate a shared request allowance across three workers and inspect a complete offline throttle-and-recovery trace. Separate quota scope, request attempts, shared pauses and controlled probes from untested distributed coordination and unfinished work.
When an API limits an integration, coordinate the workers that share its quota. Give them one request budget, one dispatch policy and one shared pause state. Count network attempts, including retries, rather than treating each worker’s remaining allowance as independent. A longer retry delay inside one process will not control the other processes still sending requests.
This LandOffer.ai workplace guide uses a fictional read-only inspection-status integration and an actual offline Python model with authored responses and a fake clock. The model checks budgeting and pause decisions; it makes no API calls and does not test distributed synchronization, a provider’s enforcement or a production recovery.

Identify the quota boundary before setting worker limits
Start with the provider’s documentation and the response that stopped the integration. Record the endpoint family, authentication scope, status, relevant headers, response time and the operation’s purpose. Keep credentials out of diagnostic records; retain a redacted account-scope identifier and resource family so another engineer can match the affected bucket. A limit may attach to a user, installation, account, IP address or particular resource; a worker name alone does not identify the quota it consumes.
RFC 6585 defines 429 as too many requests within a period and allows a Retry-After header, but leaves user identification and request-counting policy to the server. It does not provide a universal requests-per-minute allowance. A missing 429 does not establish absence of throttling. Conversely, a 403 can have other causes; inspect the provider’s documented response body and headers before classifying it as a quota failure.
GitHub provides a useful named example. Its REST API rate-limit documentation distinguishes primary allowances from secondary limits and documents different authentication and resource scopes. Secondary restrictions can apply even when a primary counter remains positive. Its response headers include the resource family, remaining allowance and reset time; those values guide pacing rather than guaranteeing an exact future dispatch capacity.
In the fictional integration, Kiran confirms that workers A, B and C share one vendor account and one documented read-request bucket. A separate write workflow is excluded. Each selected GET costs one unit under the invented contract. These assumptions belong to this case; they are not GitHub’s limits or a claim that all APIs charge requests equally.
The team also inventories other consumers of the same account. A header is an observation, not a reservation against calls another integration might send next. Without coordination across those consumers, the local estimate can become stale. Kiran retains provider responses as evidence and keeps the dispatcher able to pause even when its own arithmetic predicts spare capacity.
Worked budget: three workers share twenty dispatch slots
At the planning snapshot, the authored response reports 24 requests remaining. Operations owner Lou reserves four as a local safety margin and authorizes twenty planned attempts. The reserve is not a provider entitlement or a pool workers may independently spend on retries. A later response can reduce the usable allowance further.
The queue contains 24 distinct GET requests. Each is one page or resource request, not an entire multi-page business job. Kiran has already expanded the work into these request units; otherwise pagination could quietly multiply the budget.
| Work lane | Requests awaiting dispatch | Planned slots | Remains queued | Accepted service choice |
|---|---|---|---|---|
| A: urgent status refresh | 8 | 8 | 0 | Lou prioritizes these requests |
| B: current-project refresh | 6 | 6 | 0 | Retain current work after lane A |
| C: historical reconciliation | 10 | 6 | 4 | Defer the final four reads |
| Combined | 24 | 20 | 4 | Keep four provider units unallocated |
In the normal, non-throttled model run, the central dispatcher admits twenty attempts, three fake-clock seconds apart, with at most one outstanding request. It retains four units and refuses the next historical request. If three workers each copied the same twenty-slot allowance, they could collectively reserve sixty attempts against the same observed 24 remaining. All sixty reservations would compete for the same observed allowance.
The slot calculation tells Lou what may enter the queue’s dispatch stage, not how many jobs will finish. A failed attempt still occupies its reserved dispatch unit in this conservative model. Retrying it requires another available unit and passes through the same queue. A timeout does not automatically refund capacity: the server may have received the request even if the client never received a response. Provider-specific accounting needs separate confirmation.
Lou accepts the four-request historical deferral and keeps its work IDs visible in the queue. Kiran records which lane owns each request, its attempt count and when it becomes eligible again. Deferred work is not labeled successful, and it is not silently discarded when the current reporting period ends.
Concurrency is a separate control. Twenty requests in a window does not mean twenty may start simultaneously. The three-second spacing and one-outstanding-request rule are deliberately conservative local choices for this fixture, not published universal limits. Real response latency would also affect throughput; the model’s instantaneous authored replies do not measure it.
Make a throttle response pause the shared scope
GitHub’s REST API best practices recommends serial requests through a queue to avoid secondary limits. It tells clients to respect Retry-After, wait for a primary reset when remaining is zero, and use increasing waits with a retry ceiling for continuing secondary-limit failures. Apply the relevant provider’s guidance to the shared quota, not just the worker that happened to receive the response.
The second fixture run injects a throttle after three successes. Its fake-clock origin is October 27, 2026 at 12:00:00 UTC. At second 9, A’s request A2 receives this authored response:
HTTP/1.1 429 Too Many Requests
Retry-After: 30
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1793102445
The delay reaches second 39; the reset timestamp reaches second 45. Kiran’s example policy waits for both constraints, then adds one second of local safety, making second 46 the earliest controlled probe. All three workers stop. The response overrides their optimistic local estimate; the case does not invent an explanation for why the observed allowance reached zero after only four attempts.
RFC 9110’s Retry-After definition permits either a nonnegative delay in seconds or an HTTP date. A reset epoch and a delay are different representations. The fixture checks both valid Retry-After forms, rejects a malformed value for an explicit fallback decision and prevents a past date from producing a negative wait. It is not a complete HTTP parser.
In an implementation, translate those observations into the dispatcher’s waiting mechanism carefully. Clock error, receipt time and delayed responses matter when interpreting absolute dates; a restart must not erase an active pause. Do not create a zero-delay busy loop when a header is absent or invalid. Follow the provider’s fallback and retain an actionable error after the bounded retry policy is exhausted.

Worked recovery record: one probe, then a slower queue
The recovery trace below is actual output from the offline state model. Statuses and headers are authored inputs, so this verifies the model’s selected decisions rather than observing a real service.
| Fake-clock second | Worker and request | Authored result | Shared decision |
|---|---|---|---|
| 0 | A, A1 | 200 | Reserve one unit and continue |
| 3 | B, B1 | 200 | Continue through the same dispatcher |
| 6 | C, C1 | 200 | Continue; no separate C allowance |
| 9 | A, A2 | 429; remaining zero | Pause the shared scope until at least 46 |
| 10, 39, 44, 45 | A, B and C admission checks | No dispatch | All workers remain blocked |
| 46 | A, A2 attempt 2 | 200; remaining 23 | One recovery probe succeeds in this scenario |
| 52 | B, B2 | 200; remaining 22 | Resume with six-second spacing |
| 58 | C, C2 | 200; remaining 21 | Keep the slower policy and reserve |
The probe uses a distinct controlled permit after the pause boundary. The model denies an early probe at 45 and a second simultaneous probe at 46. It does not manufacture a full quota refill merely because a reset timestamp passed. Only the authored successful response supplies the new observed remaining value, 23, through an explicit state transition in this model; after reserving four, that leaves nineteen potential units before subsequent activity or responses change the estimate.
The normal budget run and this failure-injection run are separate experiments. The first admits twenty successful authored requests and leaves four queued. The recovery run makes seven attempts for six distinct successful requests, leaving eighteen of the original 24 requests unfinished. The retry of A2 is counted twice as network work. Neither record supports announcing that the whole reconciliation completed.
The saved Python 3.12.14 run passes 42 assertions, including the twenty normal admissions, shared blocking checks, recovery sequence, header-time cases and reserve accounting. Its single-process sequential calls model a central admission decision. They do not prove that multiple real processes can atomically reserve slots, recover a lease after a crash or preserve queue state.
Kiran therefore leaves implementation approval open. The next review must inspect the real shared-state mechanism, all consumers of the quota and behavior during a dispatcher outage. If coordination is unavailable, this proposed design pauses sending rather than allowing each worker to create a fresh private allowance. That is an explicit design choice awaiting implementation review, not a behavior demonstrated by the offline model.
The case’s proposed retry boundary is three total attempts for a read, including the initial attempt, and a 90-second task deadline, whichever ends first. If the next permitted dispatch would fall beyond that deadline, the job becomes pending owner review rather than bypassing the pause. The trace exercises only A2’s second attempt; it does not test retry exhaustion or prove completion before the deadline. A timed-out write would require separate outcome reconciliation and replay-safety evidence, which this read-only case excludes.
Queue priority also needs an explicit limit. Lou accepts the four-request historical deferral for this planning window, not unlimited starvation of lane C. Kiran records the oldest queued request and a review point at the next budget observation. If new urgent work continuously displaces the backlog, Lou must choose a revised freshness expectation, a smaller scope or approved capacity. A dispatcher cannot settle that service trade-off by itself. Keeping a request durable and visible is useful only if an owner can decide what to do when its acceptable age is exceeded.
For a real review, include a dispatcher restart, stale response, overlapping consumers and loss of shared storage among the missing checks. The sequential fixture has no thread race or persistence failure to expose. The handoff lists those gaps explicitly; the central Python object has not been tested as a production queue.
Reduce requests without hiding unfinished work
Before adding workers, look for requests that need not be sent. A cached answer may serve a repeated read only when its identity, authorization scope and freshness are appropriate. Coalescing simultaneous requests for the same authorized resource can avoid duplicate fetches. Neither tactic permits sharing another tenant’s response or presenting stale inspection status as current.
Webhooks may reduce polling where the provider supports them, but they need their own delivery and reconciliation design. Conditional reads can also help: GitHub documents that a correctly authorized conditional request returning 304 does not consume its primary allowance. That documented benefit should not be generalized to every provider, every response or every secondary limit. Cache hits, conditional network attempts and suppressed duplicate jobs should appear separately in the integration’s accounting.
Lou’s completed fictional decision record retains the urgent and current-project lanes, defers historical work when capacity is insufficient and asks Kiran to review the shared dispatcher before an authorized trial. A second throttle would extend the shared pause under the chosen bounded policy; after exhaustion, the queue remains pending and the owner receives an internal operational error through an approved implementation. No notification is actually sent here.
Measure queue age, attempted and completed requests, throttle responses, current pause deadline and unresolved work by lane. A lower error count is not enough if work is being dropped or remains indefinitely stale. A quota increase, if available and approved, also does not remove secondary restrictions or repair uncoordinated dispatch.
For further workplace guides, return to LandOffer.ai. Start this repair with one quota-scope inventory and a request-unit ledger, then ask the integration owner to review the shared pause, request priorities, retry ceiling and unresolved coordination tests. Those artifacts give the authorized implementer something concrete to test before more workers make the integration busier.


