API Rate Limits and Throttling: Designing Integrations That Stay Under the Limit
An integration stays under API rate limits by knowing every limit it depends on, pacing its own calls on the client side, and cutting the number of calls it needs. When a provider does push back, the integration retries only in the ways that provider asks for, waiting exactly as long as the response tells it to.
The advice here is for integration architects, heads of enterprise applications and CIOs who manage API rate limits and throttling across Salesforce, Microsoft Graph and Shopify. By the end, you will be able to inventory every limit, pace every worker and place throttling correctly in your integration layer architecture.
What is the difference between rate limits, throttling and quotas?
A rate limit caps how many requests a client may send in a short window, while a quota caps total usage over a longer period, such as the 24-hour allocation Salesforce applies. Throttling is what a provider does once either cap is reached: it slows, queues or rejects further requests.
A concurrency limit caps how many requests may be in flight at once. A cost-based limit charges each request points that reflect the work it asks the server to do. Client-side throttling is your own deliberate pacing of outbound calls, so the provider never has to throttle you.
| Control | What it caps | Typical window | What the client sees |
|---|---|---|---|
| Rate limit | Requests per unit of time | Seconds to minutes | 429 Too Many Requests, often with Retry-After |
| Quota or daily allocation | Total requests per period | 24 hours or a billing period | Rejections until the allocation renews |
| Concurrency limit | Requests in flight at once | Instantaneous | New requests rejected while slots are full |
| Cost-based limit | Computed points per request | Per second, restored continuously | Cost and throttle status in the response |
| Throttling | Provider response to a breached limit | Duration of the breach | Requests slowed, queued or rejected |
How do API providers enforce rate limits?
Providers enforce rate limits by counting requests with a fixed window, sliding window, token bucket, leaky bucket or cost-based model. The model decides how bursts are treated, so pacing that suits one provider can still trigger throttling at another with the same nominal limit.
A fixed window resets its counter at each clock boundary, so a client can spend a full allowance either side of the boundary. A sliding window counts over a trailing interval, which removes that burst. A token bucket refills tokens at a fixed rate up to a capacity, allowing bursts up to the bucket size.
A leaky bucket drains queued requests at a constant rate and rejects overflow. A cost-based model charges points by computed complexity, and the Shopify GraphQL Admin API rate limits range from 100 points per second on Standard to 2,000 on Commerce Components. Salesforce adds a longer layer on top: a 24-hour allocation shared across the whole org.
| Model | How it counts | Burst behavior | Platform example |
|---|---|---|---|
| Fixed window | Per clock interval | Bursts at boundaries | Generic pattern |
| Sliding window | Trailing interval | Boundary bursts removed | Azure API Management rate-limit-by-key policy |
| Token bucket | Tokens refilled at a set rate | Bursts up to capacity | Amazon API Gateway |
| Leaky bucket | Constant drain rate | Bursts smoothed, overflow rejected | Most Shopify limits |
| Cost-based | Points per request | Heavy requests drain budget faster | Shopify GraphQL Admin API |
Why do integrations hit rate limits in production?
Integrations hit rate limits in production when call volume rises beyond what testing covered: initial loads, period-end jobs, polling loops, retry storms and workers scaled out without a shared limiter. The hardest cause to see is several integrations drawing on one shared allocation that nobody owns.
Initial loads and period-end bulk jobs push months of records through endpoints sized for daily changes. Polling spends calls asking whether anything changed, and every empty poll still counts. A retry storm is a surge of retries from many clients at once, adding load just as the provider sheds it.
Scaled-out workers multiply the call rate unless they share one limiter. An N plus 1 pattern, fetching a list and then calling once per item, turns one page of results into one request per record.
Salesforce enforces API limits across all API calls in the org, not per user, so a new marketing sync and an established ERP feed compete for the same 24-hour pool. Allocation planning belongs in the same review as your broader Salesforce integration architecture patterns.
How do you read the limit signals an API sends back?
An API signals a limit through the 429 Too Many Requests status code, a Retry-After header that states how long to wait, and headers that report remaining quota and reset time. Integrations should parse these signals on every response, not only on errors, and treat their units as provider-specific.
RFC 6585 Additional HTTP Status Codes, published in April 2012, defines 429 as the response when a user has sent too many requests in a given amount of time. Retry-After may be expressed as seconds or as an HTTP date.
Vendor header names and units vary. With Expose Headers on, the MuleSoft Rate Limiting policy returns X-Ratelimit-Remaining, X-Ratelimit-Limit and X-Ratelimit-Reset, reporting the reset in milliseconds while Retry-After uses seconds. Salesforce returns the REQUEST_LIMIT_EXCEEDED exception code, and the Shopify GraphQL Admin API returns an extensions.cost object with requestedQueryCost, actualQueryCost and throttleStatus.
The IETF RateLimit header fields draft standardizes these fields, but it is still an Internet-Draft, draft-ietf-httpapi-ratelimit-headers-11 as of May 2026, not an RFC. RateLimit-Policy declares quota (q), quota unit (qu), window (w) and partition key (pk), and RateLimit reports remaining quota (r) and effective window (t).
HTTP/1.1 429 Too Many Requests
Content-Type: application/json
Retry-After: 30
RateLimit-Policy: "default";q=100;w=10
RateLimit: "default";r=0;t=30
{“error”: “too_many_requests”, “detail”: “Quota for this window is exhausted. Retry after 30 seconds.”}
How do you design an integration that stays under the limit?
An integration stays under the limit by design when three things exist before go-live. It needs an inventory of every limit it depends on, a budget that divides each allocation among its consumers, and a shared client-side limiter pacing every worker.
Build a limit inventory for every endpoint
A limit inventory records the published limit, scope, signals and owner for every endpoint an integration calls. Build it from official documentation, because Salesforce says paid orgs may occasionally exceed the daily limit but customers should not rely on it.
| Endpoint | Published limit | Window | Scope | Concurrency cap | Cost model | Signals returned | Owner |
|---|---|---|---|---|---|---|---|
| Salesforce REST API, Enterprise Edition production org | 100,000 requests, plus 1,000 per Salesforce license, plus purchased add-ons | 24 hours | Org, across all API calls | 25 long-running requests of 20 seconds or longer | Request count | REQUEST_LIMIT_EXCEEDED | Named platform owner |
The Salesforce API request limits and allocations page is the source for each value. Record sandbox and production orgs as separate rows, because their allocations differ.
Budget the allocation across consumers
In our integration practice, we plan sustained usage at no more than 70 to 80 percent of a published allocation and reserve 10 to 20 percent of daily quotas for administrative and ad hoc use. The remainder becomes a named ceiling per consuming application.
Pace calls with a shared client-side limiter
A client-side limiter works only when every worker draws from the same bucket, because private buckets sized to the full limit multiply the real call rate by the worker count. In our integration practice, we size limiters to the budgeted ceiling and back them with a central store such as Redis.
import threading
import time
class TokenBucket:
"""Thread-safe token bucket that refills at rate tokens per second up to burst."""
def __init__(self, rate, burst):
if rate <= 0 or burst <= 0:
raise ValueError("rate and burst must be positive")
self.rate = float(rate)
self.capacity = float(burst)
self.tokens = float(burst)
self.updated = time.monotonic()
self.lock = threading.Lock()
def acquire(self, cost=1.0):
"""Block until cost tokens are available, then spend them."""
if cost > self.capacity:
raise ValueError("cost exceeds bucket capacity")
while True:
with self.lock:
now = time.monotonic()
refill = (now - self.updated) * self.rate
self.tokens = min(self.capacity, self.tokens + refill)
self.updated = now
if self.tokens >= cost:
self.tokens -= cost
return
wait = (cost - self.tokens) / self.rate
time.sleep(wait)
# Sharing one bucket across every worker with Redis:
# keep "tokens" and "updated" in one hash per provider and scope, for example
# "ratelimit:salesforce:prod-org", and run the refill-and-spend logic as a Lua
# script through EVAL so it executes atomically on the Redis server.
# Read the clock with the Redis TIME command so all workers share one time source.
limiter = TokenBucket(rate=8, burst=10) # 80 percent of 100 calls per 10 seconds
How do you reduce the number of API calls an integration needs?
The cheapest call is the one an integration never makes. Reduce volume by combining requests into batches, moving high-volume loads to bulk APIs, replacing polling with delta queries, change notifications or webhooks, requesting only needed fields, caching reference data and buffering bursts in a queue.
Batch and composite requests pack several operations into one HTTP call, cutting round trips but not always throttling. The Microsoft Graph throttling guidance states that each request inside a JSON batch is evaluated individually, so a batch can return 200 while sub-requests return 429. Microsoft SDKs do not retry those throttled sub-requests, so your code must resubmit them.
A bulk API moves large record sets in a few job requests instead of one call per record. Request only the fields each target needs, and cache slow-changing reference data such as currencies and picklists.
Polling spends calls whether or not anything changed, so Microsoft recommends delta query and change notifications instead. A delta query returns only records changed since the previous request, and a change notification pushes an event when a resource changes. A webhook is an HTTP callback a provider sends when an event occurs, so review how webhooks deliver events to consumers before cutting over.
Queue-based load leveling places a message broker between producers and the API, so bursts land in a queue that a consumer drains at the budgeted rate. Choosing between RabbitMQ, Kafka and Azure Service Bus depends on the ordering, replay and throughput each flow needs.
How should retries behave when an integration is throttled?
A throttled integration should retry only on 429, 503 and network timeouts, wait for the Retry-After interval when one is given, and otherwise back off exponentially with full jitter. Retries need a capped delay, a capped attempt count, one owning layer, idempotency keys and a dead letter path.
Exponential backoff doubles the maximum wait after each failed attempt, and full jitter picks a random delay between zero and that maximum so clients do not retry in sync. Retry amplification is the multiplication of load when several layers retry the same call.
The AWS Builders’ Library guidance on retries and backoff calls retries selfish under overload and warns that five layers each retrying three times can multiply database load by up to 243 times. Retry at one point in the stack.
- Retry only on 429, 503 and network timeouts; fail fast on other client errors.
- Cap attempts and add a retry budget, a token bucket that limits retries as a share of traffic.
- Send an idempotency key, a unique value per logical write, on every POST so a retried write applies once.
- Open a circuit breaker, which pauses calls for a cooling period, when throttling persists.
- Route exhausted payloads to a dead letter queue, which holds failed messages for replay.
Idempotency must hold end to end, so pair client-side keys with consumers that tolerate duplicate deliveries downstream. The function below reuses one key across every retry of a write.
import random
import time
import uuid
from datetime import datetime, timezone
from email.utils import parsedate_to_datetime
import requests
def retry_after_seconds(value):
# Retry-After is either delta-seconds ("30") or an HTTP-date.
if not value:
return None
if value.strip().isdigit():
return int(value)
try:
when = parsedate_to_datetime(value)
return max(0.0, (when - datetime.now(timezone.utc)).total_seconds())
except (TypeError, ValueError):
return None
def call_with_retry(method, url, payload=None, dead_letter=None,
base=0.5, cap=30.0, max_attempts=5):
# One key per logical write, reused on every retry of that write.
headers = {"Idempotency-Key": str(uuid.uuid4())} if method == "POST" else {}
for attempt in range(max_attempts):
try:
resp = requests.request(method, url, json=payload, headers=headers, timeout=30)
if resp.status_code not in (429, 503):
return resp
delay = retry_after_seconds(resp.headers.get("Retry-After"))
except requests.Timeout:
delay = None
if attempt < max_attempts - 1:
if delay is None:
delay = random.uniform(0, min(cap, base * 2 ** attempt))
time.sleep(delay)
if dead_letter is not None:
dead_letter(method, url, payload)
return None
Where should throttling live in your integration architecture?
Throttling lives in two places with two jobs. Gateway policies protect your own APIs from inbound traffic, while client-side pacing in the integration layer keeps outbound calls inside each provider’s published limits. Using one mechanism for both jobs leaves either your backends or your provider allocations unprotected.
On MuleSoft Anypoint Platform, the Rate Limiting policy for Mule Gateway enforces a quota per time window and returns 429 to HTTP APIs when it runs out. An Identifier DataWeave expression splits traffic into quota groups, and MuleSoft recommends Spike Control instead for protecting a backend.
The Amazon API Gateway throttling documentation explains that usage plans identify clients by API key and that limits apply in order of precedence. Usage plan per-client or per-method limits come first, then stage per-method limits, then account-level limits per Region, then AWS Regional throttling.
The Azure API Management rate-limit-by-key policy counts calls per renewal period, a sliding window of up to 300 seconds, against a counter key you define. An optional increment-condition controls which responses count, and throttled callers get 429 with Retry-After by default.
<policies>
<inbound>
<base />
<!-- 100 calls per 60-second sliding window per subscription; only 2xx responses count -->
<rate-limit-by-key calls="100"
renewal-period="60"
counter-key="@(context.Subscription.Id)"
increment-condition="@(context.Response.StatusCode / 100 == 2)" />
</inbound>
<backend>
<base />
</backend>
<outbound>
<base />
</outbound>
<on-error>
<base />
</on-error>
</policies>
Which rate limit strategy fits your integration estate?
Rate limit strategies work best in combination: a shared limiter for steady traffic, queue-based load leveling for bursts, bulk or batch APIs for volume, and webhooks or change notifications to retire polling. Gateway policies protect your own APIs, and buying allocation pays off only after avoidable calls are gone.
| Strategy | Best for | Limit type addressed | Build effort | Latency impact |
|---|---|---|---|---|
| Shared client-side limiter | Many workers, one provider | Rate and concurrency | Medium | Small wait under load |
| Queue-based load leveling | Bursty producers | Rate limits in bursts | Medium to high | Queuing delay |
| Bulk or batch APIs | Initial and high-volume loads | Daily quotas | Low to medium | Longer jobs, fewer calls |
| Change notifications or webhooks | Replacing polling | Quota spent on polling | Medium | Lower |
| Gateway policies | Protecting your own APIs | Inbound rate and spikes | Low | Rejected callers wait |
| Buying more allocation | Growth after optimization | Daily quotas | Low, recurring cost | None |
In our integration practice, we start with the shared limiter and the polling inventory, since they remove the most throttling for the least build effort. For limiters, queues and bulk loaders built into existing pipelines, our custom integration development services cover design through rollout.
What breaks rate limit designs at scale?
Rate limit designs break at scale when assumptions that held for one worker, one environment or one application stop holding. The usual causes are per-node limiters guarding a shared quota, unit mismatches, throttled batch sub-requests hidden behind a 200, sandbox allocations that differ from production, and new apps consuming shared allocations.
The MuleSoft Distributed option only takes effect in high-availability cluster mode, so per-node limiters can each enforce a full copy of a shared quota. A client that reads the MuleSoft millisecond reset value as seconds also waits far longer than intended.
Microsoft Graph batches can return 200 while sub-requests return 429, which hides throttling from monitoring that checks only the outer status. Sandbox load tests mislead too, since a Salesforce Full sandbox gets 5,000,000 calls per 24 hours.
Amazon API Gateway treats throttles as best-effort targets, not guaranteed ceilings, so clients must handle 429 even within limits. A newly connected app can also quietly drain a shared Salesforce org allocation.
How does this change for enterprises, SMEs and startups?
The principles hold at every size, but the operating model changes. Enterprises need central quota allocation and governance, SMEs gain most from platform-native throttling, bulk APIs and off-peak scheduling, and startups should build webhooks and Retry-After handling into every client from day one.
Enterprises should allocate each shared quota centrally, with a named ceiling per consuming application and approval before a new app connects. Gateway policies and API governance enforce those ceilings, so one team’s bulk job cannot starve another team’s order feed.
SMEs usually run one iPaaS, an integration platform delivered as a cloud service, so its platform-native throttling beats custom limiters. Move heavy jobs to bulk APIs and run them off-peak.
Startups should prefer webhooks over polling and handle Retry-After in every client from the first release, before enterprise customers run performance reviews. Retrofitting rate awareness later costs more than building it in.
Which metrics prove the design is working?
Five metrics show whether an integration stays under its limits: 429 responses per 1,000 calls, peak hourly share of the daily allocation consumed, retry ratio, queue depth with oldest message age, and end-to-end data freshness. Track each one per provider and per consuming application.
The 429 rate should trend toward zero once pacing works. Peak hourly share reveals bursts a daily total hides, and the retry ratio, retries divided by first attempts, exposes amplification early. Queue depth and oldest message age show whether consumers keep pace, while freshness measures what the business feels. Feed all five into production integration monitoring and alerting with budget-based thresholds.
Frequently Asked Questions
A 429 status code signals an exceeded limit, and throttling is the provider’s response to it. Retries should wait for Retry-After, sustained use should stay well below the published allocation, and limits can often be raised through add-ons or higher plan tiers.
What HTTP status code means an API rate limit was exceeded?
The 429 Too Many Requests status code, defined in RFC 6585 in April 2012, means a client sent too many requests in a given amount of time. The response may include a Retry-After header. Salesforce also signals some limits with the REQUEST_LIMIT_EXCEEDED exception code.
What is the difference between rate limiting and throttling?
Rate limiting is the rule: a cap on requests per defined window. Throttling is the enforcement, where the provider slows, queues or rejects requests once the cap is reached. Client-side throttling is your own pacing, so the provider never has to enforce the rule.
Should an integration retry immediately after a 429 response?
No. Wait for the Retry-After interval when the response includes one; otherwise use exponential backoff with full jitter, a capped delay and a capped attempt count. Immediate retries add load while the provider is shedding it and can start a retry storm.
How much headroom should an integration leave below a published API limit?
In our integration practice, we keep sustained usage no higher than 70 to 80 percent of a published allocation and hold back 10 to 20 percent of daily quotas for administrative and ad hoc work. That headroom absorbs bursts, retries and new consumers.
Can you increase an API provider’s rate limit?
Often, yes. Salesforce adds capacity through purchased add-ons, and the Shopify GraphQL Admin API rate rises from 100 points per second on Standard to 2,000 on Commerce Components. Remove avoidable calls first, because added allocation costs money and waste scales with it.
Start with the limit inventory: list every endpoint, its published limit, its scope and its owner, then compare peak hourly usage against the budget. That one exercise shows where a shared limiter, a queue or a bulk API will remove the most 429 responses. If the inventory exposes shared allocations nobody owns, assign owners before adding capacity.