What rate limits does a social data API have, and how should I handle 429 errors?

When you scrape a platform yourself, a 429 is a warning about your account: twscrape users know the message Ban detected: 429, (88) Rate limit exceeded. From a data API, a 429 only means you are sending faster than your key allows. Nothing gets banned, nothing is charged, and the response tells you when to try again.

How the limit works

Each key has a token bucket: a burst of calls you can make at once, refilled at a steady rate per second.

curl -s "https://api.okdata.online/v1/account" -H "Authorization: Bearer $OKDATA_API_KEY" | jq '.data.rate_limit'
# {"tier": "...", "burst": ..., "per_second": ...}

Every keyed response also carries:

Header	Meaning
X-RateLimit-Limit	burst size
X-RateLimit-Remaining	calls left in the bucket right now
X-RateLimit-Reset	seconds until the bucket is full again
Retry-After	on a 429 only: seconds to wait

The 429 body is the standard error: code: "rate_limited", charged: false, retry: "yes" and retry_after_seconds.

Pace the job instead of fighting 429s

import os, time, requests

H = {"Authorization": f"Bearer {os.environ['OKDATA_API_KEY']}"}
limit = requests.get("https://api.okdata.online/v1/account", headers=H, timeout=30).json()["data"]["rate_limit"]
gap = 1 / limit["per_second"]                     # steady pace the bucket can sustain

def get(path, **params):
    while True:
        r = requests.get(f"https://api.okdata.online{path}", headers=H, params=params, timeout=60)
        body = r.json()
        if body["code"] != "rate_limited":
            time.sleep(gap)
            return body
        time.sleep(body["retry_after_seconds"])     # free; just wait and resend

for bv in ["BV1xx411c7mD"]:
    page = get("/v1/bilibili/videos/comments/list", id=bv, sort="hot")

Run several workers only up to the burst size, and lower the pace if X-RateLimit-Remaining keeps hitting zero.

Plan big jobs by time, not just money

At a refill rate of r calls a second, N calls take about N ÷ r seconds. The cost does not change with speed: Bilibili video comments is $0.002 a page, Bilibili video search $0.002, and Weibo post search $0.002, whether you send them in a burst or spread over a night. If a deadline needs more throughput than your key allows, ask hello@okdata.online about a higher tier.

Gotchas

• Limits are per key. Two scripts sharing one key share one bucket.
• A retry with the same Idempotency-Key while the first attempt is still running also returns rate_limited, with a one-second wait.
• Platform-side limits still exist upstream. When a source is saturated you get upstream_busy, also free; back off longer.

The errors page lists every code with its retry advice.