Answers

What happens to a social data API when the platform changes or a source goes down?

Short answer

A good one absorbs the break so your code does not change: it tries another source where it has one, returns an error you are not charged for when every source fails, and keeps the same fields. On okdata, an upstream failure comes back as a free, retryable error, so a retry an hour later costs only the normal price, such as $0.006 for X search.

Platforms change constantly. Douyin rotates request signatures, endpoints get retired without notice, and open-source tools stop working until someone fixes them. A paid data API does not make that go away. What you are paying for is someone else handling it, and the useful question for any vendor is what you see while they do.

What okdata does when something breaks

  • Failover. Each endpoint has its data sources in a fixed order. Where there is more than one, an upstream failure, overload or timeout moves the call to the next source before you see anything.
  • Free failures. If every source fails, you get upstream_failed (502) or upstream_busy (503) with charged: false and retry: "yes". A Douyin search that fails costs nothing; the retry costs the normal $0.02.
  • The same fields. Every source maps onto the same objects. A field one source cannot fill is null, and meta.unavailable says why, so your parser does not break.
  • Visible status. GET https://api.okdata.online/v1/catalog/status needs no key and lists any paused endpoint; the same is on the status page.

One limit: a page cursor belongs to the source that issued it. If that source fails mid-listing, the next page fails too, free of charge; wait and retry, or start the listing again.

A job that rides out an outage

import os, time, requests

H = {"Authorization": f"Bearer {os.environ['OKDATA_API_KEY']}"}

status = requests.get("https://api.okdata.online/v1/catalog/status", timeout=30).json()["data"]
paused = [e["id"] for e in status["endpoints"] if e["status"] != "available"]
if "x.posts.search" in paused:
    raise SystemExit("X search is paused right now; try later")

def call(path, **params):
    for attempt in range(5):
        body = requests.get(f"https://api.okdata.online{path}", headers=H, params=params, timeout=90).json()
        if body["code"] == "ok":
            return body
        if body.get("retry") != "yes":          # not_found, invalid_params: stop
            raise RuntimeError(f"{body['code']}: {body['message']}")
        time.sleep(body.get("retry_after_seconds") or 2 ** attempt * 5)
    raise RuntimeError("still failing; check the status page")

posts = call("/v1/x/posts/search", query="your brand", sort="latest")

What to ask any vendor

  1. When a platform changes, how long are you usually down, and how do I find out?
  2. Am I charged for calls that fail during an outage?
  3. Is there a second source for the endpoints I depend on?
  4. Do field names stay the same when a source changes?

Endpoints such as X post search and YouTube video comments sit behind the same rules. The full error table, with which codes are charged and which are worth retrying, is on the errors page.

Updated