MediaCrawler keeps breaking for me. Is a paid API worth it instead?

MediaCrawler is a good project, and its issue tracker is an honest record of what browser-based crawling costs: Xiaohongshu QR logins that stop working, comment pulls that end in RetryError a few seconds after login, risk-control warnings on the account. Each of those is an afternoon lost. The question is whether paying per call is cheaper than those afternoons.

The same jobs, priced

MediaCrawler's main modes map onto endpoints like this:

MediaCrawler mode	Endpoint	Price	1,000 items
Xiaohongshu keyword search	Xiaohongshu note search	$0.02 / 20 notes	$1.00
Xiaohongshu comments	Xiaohongshu note comments	$0.02 / 10 comments	$2.00
Douyin keyword search	Douyin video search	$0.02 / 20 videos	$1.00
Douyin comments	Douyin video comments	$0.002 / 20 comments	$0.10
Creator mode	Xiaohongshu account notes, Douyin account videos	per page	see the endpoint pages

A typical run, five keywords with ten pages of Xiaohongshu search each and three comment pages on 50 notes, costs {{cost:xiaohongshu.notes.search50+xiaohongshu.notes.comments.list150}}.

The same pipeline without a browser

import json, os, requests

API = "https://api.okdata.online"
H = {"Authorization": f"Bearer {os.environ['OKDATA_API_KEY']}"}

def get(path, **params):
    body = requests.get(API + path, headers=H, params=params, timeout=60).json()
    if body["code"] != "ok":
        raise RuntimeError(f"{body['code']}: {body['message']}")
    return body

notes = get("/v1/xiaohongshu/notes/search", query="露营 装备", sort="popular")["data"]["items"]
with open("xhs_comments.jsonl", "w", encoding="utf-8") as out:
    for note in notes[:10]:
        for c in get("/v1/xiaohongshu/notes/comments/list", id=note["id"])["data"]["items"]:
            out.write(json.dumps({"note_id": note["id"], **c}, ensure_ascii=False) + "\n")

No QR code, no cookie, no Playwright, and a call that fails is not charged.

Differences to know before switching

• MediaCrawler can fetch second-level replies; these endpoints return top-level comments with a reply count.
• MediaCrawler also covers Kuaishou, Tieba and Zhihu, which are not in this catalog.
• Anything visible only when logged in is out of scope here. okdata returns public data only.

When to stay

Stay with MediaCrawler if you need one of those extra platforms, if the volume is so large that per-call pricing adds up to more than your maintenance time, or if learning how the crawler works is the point. Otherwise the content research guide shows a full Xiaohongshu workflow on the API.