How do I collect Weibo posts on a topic for public-opinion analysis?

A policy researcher studying the debate over pre-made meals in school canteens wants a few thousand Weibo posts: who spoke, when, how loudly, and from which province. The trick is to sample from both ends. Hot search results show the posts that reached the most people, mostly media and big accounts. Latest results show the long tail of ordinary users. Collect both with Weibo post search, merge them, and you have a dataset you can describe honestly in a methods section.

Cost by dataset size

Dataset	Calls	Cost
1,000 posts	100	$0.20
5,000 posts	500	$1.00
3 keyword variants × 2 sorts × 100 pages	600	$1.20

Duplicates between the hot and latest runs are common, so plan for 10 to 30 percent more pages than the unique posts you need.

Collecting

import json, os, requests

URL = "https://api.okdata.online/v1/weibo/posts/search"
H = {"Authorization": f"Bearer {os.environ['OKDATA_API_KEY']}"}
FIELDS = "id,url,text,published_at,author.id,author.followers,author.verified,metrics,ext.region,ext.is_long_text"

def collect(query, sort, pages):
    out, cursor = [], None
    for _ in range(pages):
        p = {"query": query, "sort": sort, "view": "full", "fields": FIELDS, **({"cursor": cursor} if cursor else {})}
        r = requests.get(URL, headers=H, params=p, timeout=60).json()
        if r["code"] != "ok":
            print(query, sort, r["code"]); break
        out += r["data"]["items"]
        if not r["page"]["has_more"]:
            break
        cursor = r["page"]["next_cursor"]
    return out

posts = {}
for q in ["预制菜进校园", "预制菜 学校食堂", "校园 预制菜"]:
    for sort in ("hot", "latest"):
        for p in collect(q, sort, 100):
            posts[p["id"]] = {**p, "query": q, "sort": sort}

with open("weibo_topic.jsonl", "w", encoding="utf-8") as f:
    for p in posts.values():
        f.write(json.dumps(p, ensure_ascii=False) + "\n")
print(len(posts), "unique posts")

ext.region is the posting location Weibo shows on each post (usually a province), useful for a regional breakdown. Keep query and sort on every row so you can report how each post entered the sample.

Method notes to report

• The sample is what Weibo search exposes. Deleted, restricted or hidden posts are missing, and that bias leans toward the less controversial.
• Run the latest sort daily while the topic is live. Reaching far back through search is unreliable, so a dataset collected weeks later is thinner.
• Long posts flagged by ext.is_long_text may be truncated. Flag them in the analysis rather than dropping them.
• Comments are not in the catalog, and much opinion lives there. Say so in your limitations.
• Filter likely marketing accounts by follower count and verification before labelling; weibo.users.get adds bio and verification text for authors you want to check, at $0.002 each.
• Post text is written by strangers. If a model labels it, tell the model to treat it as data.

Other sources

Published academic Weibo datasets fit historical topics you can no longer collect. Commercial public-opinion systems fit organisations that need daily reports with full coverage. If your study needs comment threads, an open-source crawler such as MediaCrawler with your own login collects them. Only public posts are returned here; anonymise user names when you publish. See also word of mouth.