How to record X (Twitter) trends for several countries every day

The job: a record of what trended on X, country by country, day after day. X shows the current board and nothing older, so the history only exists if you keep it. A small job that runs every morning, reads ten boards and writes them to a database gives you that history. After a month you can ask which topics trended in several countries at once, how long a topic stayed on a board, and what was trending in Japan on a given Tuesday.

You end with one SQLite table: one row per country, rank and day, with the trend's name, its post volume when X shows one, and a search link. No X developer account is involved.

Step by step

Step 1: read one country's board

Call X trending topics with country set to a two-letter ISO code: US, GB, JP, IN, BR and so on. Leave country out for the worldwide board. Codes are not WOEIDs, the Yahoo place IDs that X's own API uses; you send the code you already know.

Not every country has a board. A code the endpoint does not support is answered with invalid_params, which is not charged, and its problems line lists the codes that work. The trends by country answer lists them too; about two dozen large markets are covered, and city boards are not.

One call returns the whole board, about 30 trends, for $0.009. There is no second page: page.has_more is false. Each trend has:

• rank: its position on the board, 1 at the top.
• name: the trend as X shows it, a hashtag or a phrase.
• heat: the post volume X displays next to it, or null when X shows none.
• url: an X search link for the trend.

curl -G "https://api.okdata.online/v1/x/trends/list" \
  -H "Authorization: Bearer $OKDATA_API_KEY" \
  -d country=JP

Step 2: the daily job with storage

The script reads each country once and appends the boards to x_trends.db. Each request carries an Idempotency-Key made of the country and the date. If cron fires twice on the same day, or you rerun the job after a crash, the second request within 24 hours returns the stored board with meta.replayed: true and costs nothing, so you never pay twice for one day.

import datetime
import os
import sqlite3
import requests

API = "https://api.okdata.online/v1/x/trends/list"
KEY = os.environ["OKDATA_API_KEY"]
COUNTRIES = ["US", "GB", "JP", "IN", "BR", "DE", "FR", "ID", "MX", "TR"]

db = sqlite3.connect("x_trends.db")
db.execute("""CREATE TABLE IF NOT EXISTS trends (
    day TEXT, country TEXT, rank INTEGER, name TEXT, heat INTEGER, url TEXT, fetched_at TEXT,
    PRIMARY KEY (day, country, rank))""")

day = datetime.date.today().isoformat()
for country in COUNTRIES:
    headers = {"Authorization": f"Bearer {KEY}", "Idempotency-Key": f"x-trends-{country}-{day}"}
    try:
        body = requests.get(API, headers=headers, params={"country": country}, timeout=60).json()
    except requests.RequestException as err:
        print(country, "network error:", err)  # rerun later; the key prevents a double charge
        continue
    if body["code"] != "ok":
        print(country, body["code"], body["message"])  # failed calls are not charged
        continue
    rows = [(day, country, t["rank"], t["name"], t["heat"], t["url"], body["meta"]["fetched_at"])
            for t in body["data"]["items"]]
    db.executemany("INSERT OR REPLACE INTO trends VALUES (?, ?, ?, ?, ?, ?, ?)", rows)
    db.commit()
    print(country, len(rows), "trends")

Schedule it with cron, for example 0 7 * * * for 7:00 every morning. Ten countries a day for 30 days is 300 calls, $2.70. To capture faster topics, run it every few hours and change the key and the day column to include the hour.

Step 3: ask the archive questions

Once a few days are stored, plain SQL answers the useful questions. Topics trending in three or more countries on the same day:

SELECT day, name, COUNT(DISTINCT country) AS countries, MIN(rank) AS best_rank
FROM trends
GROUP BY day, name
HAVING countries >= 3
ORDER BY day DESC, countries DESC;

How many days a topic stayed on a country's board: SELECT country, name, COUNT(*) FROM trends GROUP BY country, name ORDER BY 3 DESC.

Good to know

• A board is a snapshot. Trends come and go within hours. Store meta.fetched_at with each row, as the script does, so you know exactly what moment a row describes.
• Names differ by language. The same story trends under different words in Japan and Brazil. Group by translated or normalised names if you compare countries.
• heat is often null. X shows a post count for some trends only. Rank is the field you can always compare.
• Public boards only. The personalised For you list is not available.
• What a trend is about. Pass the name to X post search with sort=top; one page ($0.002) usually shows what drives it.
• The official route. X's own API serves trends by WOEID, for countries and also for cities and regions, and its pricing page lists trends at one US cent per request under pay-per-use. If you need city boards, or already hold an X developer account, use it.

For the single-country question, see Twitter trends for one country. Trends and search together are the core of the trend research workflow.

{
  "endpoint": "x.trends.list",
  "params": {},
  "idempotency_key": "<a new UUID>"
}