Sync incrementally instead of re-fetching

Every response carries api.timestamp. Send it back on the next request as the timestamp query parameter and you get only the records that changed since then. This is the difference between pulling 87 records and pulling 2.

If you are polling any catalogue or list on a schedule and not using this, you are re-downloading a dataset that did not change.

Verified on production

Every number in this recipe comes from live calls against /v2/incidents. The boundary behaviour in particular was measured, not inferred - it is the one detail that decides whether your sync produces duplicates.

1. The mechanism

Two fields do the work:

Field	Where	Meaning
api.timestamp	response envelope	server time when this response was generated
ut	on each record	UNIX timestamp of that record's last update

You store api.timestamp and replay it. The full loop:

# 1. pierwszy, pełny przebieg
curl "https://api.statscore.com/v2/incidents?token=YOUR_TOKEN&sport_id=5&limit=250"

{
  "api": {
    "timestamp": 1785843788,
    "method": { "name": "incidents.index", "total_items": 87 },
    "data": { "incidents": [ { "id": 400, "name": "In possession", "ut": 1484735880 }, "…" ] }
  }
}

# 2. kolejny przebieg - podajemy zapamiętany timestamp
curl "https://api.statscore.com/v2/incidents?token=YOUR_TOKEN&sport_id=5&limit=250&timestamp=1749470582"

{
  "api": {
    "timestamp": 1785843817,
    "method": { "name": "incidents.index", "total_items": 2 },
    "data": {
      "incidents": [
        { "id": 3106, "name": "Play short-handed",      "ut": 1784109946 },
        { "id": 3107, "name": "Player return to pitch", "ut": 1784109946 }
      ]
    }
  }
}

87 → 2. Note that total_items also drops: it reports the size of the filtered result, not the size of the collection. Do not use it as a record count for your local store.

2. The boundary is inclusive - your writes must be idempotent

This is the part that quietly corrupts data if you get it wrong.

The filter is ut >= timestamp, not >. Measured directly: the two records above have ut: 1784109946, and a request with timestamp=1784109946 still returns both of them.

Request	total_items
no timestamp	87
timestamp=1749470582	2
timestamp=1784109946 - exactly the records' own ut	2

So every sync re-delivers the records that sat on the previous boundary.

Upsert by id. Never append.

Because the boundary is inclusive, a naive INSERT loop duplicates a row on every single poll that returns anything. The overlap is a feature - it guarantees you cannot miss a record that was written in the same second as your cursor - but it makes idempotency mandatory, not optional.

Key every write on the record's id and upsert. Then re-delivery is free and losing a cursor is survivable.

3. Cursor discipline

Three rules, each of which has a failure mode attached:

Store api.timestamp, not max(ut). The response timestamp is server time and is always at least the highest ut in the payload. Deriving your cursor from max(ut) looks equivalent but breaks on an empty result - there is no ut to take a maximum of, so you either keep the old cursor (correct, by accident) or write 0 (a full re-sync).

Advance the cursor only after the write succeeds. If you commit the cursor first and the write then fails, those records are gone from your next window forever. Order: fetch → write → commit cursor.

Do not share one cursor across endpoints. Each collection moves at its own pace; one cursor per (endpoint, filter) pair. A cursor for sport_id=5 says nothing about sport_id=1.

import { apiGet } from "./client.js";        // z recipe 01

/**
 * Przyrostowa synchronizacja jednej kolekcji.
 * Kursor to `api.timestamp` z odpowiedzi, NIE max(ut) - max(ut) nie istnieje
 * przy pustym wyniku.
 */
export async function syncCollection({ path, key, params = {}, cursor, upsert }) {
  // apiGet zwraca api.data, a my potrzebujemy też api.timestamp - stąd wariant raw
  const body = await apiGetRaw(path, { ...params, ...(cursor ? { timestamp: cursor } : {}) });

  const records = body.api.data?.[key] ?? [];

  // Granica jest INKLUZYWNA (ut >= timestamp), więc rekordy z poprzedniej granicy
  // przyjdą ponownie. Zapis musi być idempotentny - klucz na id.
  for (const r of records) await upsert(r);

  // kursor przesuwamy DOPIERO po udanym zapisie
  return { cursor: body.api.timestamp, changed: records.length };
}

async function apiGetRaw(path, params) {
  const url = new URL("https://api.statscore.com" + path);
  url.searchParams.set("token", await getToken());
  for (const [k, v] of Object.entries(params)) {
    if (v != null) url.searchParams.set(k, String(v));
  }
  const res = await fetch(url, { headers: { Accept: "application/json" } });
  const body = await res.json();
  // logujemy ścieżkę, nie URL - URL zawiera token
  if (!res.ok) throw new Error(`GET ${path} → HTTP ${res.status}`);
  return body;
}

# Python - ten sam kontrakt
def sync_collection(path: str, key: str, cursor: int | None, upsert, **params):
    if cursor:
        params["timestamp"] = cursor
    body = api_get_raw(path, **params)

    records = (body["api"].get("data") or {}).get(key) or []

    # granica inkluzywna → rekordy z poprzedniej granicy wrócą; upsert po id
    for r in records:
        upsert(r)

    # kursor przesuwamy po zapisie, nie przed
    return {"cursor": body["api"]["timestamp"], "changed": len(records)}

4. Respect the per-endpoint cache

Every endpoint has a cache time, and polling faster than it just burns quota against a response that cannot have changed. /v2/stages/{id}/bracket caches for 60 seconds, so a bracket poller tighter than 60 s is pure waste.

Cache times are in the reference, as x-cache-seconds

Every documented operation carries its cache window, so you can read it programmatically instead of guessing. Almost all of them are 60 seconds; the one exception is /v2/events, which is 30 - assume a single value for the whole API and you poll the event list half as often as you could.

The numbers do not come from the upstream OpenAPI document, which omits them - we add them in the build from the per-resource pages. A handful of operations still have no published value; those carry no x-cache-seconds rather than a guessed one.

Two mechanisms, different jobs - use both:

	timestamp	If-Modified-Since → 304
Saves	payload size	the whole response body
Granularity	per record	per response
Cursor to keep	api.timestamp	HTTP date

timestamp narrows what comes back; 304 Not Modified skips the transfer entirely when nothing changed at all. A poller can send both.

5. What this is not for

timestamp is a change filter, not a livescore mechanism. For a match in progress, the useful signals are /v2/livescore and /v2/feed (recipe 05) - the update rate there is driven by play, and you want the current state, not a diff.

Use incremental sync for the things that change slowly and that you mirror locally:

Collection	Why mirror it
/v2/incidents	87 records for soccer alone; ids and attributes you resolve on every timeline render
/v2/statuses	369 records; needed to label every match
/v2/sports, /v2/areas, /v2/languages	tiny, near-static, and you should never hardcode the ids
/v2/competitions, /v2/seasons, /v2/participants	large, slow-moving, expensive to re-pull

Errors

Status	internal_code	Cause
400	3	timestamp older than the allowed minimum - the message carries the minimum; drop the parameter for a full re-sync
400	4	malformed timestamp - it is UNIX seconds, not milliseconds and not ISO 8601
401	2	expired token - re-exchange, keep the cursor

A too-old cursor is a 400, not a silent full sync

There is a minimum allowed timestamp. A process that was down for a long time will come back and get rejected rather than quietly re-downloading everything. Handle internal_code: 3 by dropping the timestamp parameter and doing a full pass - and treat it as a signal that your sync was broken for longer than you thought.

Next

• Get a token and make your first call - the apiGet this builds on
• Build a livescore board - for live state, not diffs
• Get one match with score and stats - resolving the ids you mirror

Full reference: GET /v2/incidents · GET /v2/statuses