Building a Live Data Provider

The contract a web service implements to deliver actual figures into a Metapad twin: what Metapad requests, how periods are calculated, and what to answer.

This article is for developers who connect another system (an ERP, a data warehouse, a public data feed) to Metapad. Model authors don't need it; for them, see Bringing Live Data into a Twin.

A live data provider is an HTTPS endpoint. When someone fetches live data for a twin, Metapad calls it once per scenario and writes the answer into that scenario's time series.

The exchange

Metapad calls you. The call happens when someone presses Fetch now, when a dashboard on a Live data scenario is opened or its time range changes, or when an AI tool asks for a fetch. You never push data.
One request per twin and scenario, covering every agent, property and timestep at once.
You do the mapping and the arithmetic. Metapad doesn't sum, average, interpolate or convert units. Each value you send is the value for one whole period.

Getting connected

Your endpoint must use HTTPS.
Its host has to be enabled for your Metapad. On app.metapad.ai, transentis does this, so contact us with the host name.
If your endpoint needs a token, it is set up together with the host, not stored in the model. Metapad sends it as Authorization: Bearer <token>. Answer 401 if it is missing or wrong.
Metapad doesn't follow redirects, waits at most 60 seconds and reads at most 32 MB of response.

The request

Metapad sends a GET to the URL entered as the twin's Provider URL and adds its parameters to the query string. Any parameters already in the URL (a tenant, a dataset) are kept, so you can use them for your own routing. Ignore parameters you don't know; new ones may be added.

GET https://provider.example/data?tenant=acme
    &twin_id=…&scenario_id=…
    &step_count=1&step_unit=month
    &epoch=2027-01-01T00:00:00%2B01:00&epoch_step_offset=12
    &start_date=2028-01-01T00:00:00%2B01:00
    &from_timestep=0&to_timestep=11

Parameter	Meaning
`twin_id`, `scenario_id`	Which twin and scenario are being filled. A provider serving several twins can choose its data by these.
`step_count`, `step_unit`	Length of one timestep, e.g. `1` `month`. Units: `second`, `minute`, `hour`, `day`, `week`, `month`, `quarter`, `year`.
`epoch`	The twin's calendar anchor, as a date and time with UTC offset.
`epoch_step_offset`	How many steps timestep 0 lies after the epoch (may be negative).
`start_date`	Start of timestep 0, for convenience. Don't calculate periods from it (see below).
`from_timestep`, `to_timestep`	The timesteps wanted, both inclusive.

All dates use the epoch's UTC offset; there are no named time zones.

Which period is timestep i

With k0 = epoch_step_offset and step = step_count × step_unit, timestep i covers

from  epoch + (k0 + i) × step       (included)
to    epoch + (k0 + i + 1) × step   (excluded)

Second to week are exact durations.
Month, quarter (3 months) and year (12 months) are calendar months: calculate each boundary in one jump from the epoch, and when the target month is shorter, use its last day.
Don't count from start_date. For months this gives different dates whenever the epoch falls on the 29th–31st. With an epoch on 31 January and k0 = 1, timestep 0 is 28 Feb – 31 Mar and timestep 1 is 31 Mar – 30 Apr. Counting from start_date would start timestep 1 on 28 March, and every value would land in the wrong period without any error.

The answer

200 with a JSON body: the values to write.
204 without a body: nothing to report.
Any other status counts as a failure and is shown to the user with its code (e.g. 401, 404, 429, 500). Metapad doesn't retry.

{
  "matching_mode": "name",
  "entries": [
    { "agent": "Berlin Store", "property": "revenue", "timestep": 0, "value": "31000" },
    { "agent": "Berlin Store", "property": "revenue", "timestep": 1, "value": "28500" }
  ],
  "meta": { "provider": "ACME ERP" }
}

Field	Rule
`matching_mode`	`"name"` or `"id"`. It must equal the twin's Match by setting, or the whole answer is refused.
`entries`	At most 50,000 (ask us if you need more). A longer list is refused, never cut off. Aggregate more coarsely or split across scenarios.
`agent`	Name mode: the agent's name, exactly. ID mode: the agent's ID.
`property`	Name mode: the property's name. ID mode: the property's ID.
`timestep`	Between `from_timestep` and `to_timestep`.
`value`	A string. Plain JSON numbers and booleans are accepted too.
`meta`	Optional and ignored by Metapad; useful for your own logging.

An answer that isn't JSON, or lacks matching_mode or entries, is refused as a whole.

Values by property type

Property type	Accepted value
Number	A decimal number: `"31000"`, `"0.25"`, `"-4e3"`
Boolean	`"true"` or `"false"`
Date	`YYYY-MM-DD` or a date and time with offset
Option	Exactly one of the property's options
JSON, Dictionary	Valid JSON text
Text, Rich text, Asset	Any string

What Metapad does with it

Entries that can't be used are skipped and counted, and the user sees the counts: unknown or ambiguous agent, unknown property, a property with a formula, a wrong value type, a timestep out of range, a duplicate (the first one wins), a malformed entry, or a value someone entered by hand (kept unless the user chose Fetch and replace manual values).

Everything else is written in one step, which a user can undo. Because a missing entry means "no data" and never clears a value, sending the same answer twice changes nothing, and a corrected figure simply replaces the old one.

Checklist

Serve GET over HTTPS and ignore unknown parameters.
Check the token; answer 401 if it is wrong.
Choose the data by twin_id / scenario_id (or your own parameters).
For each timestep, calculate its period from the epoch as above.
Aggregate your raw data per agent, property and period.
Answer 200 with the twin's matching_mode and the values as strings, or 204.
Stay under 50,000 entries and well under 60 seconds.

A minimal provider

# Flask. Months are counted in one jump from the epoch, clamped to the month end.
import calendar, datetime
from flask import Flask, request, jsonify, abort

app = Flask(__name__)
UNIT_MONTHS = {"month": 1, "quarter": 3, "year": 12}
UNIT_SECONDS = {"second": 1, "minute": 60, "hour": 3600, "day": 86400, "week": 604800}

def add_months(d, n):
    t = d.month - 1 + n
    y, m = d.year + t // 12, t % 12 + 1
    return d.replace(year=y, month=m, day=min(d.day, calendar.monthrange(y, m)[1]))

def period_start(epoch, count, unit, k):
    if unit in UNIT_SECONDS:
        return epoch + datetime.timedelta(seconds=k * count * UNIT_SECONDS[unit])
    return add_months(epoch, k * count * UNIT_MONTHS[unit])

@app.get("/data")
def data():
    if request.headers.get("Authorization") != "Bearer s3cr3t":
        abort(401)
    a = request.args
    epoch = datetime.datetime.fromisoformat(a["epoch"])
    k0, count, unit = int(a["epoch_step_offset"]), int(a["step_count"]), a["step_unit"]
    entries = []
    for i in range(int(a["from_timestep"]), int(a["to_timestep"]) + 1):
        lo = period_start(epoch, count, unit, k0 + i)
        hi = period_start(epoch, count, unit, k0 + i + 1)
        for agent, revenue in revenue_by_agent(lo, hi):   # your data source
            entries.append({"agent": agent, "property": "revenue",
                            "timestep": i, "value": str(revenue)})
    if not entries:
        return "", 204
    return jsonify({"matching_mode": "name", "entries": entries})

Next steps

Bringing Live Data into a Twin — what model authors see
Putting a Twin on a Calendar — where epoch and step come from