Observability

Every chat turn and every workflow run leaves a trace the box records by itself: the steps it took, what each one was given and returned, how long it took and what it spent in tokens. Every line your code prints is captured, stamped with the run and the step that printed it, and kept for 30 days. All of it lives in your box, the container running your workspace, in your own infrastructure. The control plane never receives it, and the box sends it out only when you point it at your own collector.

Two halves, one store:

Half	What it is	Where you read it
The trace	The steps of a run, recorded by the runtime, kept in the box's Postgres.	Studio under Runs, stackbone runs get, and your APM as one trace per run over OTLP.
The logs	The lines your tools and steps print, plus the runtime's own, run by run.	Studio under Logs (live tail and 30-day history), stackbone logs tail, the box's stdout, and your log backend over OTLP.

What a run records

A run is one chat turn or one workflow run. As it goes, the runtime writes one row per thing that happened, and each row is one bar on the trace:

Step	Written when	What the row holds
Activity	A 'use step' function of a workflow runs.	Its name, timing and result.
Model	An agent turn, or a call through stackbone.ai, reaches a model.	The model, the provider, latency, and input and output tokens.
Tool	The model calls one of your tools.	The tool's arguments and what it returned.
Subagent	A deep agent hands work to a subagent.	The task it was given and the reply. The subagent's own tools hang off it.
Connection	Your code calls a connector through Stackbone Connect.	The connector, the operation, the arguments and the response.
Approval	The run pauses for a person: requestApproval(), or a tool that waits.	The payload the reviewer saw and the decision that resumed it.
Guardrail	A guardrail inspects the turn.	What it inspected and the verdict it reached.

Rows nest, so a subagent's tool calls sit under the subagent, and a workflow's delegation to an agent sits under that step. Waits between steps show as gaps on the timeline with their length.

A workflow run: three steps, a two-second wait before the last one, and what the first step returned.

Open a run under Runs in Studio and you get the waterfall, the list of steps, and an inspector for whichever step you click. The header carries the run's status, trigger, duration and total tokens. Run Input and Final Output sit at the top and bottom of the step list. A run that paused for a person shows the pending decision above the trace. Copy run details puts the whole run and its trace on the clipboard as JSON, for a ticket or a coding agent.

Sessions groups chat turns into conversations, so a thread reads as one row with its token total and each turn opens its own trace. From a terminal, stackbone runs list and stackbone runs get <id> read the same rows.

You instrument nothing to get any of this. The trace is a projection of what the runtime already recorded, which is also why it survives a restart: the rows live in the box's own Postgres next to the run itself. If something is missing from a trace, it needs a new kind of step rather than a second tracer.

Log from your code

Inside a tool or a workflow step, log with plain console.*:

async function deliver(to: string, scope: string) {
  'use step';
  console.info('digest delivered', { to, scope, templateId: 'digest-v3' });
  return { to, scope, delivered: true };
}

The line shows up in Studio under Logs with its level, its time, its message and the fields you passed in the object, stamped with the run and the step that printed it. Open a failed step in a run and you see exactly what it printed. The screen has two tabs: Live tail for what is happening now, and Stored history for the 30 days the box kept. Filter by level, by run, by time range or by a substring, and pause the live tail to read. The run id is the one filter the URL carries, so a link you copy reopens the view filtered to that run.

What the steps above printed, on the live tail of the Logs screen.

Keep the message a short, stable phrase and put the variable data in an object: the fields stay filterable, a formatted string does not. The SDK page covers levels, Error arguments and why a log line belongs inside a step rather than in the workflow body.

The CLI reads the same store:

stackbone logs tail --follow                 # everything, live
stackbone logs tail --run <run-id> --follow  # one run
stackbone logs tail --since 15m --limit 200  # a window

What happens to a line

Everything the box emits goes through one pipeline before it is stored: your console.* lines, the SDK's structured lines and the runtime's own events. The same code runs under stackbone dev, in a self-hosted box and in a deployed one, so what you debug locally is what happens in production.

The pipeline...	So that...
stamps each line with the run and the step that printed it	a line is never an orphan. The step id comes from the workflow step that is executing, so a line from a tool inside an agent chat turn carries that turn's run and groups there. A line printed outside any run (a scheduled job, boot output) is stored with no run and the name of what produced it, so it stays filterable.
masks secrets	an API key printed by mistake never reaches the store or the export. The values of your workspace's own secrets are masked exactly; bearer tokens, key prefixes and credentials inside URLs are masked by pattern. Nothing else is rewritten: an email address stays searchable.
caps size and volume	a talkative loop degrades your logs instead of filling the disk. An oversized line is cut and marked as truncated. Past a hard number of lines per run the rest are dropped and counted, with one marker line saying how many.
writes in batches and never blocks	a database hiccup costs you log lines, never a run. When the database was unreachable, one marker line names the interval and how many lines were lost, so a gap in the record is declared instead of looking like silence.

Where the evidence lives, and for how long

This table is the box's retention statement: what it stores about your end users, where, for how long, who can read it and what removes it.

What	Where	How long	Who reads it	How it is deleted
Runs, steps, sessions, approvals	The box's own Postgres.	Until you delete them. They survive restarts and redeploys.	Every seat with runs:read, viewers and approvers included.	By deleting the run.
Log lines	The box's own Postgres.	30 days by default, configurable between 7 and 90 in the workspace settings, within a storage budget in gigabytes. Both keys are accepted from an owner or an admin only.	owner, admin and member, behind logs:read. viewer and approver keep runs and lose the raw dump.	Automatically at the end of the window. Deletion on request is not built yet (see below).
Prompt versions, config versions	The box's own Postgres.	Until you delete them.	Every seat with runs:read.	From their own screens.

Nothing on this page reaches the control plane. It holds your organization, its members and the record of where your box lives; runs, logs and payloads stay in your infrastructure. See Where do agents actually run?.

Retention and the storage budget

Log retention (30 days by default, 7 to 90) and the storage budget in gigabytes are two keys in the workspace settings in Studio, and the box accepts a write to either one from an owner or an admin only. A member writes the log lines and reads them, and cannot shorten the window they live in. One hourly job deletes the days past the window in small batches, never in one mass delete, and deletes the oldest day early when the budget is crossed, so a runaway loop degrades retention instead of taking the box down and is caught within the hour. The job appears on the Recurring jobs screen with its last execution, like every other timer the box runs.

Archived into your own bucket

When a day expires, the same job first writes it as one compressed object into the box's own object storage, under a reserved prefix laid out by organization, installation and date, records a digest of that object chained to the previous day's, and only then deletes the day's rows. Because each day's record carries the previous day's digest, a day removed or replaced is as detectable as a day edited. A weekly verification, and a button for an on-demand one, re-reads the objects and walks the chain; the settings screen shows when it last passed and states the encryption posture of the writes. Archiving is on by default wherever the box has object storage, and the screen says so plainly when it does not. The objects' lifecycle is yours; the box never expires them.

Deleted on request, and never quietly

Deletion is a first-class surface on the box: by run, by date range, by the whole box, and by person when your code seals a subject identifier into the run's context, so every line captured in that run inherits it. Deletion needs logs:read plus the admin or owner seat. A legal hold, one switch in the workspace settings with the identity and timestamp of whoever set it, suspends the hourly purge, the early delete under the storage budget and every deletion surface until it is lifted; ingest, archiving and verification carry on, and the screen states that a held box will outgrow its budget. Every deletion, every export and every raw read is recorded in the box's activity trail, so erasing evidence cannot also erase the proof of the erasure.

Ship it to the stack you already run

You keep the canonical copy in the box. Getting a copy out is optional, and it never blocks a run: a collector that is down costs you a dashboard, never the evidence, and the export failure count shows on the settings screen.

A deployed box always writes the same scrubbed lines as NDJSON to its standard output, so Promtail, Alloy, Vector, Fluent Bit or your cloud's agent pick them up like any other container. Under stackbone dev your terminal keeps showing the original console.* line instead, so the local console stays readable.

The OTLP export turns on when you name an endpoint. Set OTEL_EXPORTER_OTLP_ENDPOINT on the container that runs the box (or the per-signal OTEL_EXPORTER_OTLP_LOGS_ENDPOINT and OTEL_EXPORTER_OTLP_TRACES_ENDPOINT, plus OTEL_EXPORTER_OTLP_HEADERS for the collector's auth) and the box pushes over OTLP/HTTP in JSON to the collector you already run: the log lines as they happen, and, when a run finishes, that run as one trace whose root is the run and whose spans are its steps. The trace id is the run id, so the id you copy from Studio is the one you search for in your APM, and every exported log line carries the id of the span that printed it. Model calls carry the GenAI semantic conventions, so an LLM observability backend renders models and token counts without an exporter per vendor; the turns of one conversation share a conversation id. A run started by one of your own services with a traceparent header on the workflow start route is exported as part of that trace. Loki (3.0 and later) and any OpenTelemetry Collector accept this export directly; the Datadog Agent takes OTLP logs over HTTP with otlp_config.logs.enabled, and everything else (Phoenix, Elastic, Dynatrace) wants a Collector in front of it. The full variable table is on the SDK page.

These are deployment settings, not workspace settings: whoever runs the container owns the collector. The settings screen shows the effective posture, endpoint set or not and the last export failure count, without offering to edit it.

Read more

• Logging & observability: logging from tools and steps, the run timeline and the export contract, from the SDK's side.
• Governance: the rest of what Studio shows about a running box.
• Recurring jobs: what the box runs on a timer.
• Logs and debugging: reading what a failing run did.
• stackbone runs and stackbone logs tail.