How do I test an agent?

Testing an agent splits into two jobs. Your own test runner covers the code around the model. Eval suites cover the model's behaviour and give CI a number to gate on.

For code that touches the database, @stackbone/sdk/db/testing exports createTestDatabase. It boots an ephemeral Postgres container, applies your migrations against it, and hands back a Drizzle handle plus a dispose(). Each call gets its own container, so parallel tests never see each other's rows. It needs Docker running and @testcontainers/postgresql as a dev dependency, which keeps your deployed image free of it. That peer is loaded only when you call createTestDatabase, so importing the subpath from a shared test helper works without it installed.

For behaviour, you build an eval suite in Studio: the case list, the criteria, the repeats and the pass mark all live there. Run it with stackbone eval <suite>, against your local stackbone dev box or against a deployed installation with --agent. It exits non-zero when the run misses the pass mark, so it works as a CI gate unchanged.

Pass --git-sha "$(git rev-parse HEAD)" to seal the commit onto the run, which lets you attribute an improvement later. Ctrl-C stops the watcher and leaves the run going, and --timeout running out does the same. The box keeps measuring and keeps spending your token budget, so stop a run from its own screen in Studio.

The pass mark is read off the suite at launch. Editing the suite while a run is in flight does not change that run's verdict.

Read more

• stackbone.database: createTestDatabase, its options and its requirements.
• stackbone eval: every flag and exit code of the command.
• Logs and debugging: reading what a failing run did.