> ## Documentation Index > Fetch the complete documentation index at: https://docs.litmus.ai/llms.txt > Use this file to discover all available pages before exploring further. # Findings and reports > What a run found, the reports built from it, and the judges and calibration behind them. 30 routes. Every example runs with `LITMUS_URL`, `LITMUS_TOKEN` and `LITMUS_ORG` (your org tag) set in your shell (see [Authentication](/authentication)). A `$NAME` in a path is an id you read from the listing route on this page; a `` in a body is a value you supply, and every field the handler reads is shown, so send only the ones you need. Fields are what the handler's source reads and writes, listed mechanically; they are not a schema. | Method | Path | Scope | | - | - | - | | GET | `/v0/orgs/{orgtag}/held-sets/{gid}/quality` | `read` | | GET | `/v0/orgs/{orgtag}/rubric` | `read` | | GET | `/v0/orgs/{orgtag}/calibration` | `read` | | POST | `/v0/orgs/{orgtag}/calibration` | `push` | | GET | `/v0/orgs/{orgtag}/calibration/{cid}` | `read` | | PATCH | `/v0/orgs/{orgtag}/calibration/{cid}` | `push` | | GET | `/v0/orgs/{orgtag}/reports` | `read` | | POST | `/v0/orgs/{orgtag}/reports` | `push` | | POST | `/v0/orgs/{orgtag}/reports/weekly` | `push` | | GET | `/v0/orgs/{orgtag}/reports/weekly/delivery` | `read` | | PUT | `/v0/orgs/{orgtag}/reports/weekly/delivery` | `push` | | GET | `/v0/orgs/{orgtag}/reports/{report_id}` | `read` | | POST | `/v0/orgs/{orgtag}/reports/{report_id}/share` | `push` | | GET | `/v0/orgs/{orgtag}/failure-modes` | `read` | | POST | `/v0/orgs/{orgtag}/failure-modes` | `push` | | POST | `/v0/orgs/{orgtag}/failure-modes/investigate` | `push` | | GET | `/v0/orgs/{orgtag}/failure-modes/proposals` | `read` | | POST | `/v0/orgs/{orgtag}/failure-modes/proposals` | `push` | | POST | `/v0/orgs/{orgtag}/failure-modes/proposals/{pid}/confirm` | `push` | | POST | `/v0/orgs/{orgtag}/failure-modes/proposals/{pid}/dismiss` | `push` | | PATCH | `/v0/orgs/{orgtag}/failure-modes/{mid}` | `push` | | POST | `/v0/orgs/{orgtag}/failure-modes/{mid}/classify` | `push` | | POST | `/v0/orgs/{orgtag}/failure-modes/{mid}/rekey` | `push` | | POST | `/v0/orgs/{orgtag}/held-sets/{gid}/pickout` | `push` | | GET | `/v0/orgs/{orgtag}/scorers` | `read` | | POST | `/v0/root` | `read` | | GET | `/v0/root` | `read` | | GET | `/v0/root/threads/{tid}` | `read` | | PATCH | `/v0/root/threads/{tid}` | `read` | | DELETE | `/v0/root/threads/{tid}` | `read` | ## `GET /v0/orgs/{orgtag}/held-sets/{gid}/quality` `GET /v0/orgs/{org}/held-sets/{gid}/quality`: one held set's quality packet — the line, the ranked losses and one row per task with each check's verdict and its reason, every term with its tier and n, an unmeasured term absent (`heldquality`). `gid` is the held set's run group. `?format=md` is the Markdown export, the tier sentence first. A reader: no model call, no run, no corpus member opened. | | | | - | - | | Token scope | `read` (any org token) | | Org comes from | the `{orgtag}` path segment; a token for another org is refused with 403 | | Policy row | `apitokens.POLICY["/v0/orgs/{orgtag}/held-sets/{gid}/quality"]["GET"]` | | CLI | — | **Path parameters:** `orgtag`, `gid` **Query parameters the handler or its helper reads:** `format` **Response fields written in the handler:** `line`, `losses_ranked`, `model_calls`, `org`, `set`, `tasks`, `terms`, `tier_sentence`, `unmeasured` ```bash theme={null} curl -sS "$LITMUS_URL/v0/orgs/$LITMUS_ORG/held-sets/$GID/quality" \ -H "Authorization: Bearer $LITMUS_TOKEN" ``` ## `GET /v0/orgs/{orgtag}/rubric` `GET /v0/orgs/{org}/rubric?persona=&use_case=`: one persona's criteria for one job, trace checks and output checks apart (`valueprofile.rubric`). | | | | - | - | | Token scope | `read` (any org token) | | Org comes from | the `{orgtag}` path segment; a token for another org is refused with 403 | | Policy row | `apitokens.POLICY["/v0/orgs/{orgtag}/rubric"]["GET"]` | | CLI | — | **Path parameters:** `orgtag` **Query parameters the handler or its helper reads:** `persona`, `use_case` **Response fields written in the handler:** `org` ```bash theme={null} curl -sS "$LITMUS_URL/v0/orgs/$LITMUS_ORG/rubric" \ -H "Authorization: Bearer $LITMUS_TOKEN" ``` ## `GET /v0/orgs/{orgtag}/calibration` `GET /v0/orgs/{org}/calibration`: every calibration set, summarised. Retired sets only with `?retired=1`. | | | | - | - | | Token scope | `read` (any org token) | | Org comes from | the `{orgtag}` path segment; a token for another org is refused with 403 | | Policy row | `apitokens.POLICY["/v0/orgs/{orgtag}/calibration"]["GET"]` | | CLI | — | **Path parameters:** `orgtag` **Query parameters the handler or its helper reads:** `retired` **Response fields written in the handler:** `org`, `sets` ```bash theme={null} curl -sS "$LITMUS_URL/v0/orgs/$LITMUS_ORG/calibration" \ -H "Authorization: Bearer $LITMUS_TOKEN" ``` ## `POST /v0/orgs/{orgtag}/calibration` `POST /v0/orgs/{org}/calibration`: a set of judge verdicts beside a person's labels (`calibration.normalize` names the body). | | | | - | - | | Token scope | `push` or `admin` | | Org comes from | the `{orgtag}` path segment; a token for another org is refused with 403 | | Policy row | `apitokens.POLICY["/v0/orgs/{orgtag}/calibration"]["POST"]` | | CLI | — | **Path parameters:** `orgtag` **Response fields written in the helper the handler answers through:** `agreement`, `by`, `created_at`, `criterion`, `disagreements`, `drift`, `human`, `id`, `illustrative`, `judge`, `kappa`, `label`, `name`, `repeatability`, `retired`, `rubric` This call changes state in the org. ```bash theme={null} curl -sS -X POST "$LITMUS_URL/v0/orgs/$LITMUS_ORG/calibration" \ -H "Authorization: Bearer $LITMUS_TOKEN" \ -H "Content-Type: application/json" \ -d '{}' ``` ## `GET /v0/orgs/{orgtag}/calibration/{cid}` `GET /v0/orgs/{org}/calibration/{cid}`: one set's summary and its labelled items. | | | | - | - | | Token scope | `read` (any org token) | | Org comes from | the `{orgtag}` path segment; a token for another org is refused with 403 | | Policy row | `apitokens.POLICY["/v0/orgs/{orgtag}/calibration/{cid}"]["GET"]` | | CLI | — | **Path parameters:** `orgtag`, `cid` **Response:** JSON: a stored record or a helper's result, answered as it is, so its keys are not written in the source and none are listed. ```bash theme={null} curl -sS "$LITMUS_URL/v0/orgs/$LITMUS_ORG/calibration/$CID" \ -H "Authorization: Bearer $LITMUS_TOKEN" ``` ## `PATCH /v0/orgs/{orgtag}/calibration/{cid}` `PATCH /v0/orgs/{org}/calibration/{cid}` with `{retired: {reason}, by}` or `{retired: null}`: a set leaves the panel, and stays on the record, restorable. Nothing else about a set is editable — its labels are what a person gave. | | | | - | - | | Token scope | `push` or `admin` | | Org comes from | the `{orgtag}` path segment; a token for another org is refused with 403 | | Policy row | `apitokens.POLICY["/v0/orgs/{orgtag}/calibration/{cid}"]["PATCH"]` | | CLI | — | **Path parameters:** `orgtag`, `cid` **Body fields the handler or its helper reads:** `by`, `retired` **Response:** JSON: a stored record or a helper's result, answered as it is, so its keys are not written in the source and none are listed. This call changes state in the org. ```bash theme={null} curl -sS -X PATCH "$LITMUS_URL/v0/orgs/$LITMUS_ORG/calibration/$CID" \ -H "Authorization: Bearer $LITMUS_TOKEN" \ -H "Content-Type: application/json" \ -d '{"by": "", "retired": ""}' ``` ## `GET /v0/orgs/{orgtag}/reports` `GET /v0/orgs/{org}/reports`: this org's reports, newest first — title, kind, source, who made it and when, how often its link was opened, and whether the link answers now. Never the link itself. `?kind=weekly` (or any report kind) lists that kind only. | | | | - | - | | Token scope | `read` (any org token) | | Org comes from | the `{orgtag}` path segment; a token for another org is refused with 403 | | Policy row | `apitokens.POLICY["/v0/orgs/{orgtag}/reports"]["GET"]` | | CLI | — | **Path parameters:** `orgtag` **Query parameters the handler or its helper reads:** `kind`, `retired` **Response fields written in the handler:** `org`, `reports`, `retired_count` ```bash theme={null} curl -sS "$LITMUS_URL/v0/orgs/$LITMUS_ORG/reports" \ -H "Authorization: Bearer $LITMUS_TOKEN" ``` ## `POST /v0/orgs/{orgtag}/reports` `POST /v0/orgs/{org}/reports` with `{kind: "run_group" | "packet" | "provider_comparison", id, title?, made_by?}`: freeze what that run group's page (or that sandbox's realism, or that provider comparison as a member reads it) shows now into one report. Nothing is re-graded or re-run; a later run of the same group leaves this report as it was. 404 when the org has no such group, sandbox or comparison. `{kind: "written", markdown, title?, made_by?}` keeps a report an operator wrote — headings, tables, links — redacted and capped; 400 when the markdown is empty or too long. | | | | - | - | | Token scope | `push` or `admin` | | Org comes from | the `{orgtag}` path segment; a token for another org is refused with 403 | | Policy row | `apitokens.POLICY["/v0/orgs/{orgtag}/reports"]["POST"]` | | CLI | — | **Path parameters:** `orgtag` **Body fields the handler or its helper reads:** `id`, `kind`, `made_by`, `markdown`, `title` **Response:** JSON: a stored record or a helper's result, answered as it is, so its keys are not written in the source and none are listed. This call changes state in the org. ```bash theme={null} curl -sS -X POST "$LITMUS_URL/v0/orgs/$LITMUS_ORG/reports" \ -H "Authorization: Bearer $LITMUS_TOKEN" \ -H "Content-Type: application/json" \ -d '{"id": "", "kind": "", "made_by": "", "markdown": "", "title": ""}' ``` ## `POST /v0/orgs/{orgtag}/reports/weekly` `POST /v0/orgs/{org}/reports/weekly` with `{by?}`: write this org's page for the previous complete ISO week now (Monday 00:00Z to Sunday 23:59:59Z), assembled by rule from stored records with no model call. IDEMPOTENT per (org, ISO week): a second call replaces that week's page under the same report id (`replaced: true`), never a second page. Created with sharing off; when the org's weekly delivery is on, the page is shared and its link posted to the org's Slack channel once per week (`delivery.state`: off, no\_install, no\_token, posted, error). When its email is on, `delivery.email.state`: off, no\_members, no\_key, sent, error — mailed once per week. | | | | - | - | | Token scope | `push` or `admin` | | Org comes from | the `{orgtag}` path segment; a token for another org is refused with 403 | | Policy row | `apitokens.POLICY["/v0/orgs/{orgtag}/reports/weekly"]["POST"]` | | CLI | — | **Path parameters:** `orgtag` **Body fields the handler or its helper reads:** `by` **Response:** JSON: a stored record or a helper's result, answered as it is, so its keys are not written in the source and none are listed. This call changes state in the org. ```bash theme={null} curl -sS -X POST "$LITMUS_URL/v0/orgs/$LITMUS_ORG/reports/weekly" \ -H "Authorization: Bearer $LITMUS_TOKEN" \ -H "Content-Type: application/json" \ -d '{"by": "<by>"}' ``` ## `GET /v0/orgs/{orgtag}/reports/weekly/delivery` `GET /v0/orgs/{org}/reports/weekly/delivery`: whether each week's page is shared and its link posted to Slack — `{on, channel, by, at}` — and whether this org has a Slack install to post through. Off by default. | | | | - | - | | Token scope | `read` (any org token) | | Org comes from | the `{orgtag}` path segment; a token for another org is refused with 403 | | Policy row | `apitokens.POLICY["/v0/orgs/{orgtag}/reports/weekly/delivery"]["GET"]` | | CLI | — | **Path parameters:** `orgtag` **Response fields written in the handler:** `delivery`, `org`, `slack_installed`, `tier` ```bash theme={null} curl -sS "$LITMUS_URL/v0/orgs/$LITMUS_ORG/reports/weekly/delivery" \ -H "Authorization: Bearer $LITMUS_TOKEN" ``` ## `PUT /v0/orgs/{orgtag}/reports/weekly/delivery` `PUT /v0/orgs/{org}/reports/weekly/delivery` with `{on, channel?, by?}`: turn weekly delivery on (a Slack channel id is required when the org has a Slack install) or off. Or `{email: {on, finding?: {a, b, main?}}, by?}`: the email target alone — the week's headline lines and the named compare's bug finding mailed to each member who has not opted out (`email_opt_out` on the member), through Resend by `RESEND_API_KEY`. Off by default; each half is changed without touching the other. Kept on the org's record with who set it. From the next page on, each week's page is shared and its link posted to that channel; turning it off shares nothing new and leaves the links already sent as they are. | | | | - | - | | Token scope | `push` or `admin` | | Org comes from | the `{orgtag}` path segment; a token for another org is refused with 403 | | Policy row | `apitokens.POLICY["/v0/orgs/{orgtag}/reports/weekly/delivery"]["PUT"]` | | CLI | — | **Path parameters:** `orgtag` **Body fields the handler or its helper reads:** `by`, `channel`, `confirm`, `email`, `on` **Response fields written in the handler:** `delivery`, `org`, `slack_installed`, `tier` This call changes state in the org. ```bash theme={null} curl -sS -X PUT "$LITMUS_URL/v0/orgs/$LITMUS_ORG/reports/weekly/delivery" \ -H "Authorization: Bearer $LITMUS_TOKEN" \ -H "Content-Type: application/json" \ -d '{"by": "<by>", "channel": "<channel>", "confirm": "<confirm>", "email": "<email>", "on": "<on>"}' ``` ## `GET /v0/orgs/{orgtag}/reports/{report_id}` `GET /v0/orgs/{org}/reports/{report_id}`: one report as its reader will see it, with who made it, its source and its opens. Never the link. | | | | - | - | | Token scope | `read` (any org token) | | Org comes from | the `{orgtag}` path segment; a token for another org is refused with 403 | | Policy row | `apitokens.POLICY["/v0/orgs/{orgtag}/reports/{report_id}"]["GET"]` | | CLI | — | **Path parameters:** `orgtag`, `report_id` **Response:** JSON: a stored record or a helper's result, answered as it is, so its keys are not written in the source and none are listed. ```bash theme={null} curl -sS "$LITMUS_URL/v0/orgs/$LITMUS_ORG/reports/$REPORT_ID" \ -H "Authorization: Bearer $LITMUS_TOKEN" ``` ## `POST /v0/orgs/{orgtag}/reports/{report_id}/share` `POST /v0/orgs/{org}/reports/{report_id}/share` with `{on: bool, named?: bool}`. On: a NEW link token, answered once as `token` and stored only as its hash; `named` puts the org's name on the shared page. Off: the link stops answering at once. Turning it on again never revives an old link. | | | | - | - | | Token scope | `push` or `admin` | | Org comes from | the `{orgtag}` path segment; a token for another org is refused with 403 | | Policy row | `apitokens.POLICY["/v0/orgs/{orgtag}/reports/{report_id}/share"]["POST"]` | | CLI | — | **Path parameters:** `orgtag`, `report_id` **Body fields the handler or its helper reads:** `named`, `on` **Response:** JSON: a stored record or a helper's result, answered as it is, so its keys are not written in the source and none are listed. This call changes state in the org. ```bash theme={null} curl -sS -X POST "$LITMUS_URL/v0/orgs/$LITMUS_ORG/reports/$REPORT_ID/share" \ -H "Authorization: Bearer $LITMUS_TOKEN" \ -H "Content-Type: application/json" \ -d '{"named": "<named>", "on": "<on>"}' ``` ## `GET /v0/orgs/{orgtag}/failure-modes` This org's known failure modes, oldest first — the feedback notes that carry a `mode` block — with the vocabularies a form offers. Read by `?audience=` as the feedback list is. | | | | - | - | | Token scope | `read` (any org token) | | Org comes from | the `{orgtag}` path segment; a token for another org is refused with 403 | | Policy row | `apitokens.POLICY["/v0/orgs/{orgtag}/failure-modes"]["GET"]` | | CLI | — | **Path parameters:** `orgtag` **Query parameters the handler or its helper reads:** `audience` **Response fields written in the handler:** `audience`, `behaviours`, `count`, `failure_modes`, `generatable`, `org`, `origins`, `statuses`, `tier`, `to_classify` ```bash theme={null} curl -sS "$LITMUS_URL/v0/orgs/$LITMUS_ORG/failure-modes" \ -H "Authorization: Bearer $LITMUS_TOKEN" ``` ## `POST /v0/orgs/{orgtag}/failure-modes` `{modes: [{title, description?, example?, reported_by?, seen_at?, status?, origin?, behaviour?, target?, tasks?, ticket?, trace?}], author | by}` or one such object — the console's form, an operator's import of a ticket export and the customer's feedback hook write the same notes. EVERY mode is checked before ANY is stored, so a batch with one bad row stores nothing and names the row. Each is a feedback note (`feedback.record`): verbatim, human tier, append-only. | | | | - | - | | Token scope | `push` or `admin` | | Org comes from | the `{orgtag}` path segment; a token for another org is refused with 403 | | Policy row | `apitokens.POLICY["/v0/orgs/{orgtag}/failure-modes"]["POST"]` | | CLI | `litmus-ai failure-modes` | **Path parameters:** `orgtag` **Body fields the handler or its helper reads:** `author`, `author_member`, `by`, `modes`, `visibility` **Response fields written in the handler:** `org` This call changes state in the org. ```bash theme={null} curl -sS -X POST "$LITMUS_URL/v0/orgs/$LITMUS_ORG/failure-modes" \ -H "Authorization: Bearer $LITMUS_TOKEN" \ -H "Content-Type: application/json" \ -d '{"author": "<author>", "author_member": "<author_member>", "by": "<by>", "modes": "<modes>", "visibility": "<visibility>"}' ``` ## `POST /v0/orgs/{orgtag}/failure-modes/investigate` `{text, author, ticket, origin_source, session_ids?, trace_ids?, window?, tools?, title?, seen_at?, member?, stats?}` — one report, investigated: pull what it names, record it as a failure mode with the trace cited, and name the sandbox its tasks go on. Writes the note (and the pulled evidence); writes no task. | | | | - | - | | Token scope | `push` or `admin` | | Org comes from | the `{orgtag}` path segment; a token for another org is refused with 403 | | Policy row | `apitokens.POLICY["/v0/orgs/{orgtag}/failure-modes/investigate"]["POST"]` | | CLI | — | **Path parameters:** `orgtag` **Body fields the handler or its helper reads:** `author`, `member`, `origin_source`, `report`, `seen_at`, `stats`, `text`, `ticket`, `title`, `window` **Response fields written in the handler:** `behaviour_words`, `cited`, `duplicate`, `failure_mode`, `generatable`, `org`, `pull`, `tier`, `trace_state`, `wanted`, `why_no_tasks` This call changes state in the org. ```bash theme={null} curl -sS -X POST "$LITMUS_URL/v0/orgs/$LITMUS_ORG/failure-modes/investigate" \ -H "Authorization: Bearer $LITMUS_TOKEN" \ -H "Content-Type: application/json" \ -d '{"author": "<author>", "member": "<member>", "origin_source": "<origin_source>", "report": "<report>", "seen_at": "<seen_at>", "stats": "<stats>", "text": "<text>", "ticket": "<ticket>", "title": "<title>", "window": "<window>"}' ``` ## `GET /v0/orgs/{orgtag}/failure-modes/proposals` `?origin=slack|console&state=proposed|confirmed|dismissed` — this org's proposals, oldest first, each with what its confirm did (the mode, the sandbox, the job, the task indexes once landed, the run link). | | | | - | - | | Token scope | `read` (any org token) | | Org comes from | the `{orgtag}` path segment; a token for another org is refused with 403 | | Policy row | `apitokens.POLICY["/v0/orgs/{orgtag}/failure-modes/proposals"]["GET"]` | | CLI | — | **Path parameters:** `orgtag` **Query parameters the handler or its helper reads:** `origin`, `state` **Response fields written in the handler:** `buckets`, `count`, `org`, `proposals`, `tier` ```bash theme={null} curl -sS "$LITMUS_URL/v0/orgs/$LITMUS_ORG/failure-modes/proposals" \ -H "Authorization: Bearer $LITMUS_TOKEN" ``` ## `POST /v0/orgs/{orgtag}/failure-modes/proposals` `{report: rpt_…}` (a report the Slack handler parked) or `{text, author, ticket?, origin_source?, tools?, seen_at?, member?}` — the report read into ONE proposed failure mode by rule (`reportproposal`), stored as a proposal in the org's store. Writes no failure mode. | | | | - | - | | Token scope | `push` or `admin` | | Org comes from | the `{orgtag}` path segment; a token for another org is refused with 403 | | Policy row | `apitokens.POLICY["/v0/orgs/{orgtag}/failure-modes/proposals"]["POST"]` | | CLI | — | **Path parameters:** `orgtag` **Body fields the handler or its helper reads:** `author`, `member`, `origin_source`, `report`, `seen_at`, `text`, `ticket`, `tools` **Response fields written in the handler:** `duplicate`, `org`, `proposal`, `tier` This call changes state in the org. ```bash theme={null} curl -sS -X POST "$LITMUS_URL/v0/orgs/$LITMUS_ORG/failure-modes/proposals" \ -H "Authorization: Bearer $LITMUS_TOKEN" \ -H "Content-Type: application/json" \ -d '{"author": "<author>", "member": "<member>", "origin_source": "<origin_source>", "report": "<report>", "seen_at": "<seen_at>", "text": "<text>", "ticket": "<ticket>", "tools": "<tools>"}' ``` ## `POST /v0/orgs/{orgtag}/failure-modes/proposals/{pid}/confirm` `{by, title?, behaviour?, bucket?, environment?, count?, target_tool?}` — a member confirms the proposal: the failure mode is written through the failure-mode store (human tier, `made_mode` naming who), and its tasks (`count`, 6 by default, 5 to 8) are written through the existing generation route on the bucket's sandbox — a write's as a job, a no-data mode's at once (201, `stored_as`, no job), read straight off that answer. Answers `{proposal, mode, environment, job, run_link}`. A second confirm answers the first's result (200). | | | | - | - | | Token scope | `push` or `admin` | | Org comes from | the `{orgtag}` path segment; a token for another org is refused with 403 | | Policy row | `apitokens.POLICY["/v0/orgs/{orgtag}/failure-modes/proposals/{pid}/confirm"]["POST"]` | | CLI | — | **Path parameters:** `orgtag`, `pid` **Body fields the handler or its helper reads:** `behaviour`, `bucket`, `by`, `count`, `environment`, `target_tool`, `title` **Response fields written in the helper the handler answers through:** `already`, `environment`, `job`, `mode`, `org`, `proposal`, `run_link`, `tier` This call changes state in the org. ```bash theme={null} curl -sS -X POST "$LITMUS_URL/v0/orgs/$LITMUS_ORG/failure-modes/proposals/$PID/confirm" \ -H "Authorization: Bearer $LITMUS_TOKEN" \ -H "Content-Type: application/json" \ -d '{"behaviour": "<behaviour>", "bucket": "<bucket>", "by": "<by>", "count": "<count>", "environment": "<environment>", "target_tool": "<target_tool>", "title": "<title>"}' ``` ## `POST /v0/orgs/{orgtag}/failure-modes/proposals/{pid}/dismiss` `{by}` — a member dismisses the proposal: nothing is written but the proposal's state and who. A confirmed one is refused (409); a dismissed one answers as it stands. | | | | - | - | | Token scope | `push` or `admin` | | Org comes from | the `{orgtag}` path segment; a token for another org is refused with 403 | | Policy row | `apitokens.POLICY["/v0/orgs/{orgtag}/failure-modes/proposals/{pid}/dismiss"]["POST"]` | | CLI | — | **Path parameters:** `orgtag`, `pid` **Response fields written in the handler:** `org`, `proposal`, `tier` This call changes state in the org. ```bash theme={null} curl -sS -X POST "$LITMUS_URL/v0/orgs/$LITMUS_ORG/failure-modes/proposals/$PID/dismiss" \ -H "Authorization: Bearer $LITMUS_TOKEN" \ -H "Content-Type: application/json" \ -d '{}' ``` ## `PATCH /v0/orgs/{orgtag}/failure-modes/{mid}` `PATCH …/failure-modes/{id}` `{behaviour?, bucket?, target?: {tool?, column?}, status?, by}`: a person classifies a mode already recorded — `failuremodes.to_classify` lists the ones nobody has, and this is what writes. Each field is checked against the record's own vocabularies; the note's words are untouched; who and when go on `mode.classified`, human tier. Another org's id answers 404, as one that does not exist. 200 with the mode, and `changed` empty when the classification said what the mode already said. | | | | - | - | | Token scope | `push` or `admin` | | Org comes from | the `{orgtag}` path segment; a token for another org is refused with 403 | | Policy row | `apitokens.POLICY["/v0/orgs/{orgtag}/failure-modes/{mid}"]["PATCH"]` | | CLI | — | **Path parameters:** `orgtag`, `mid` **Body fields the handler or its helper reads:** `author`, `by` **Response fields written in the handler:** `changed`, `failure_mode`, `generatable`, `org`, `tier` This call changes state in the org. ```bash theme={null} curl -sS -X PATCH "$LITMUS_URL/v0/orgs/$LITMUS_ORG/failure-modes/$MID" \ -H "Authorization: Bearer $LITMUS_TOKEN" \ -H "Content-Type: application/json" \ -d '{"author": "<author>", "by": "<by>"}' ``` ## `POST /v0/orgs/{orgtag}/failure-modes/{mid}/classify` `POST …/failure-modes/{id}/classify` `{reader?: rule | model, holding?: [tool], model?, dry_run?, by}`: the sessions the org's feedback sheet reported, read against this `false_no_data` mode (`modesessions`). Every sheet note about a stored trace opens its session; the rule reads each (tier deterministic, no model) and, with `reader: model`, one model call per session reads it too (tier `llm_derived`, its entity kept only when cited). `holding` names the tools that hold the rows as the customer's traces call them; the mode's `target.tool` otherwise. `dry_run` with the model reader answers the estimated cost and writes nothing. The reading is written on the mode's record (`sessions_read`) for the mint to ground its tasks in (`…/environments/{id}/tasks {failure_mode: {id, from_sessions}}`). Another org's id is a 404. | | | | - | - | | Token scope | `push` or `admin` | | Org comes from | the `{orgtag}` path segment; a token for another org is refused with 403 | | Policy row | `apitokens.POLICY["/v0/orgs/{orgtag}/failure-modes/{mid}/classify"]["POST"]` | | CLI | — | **Path parameters:** `orgtag`, `mid` **Body fields the handler or its helper reads:** `author`, `by`, `dry_run`, `holding`, `model`, `reader` **Response fields written in the handler:** `counts`, `dry_run`, `entities`, `entries`, `estimate`, `failure_mode`, `holding`, `holding_seen_in`, `org`, `reader`, `sessions`, `with_turns` This call changes state in the org. ```bash theme={null} curl -sS -X POST "$LITMUS_URL/v0/orgs/$LITMUS_ORG/failure-modes/$MID/classify" \ -H "Authorization: Bearer $LITMUS_TOKEN" \ -H "Content-Type: application/json" \ -d '{"author": "<author>", "by": "<by>", "dry_run": "<dry_run>", "holding": "<holding>", "model": "<model>", "reader": "<reader>"}' ``` ## `POST /v0/orgs/{orgtag}/failure-modes/{mid}/rekey` `POST …/failure-modes/{id}/rekey` `{environment, by, dry_run?}`: a held `false_no_data` set's answer key, re-written so it rejects a sprayed or padded list (`failuremodes.rekey_false_no_data`). The requests and the oracle stay; each task of this mode on the environment gains `answer_excludes` over the holding table's other rows, executed against the right answer and five wrong ones before it is kept. The world is drawn as the environment's trials draw it and the holding tool is called on a scratch copy. `dry_run` answers the counts and writes nothing. \$0: no model call. Another org's id is a 404. | | | | - | - | | Token scope | `push` or `admin` | | Org comes from | the `{orgtag}` path segment; a token for another org is refused with 403 | | Policy row | `apitokens.POLICY["/v0/orgs/{orgtag}/failure-modes/{mid}/rekey"]["POST"]` | | CLI | — | **Path parameters:** `orgtag`, `mid` **Body fields the handler or its helper reads:** `by`, `dry_run`, `environment` **Response fields written in the handler:** `board`, `dry_run`, `environment`, `failure_mode`, `model_calls`, `note`, `rekeyed`, `rekeyed_as`, `tier`, `usd` This call changes state in the org. ```bash theme={null} curl -sS -X POST "$LITMUS_URL/v0/orgs/$LITMUS_ORG/failure-modes/$MID/rekey" \ -H "Authorization: Bearer $LITMUS_TOKEN" \ -H "Content-Type: application/json" \ -d '{"by": "<by>", "dry_run": "<dry_run>", "environment": "<environment>"}' ``` ## `POST /v0/orgs/{orgtag}/held-sets/{gid}/pickout` `POST /v0/orgs/{org}/held-sets/{gid}/pickout` `{dry_run?, by, seed?}`: the model pick-out read of a held set. Each task's request is shown among 4 of the org's real requests, shuffled by a seeded draw, to one pinned model (`pickout.model()`, a pin of `configs/providers.yaml`), which names the one not written by a user and quotes it; an unresolved quote is refused. The term reads "picked out K of N (chance 1 in 5)", tier `llm_derived`, never summed with the deterministic band term. `dry_run` answers the estimated cost and calls no model. Otherwise the response carries the cost before (`estimate`) and after (`cost`, the provider's counts), and the verdicts are written on the held set's group record (`pickout.FIELD`) — never a real request, and the model's reason only when it quotes the task. `gid` is the held set's run group; another org's group is a 404. | | | | - | - | | Token scope | `push` or `admin` | | Org comes from | the `{orgtag}` path segment; a token for another org is refused with 403 | | Policy row | `apitokens.POLICY["/v0/orgs/{orgtag}/held-sets/{gid}/pickout"]["POST"]` | | CLI | — | **Path parameters:** `orgtag`, `gid` **Body fields the handler or its helper reads:** `by`, `dry_run`, `seed` **Response fields written in the handler:** `cost`, `dry_run`, `estimate`, `group`, `model`, `org`, `prompt_version`, `real_n`, `seed`, `tasks`, `term`, `tier`, `verdicts` This call changes state in the org. ```bash theme={null} curl -sS -X POST "$LITMUS_URL/v0/orgs/$LITMUS_ORG/held-sets/$GID/pickout" \ -H "Authorization: Bearer $LITMUS_TOKEN" \ -H "Content-Type: application/json" \ -d '{"by": "<by>", "dry_run": "<dry_run>", "seed": "<seed>"}' ``` ## `GET /v0/orgs/{orgtag}/scorers` THE ORG'S SCORERS: every check kind in use on the org's standing tasks — its name and tier, what runs verbatim (the detector's read as executed, or the judge's prompt as sent, with each value the task supplies named as a slot), its version digest, where it is used (checks, tasks, sandboxes) and its last verdicts (met / missed / no-verdict kinds over the stored trials of each task's newest run). | | | | - | - | | Token scope | `read` (any org token) | | Org comes from | the `{orgtag}` path segment; a token for another org is refused with 403 | | Policy row | `apitokens.POLICY["/v0/orgs/{orgtag}/scorers"]["GET"]` | | CLI | — | **Path parameters:** `orgtag` **Response fields written in the handler:** `judge`, `org`, `scorers` ```bash theme={null} curl -sS "$LITMUS_URL/v0/orgs/$LITMUS_ORG/scorers" \ -H "Authorization: Bearer $LITMUS_TOKEN" ``` ## `POST /v0/root` One sentence in; the path that was chosen and what each step returned. | | | | - | - | | Token scope | `read` (any org token) | | Org comes from | `org` in the JSON body; filled in from your token when you leave it out | | Policy row | `apitokens.POLICY["/v0/root"]["POST"]` | | CLI | — | **Body fields the handler or its helper reads:** `confirm`, `context`, `converse`, `greet`, `human`, `looking_at`, `model`, `org`, `path`, `paths`, `request`, `slots`, `stream`, `thread`, `uploaded`, `via`, `viewer` **Response fields written in the helper the handler answers through:** `asked`, `org`, `outcome`, `reply`, `routing`, `steps` (or a stream of server-sent events) This call changes state in the org. ```bash theme={null} curl -sS -X POST "$LITMUS_URL/v0/root" \ -H "Authorization: Bearer $LITMUS_TOKEN" \ -H "Content-Type: application/json" \ -d '{"confirm": "<confirm>", "context": "<context>", "converse": "<converse>", "greet": "<greet>", "human": "<human>", "looking_at": "<looking_at>", "model": "<model>", "path": "<path>", "paths": "<paths>", "request": "<request>", "slots": "<slots>", "stream": "<stream>", "thread": "<thread>", "uploaded": "<uploaded>", "via": "<via>", "viewer": "<viewer>"}' ``` ## `GET /v0/root` The paths this surface holds, and the record of what has been asked. | | | | - | - | | Token scope | `read` (any org token) | | Org comes from | the `org` query parameter; forced to your token's org | | Policy row | `apitokens.POLICY["/v0/root"]["GET"]` | | CLI | — | **Query parameters the handler or its helper reads:** `limit`, `org`, `readiness` **Response fields written in the handler:** `answers_kept_since`, `journal_file`, `org`, `paths`, `readiness`, `requests`, `tally`, `threads` ```bash theme={null} curl -sS "$LITMUS_URL/v0/root?org=$LITMUS_ORG" \ -H "Authorization: Bearer $LITMUS_TOKEN" ``` ## `GET /v0/root/threads/{tid}` One conversation, every turn, with the answers that were kept. | | | | - | - | | Token scope | `read` (any org token) | | Org comes from | the `org` query parameter; forced to your token's org | | Policy row | `apitokens.POLICY["/v0/root/threads/{tid}"]["GET"]` | | CLI | — | **Path parameters:** `tid` **Query parameters the handler or its helper reads:** `org` **Response:** JSON: a stored record or a helper's result, answered as it is, so its keys are not written in the source and none are listed. ```bash theme={null} curl -sS "$LITMUS_URL/v0/root/threads/$TID?org=$LITMUS_ORG" \ -H "Authorization: Bearer $LITMUS_TOKEN" ``` ## `PATCH /v0/root/threads/{tid}` Rename a conversation. `title` in the body; the org is the token's. The derived title is replaced and the thread remembers it was renamed, so a later turn does not overwrite what a person chose. | | | | - | - | | Token scope | `read` (any org token) | | Org comes from | `org` in the JSON body; filled in from your token when you leave it out | | Policy row | `apitokens.POLICY["/v0/root/threads/{tid}"]["PATCH"]` | | CLI | — | **Path parameters:** `tid` **Body fields the handler or its helper reads:** `org`, `title` **Response:** JSON: a stored record or a helper's result, answered as it is, so its keys are not written in the source and none are listed. This call changes state in the org. ```bash theme={null} curl -sS -X PATCH "$LITMUS_URL/v0/root/threads/$TID" \ -H "Authorization: Bearer $LITMUS_TOKEN" \ -H "Content-Type: application/json" \ -d '{"title": "<title>"}' ``` ## `DELETE /v0/root/threads/{tid}` Delete a conversation and every turn in it. There is no undo and no archive: the file is removed. The journal line each turn wrote stays, because it is the routing record and holds no answer. | | | | - | - | | Token scope | `read` (any org token) | | Org comes from | the `org` query parameter; forced to your token's org | | Policy row | `apitokens.POLICY["/v0/root/threads/{tid}"]["DELETE"]` | | CLI | — | **Path parameters:** `tid` **Query parameters the handler or its helper reads:** `org` **Response fields written in the handler:** `deleted`, `org` This call changes state in the org. ```bash theme={null} curl -sS -X DELETE "$LITMUS_URL/v0/root/threads/$TID?org=$LITMUS_ORG" \ -H "Authorization: Bearer $LITMUS_TOKEN" ``` This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.