403.
Sources
22 tools. List only these with?chapters=sources on the server URL. Routes: Sources.
| Tool | Route | Scope | What it does |
|---|---|---|---|
connectors_get | GET /v0/orgs/{orgtag}/connectors | read | GET /v0/orgs/{org}/connectors: the catalog, per category, with each standard connector’s inherited eval set and the computed promote-to-VIP flag (connectorcatalog). |
connectors_post | POST /v0/orgs/{orgtag}/connectors | push | POST /v0/orgs/{org}/connectors with {connectors: [...], by}: an idempotent upsert of catalog rows. |
connector_patch | PATCH /v0/orgs/{orgtag}/connectors/{cid} | push | PATCH /v0/orgs/{org}/connectors/{cid} with the fields to change and by — a tier, a new version (marks a rerun due), a regression rate. |
connections_rotations_get | GET /v0/orgs/{orgtag}/connections/rotations | read | This org’s rotation record: when every link was turned over, and by whom. |
connections_rotate_post | POST /v0/orgs/{orgtag}/connections/rotations | push | Turn over EVERY live connector URL this org holds. |
schedules_list | GET /v0/orgs/{orgtag}/schedules | read | This org’s schedules. |
schedules_post | POST /v0/orgs/{orgtag}/schedules | push | Declare a cadence: {kind, surface, cadence, anchor, …}. |
schedules_get | GET /v0/orgs/{orgtag}/schedules/{sid} | read | One schedule. |
schedules_patch | PATCH /v0/orgs/{orgtag}/schedules/{sid} | push | Change a schedule’s cadence, anchor, label or pinned spec. |
schedules_delete | DELETE /v0/orgs/{orgtag}/schedules/{sid} | push | Stop a schedule. |
sources_list | GET /v0/orgs/{orgtag}/sources | read | This org’s sources. |
sources_post | POST /v0/orgs/{orgtag}/sources | push | Connect a source: {kind, surface, external_id, auth?, cadence?, consent?}. |
sources_datadog_oauth_start | GET /v0/orgs/{orgtag}/sources/datadog/oauth/start | admin | Start Connect Datadog: ?by, site, service, surface, consent=granted, data_access, ml_app?, requests_ml_app?, requests_select?, label?, cadence? → 302 to Datadog’s consent page, the URL also in the body as authorize_url for the console’s server-side hop. |
sources_datadog_oauth_callback | GET /v0/orgs/{orgtag}/sources/datadog/oauth/callback | admin | Finish Connect Datadog: ?code, state, domain?, by (or ?error, state). |
sources_datadog_connect | POST /v0/orgs/{orgtag}/sources/datadog/connect | admin | Connect (or reconnect) this org’s Datadog from two keys and a site. |
sources_get | GET /v0/orgs/{orgtag}/sources/{sid} | read | One source. |
sources_patch | PATCH /v0/orgs/{orgtag}/sources/{sid} | push | Change a source’s cadence, consent, credential names, label, evidence, or the record of its last sync. |
sources_delete | DELETE /v0/orgs/{orgtag}/sources/{sid} | admin | Disconnect a source. |
sources_sync | POST /v0/orgs/{orgtag}/sources/{sid}/sync | push | Pull this source now: POST /v0/orgs/{org}/sources/{sid}/sync. |
sources_pulled | POST /v0/orgs/{orgtag}/sources/{sid}/pulled | push | Record a finished LLM Observability pull: POST …/sources/{sid}/pulled. |
sources_labels | POST /v0/orgs/{orgtag}/sources/{sid}/labels | push | A label stream’s batch, from an operator’s pull: POST …/sources/{sid}/labels with {format?, name?, items}. |
connections | GET /v0/connections | read | One org’s minted connections: world, urls, and whether each is live. |
Evidence
44 tools. List only these with?chapters=evidence on the server URL. Routes: Evidence.
| Tool | Route | Scope | What it does |
|---|---|---|---|
record_feedback · shaped | POST /v0/orgs/{orgtag}/feedback | push | {text, author | by, about?: {kind, id, index?}, polarity?, context?, seen?: {head, route?}, origin?, origin_source?, fields?, filed_by?}. |
failure_mode_proposals · shaped | GET /v0/orgs/{orgtag}/feedback | read | This org’s notes, oldest first. |
make_failure_modes · shaped | POST /v0/orgs/{orgtag}/feedback | push | {text, author | by, about?: {kind, id, index?}, polarity?, context?, seen?: {head, route?}, origin?, origin_source?, fields?, filed_by?}. |
reports_ledger · shaped | GET /v0/orgs/{orgtag}/feedback/ledger | read | GET /v0/orgs/{org}/feedback/ledger — the feedback sheet’s reports against Datadog’s retention: how many are read, waiting on the Datadog pull, past retention, not found or undated, when the oldest waiting session leaves Datadog (a UTC date), and the line the Sources page prints. |
reconvert_evidence · shaped | POST /v0/orgs/{orgtag}/evidence/{eid}/reconvert | push | Re-convert one stored Datadog record through today’s converter: POST /v0/orgs/{org}/evidence/{eid}/reconvert, {source?, by?}. |
evidence_post | POST /v0/evidence | push | Store one evidence record: POST /v0/evidence. |
evidence_list | GET /v0/evidence | read | ?org= narrows to one customer. |
corpus_post | POST /v0/orgs/{orgtag}/corpus | push | POST /v0/orgs/{org}/corpus with {items: [{text, provenance: {site, date, url}, licence}], licence?, site?} — short public requests about personal-agent use, stored as text and provenance only. |
pipeline_post | POST /v0/orgs/{orgtag}/pipeline | push | Run the whole pipeline for one org+surface as a job. |
pipeline_get | GET /v0/orgs/{orgtag}/pipeline | read | The org’s pipeline runs, newest first — the status row the console shows. |
evidence_get | GET /v0/evidence/{eid} | read | ?summary=1 returns a PREVIEW instead of the record. |
evidence_assign | POST /v0/evidence/{eid}/assign | push | Set the org/surface an existing evidence record belongs to. |
evidence_toolspec | GET /v0/evidence/{eid}/toolspec | read | Per-tool parameter value spaces for one source — the mock spec. |
evidence_attributes_post | POST /v0/evidence/{eid}/attributes | push | File per-trace EXTRACTED attributes (X55) beside the record. |
evidence_attributes_get | GET /v0/evidence/{eid}/attributes | read | The stored per-trace attributes, as stored. |
evidence_extract | POST /v0/evidence/{eid}/extract | push | Kick off attribute extraction for one record. |
evidence_extraction_get | GET /v0/evidence/{eid}/extraction | read | What a document contributed, as pinned: the current pin in full and every version it has had; ?pin=<id> reads one version, superseded ones included. |
evidence_re_extract | POST /v0/evidence/{eid}/re-extract | push | Re-read a document (or every document of a corpus, and redraw its mapping). |
facets_post | POST /v0/facets | push | File an induced facet space. |
facets_list | GET /v0/facets | read | ?org= narrows to one customer, exactly as evidence does. |
facets_get | GET /v0/facets/{fid} | read | No description. |
evidence_match | POST /v0/evidence/{eid}/match | push | How many runs match a selection — and what each next value would cost. |
worldplan_post | POST /v0/evidence/{eid}/worldplan | push | Mint a world plan from an attribute selection over one source. |
worldplan_coverage | POST /v0/evidence/{eid}/worldplans/coverage | push | Mint k worlds covering the corpus’s attribute space — the default mint. |
after_ingest_get | GET /v0/orgs/{orgtag}/after-ingest | read | GET /v0/orgs/{org}/after-ingest?since=<epoch s> — what the org’s last pass found: its one-line summary and counts, the open proposals with their measured reasons, the starters it wrote, the tally per proposal kind, and — with since, a member’s last visit — the operations first seen after it. |
after_ingest_post | POST /v0/orgs/{orgtag}/after-ingest | push | POST /v0/orgs/{org}/after-ingest — run the pass now, the one an ingestion queues. |
after_ingest_accept | POST /v0/orgs/{orgtag}/after-ingest/proposals/{pid}/accept | push | POST /v0/orgs/{org}/after-ingest/proposals/{pid}/accept — the one press. |
after_ingest_dismiss | POST /v0/orgs/{orgtag}/after-ingest/proposals/{pid}/dismiss | push | POST …/after-ingest/proposals/{pid}/dismiss — not now. |
org_changes_get | GET /v0/orgs/{orgtag}/changes | read | GET /v0/orgs/{org}/changes?since=<epoch s>&page=home|ontology — what the org’s standing pass changed after since (a member’s last visit; absent is every kept line), as lines newest first: what moved in the ontology, the sessions it pulled, each sandbox it re-drew or put back, the tasks it generated and what waits on tomorrow’s budget. |
feedback_retire | DELETE /v0/orgs/{orgtag}/feedback/{fid} | admin | {by?} — retire one note: retired_at and retired_by on its evidence row, and every list, the derive pass and the run verdicts skip it from now on. |
feedback_derive | POST /v0/orgs/{orgtag}/feedback/derive | push | {limit?} — ONE model pass over the notes no earlier pass read, writing PROPOSALS (dimension, criterion_kind, review_rule, generator_gate, analyzer_detector) into the proposed axes built, each citing the notes that motivate it and naming the stage it would change. |
statements_list | GET /v0/orgs/{orgtag}/statements | read | GET /v0/orgs/{org}/statements — the statements on record (never their words; ?id= returns one with its text) and the stated-buckets proposals made from them, any status. |
statements_post | POST /v0/orgs/{orgtag}/statements | push | {text, author | by, origin: upload|paste|home, file_name?, derive?} — store the words verbatim (tier human), then, unless derive is false, run the ONE derive pass on them: its estimate is said before the call and its spend after, on the answer. |
statements_derive | POST /v0/orgs/{orgtag}/statements/derive | push | {statement?} — ONE model pass over one statement no earlier pass read (the one named, else the oldest): at most one stated-buckets proposal, each bucket with its label, one-sentence profile and the sentence of the source it came from. |
labels_get | GET /v0/orgs/{orgtag}/labels | read | GET /v0/orgs/{org}/labels[?surface=]: the org’s labels (ids, kinds, whether each has a correction and what it became), per-stream counts, the production-only opt-in, and the superset guard over the standing tasks. |
labels_post | POST /v0/orgs/{orgtag}/labels | push | The webhook: POST /v0/orgs/{org}/labels with {surface, labels: [{trace, polarity, correction?}]} (or one label’s fields at the top). |
labels_policy_get | GET /v0/orgs/{orgtag}/labels/policy | read | GET /v0/orgs/{org}/labels/policy: whether the org has said that a production-only dataset is what it wants, who said it and when. |
labels_policy_put | PUT /v0/orgs/{orgtag}/labels/policy | push | PUT /v0/orgs/{org}/labels/policy with {production_only, by}: the org’s answer to whether a production-only dataset is what it wants. |
corpus_get | GET /v0/orgs/{orgtag}/corpus | read | GET /v0/orgs/{org}/corpus: the corpus’s counts by licence and site, the hand-set use cases its items READ as (the keyword rule), the org’s public-demand toggle, and every candidate value with its status. |
corpus_derive | POST /v0/orgs/{orgtag}/corpus/derive | push | POST /v0/orgs/{org}/corpus/derive with {limit?} — ONE model pass over the items no earlier pass read, clustering them into use cases and phrasing registers the hand-set axes do not name. |
corpus_policy_get | GET /v0/orgs/{orgtag}/corpus/policy | read | GET /v0/orgs/{org}/corpus/policy: whether public demand shapes this org’s datasets, who said so and when. |
corpus_policy_put | PUT /v0/orgs/{orgtag}/corpus/policy | push | PUT /v0/orgs/{org}/corpus/policy with {public_demand, by}: the org’s answer to whether accepted corpus values are dealt beside the hand-set situations on its generations. |
org_records | GET /v0/orgs/{orgtag}/records | read | Every record this org holds, by kind: the ids of each kind a copy takes, and a count with its reason for each kind that stays behind. |
org_record_get | GET /v0/orgs/{orgtag}/records/{kind}/{rid} | read | One record of a kind store.KINDS copies, exactly as stored. |
Ontology
27 tools. List only these with?chapters=ontology on the server URL. Routes: Ontology.
| Tool | Route | Scope | What it does |
|---|---|---|---|
org_scenarios_get | GET /v0/orgs/{orgtag}/situations | read | GET /v0/orgs/{org}/situations. |
org_readiness_get | GET /v0/orgs/{orgtag}/criteria/readiness | read | GET /v0/orgs/{org}/criteria/readiness[?agent=]. |
dimension_value_profile_post | POST /v0/orgs/{orgtag}/dimensions/values/{value_id}/profile | push | POST …/dimensions/values/{value_id}/profile with any of weight, profile, jobs, personas, and by (valueprofile.set_profile). |
dimensions_persona_draw | POST /v0/orgs/{orgtag}/dimensions/personas/{value}/draw | push | POST /v0/orgs/{org}/dimensions/personas/{value}/draw with {by, surface?}: a member’s press on a persona’s card — its sandbox’s generate job now, the day’s budget notwithstanding (a press, as accepting a proposal is), on the pass’s ledger like any draw. |
dimensions_personas_get | GET /v0/orgs/{orgtag}/dimensions/personas | read | GET|POST /v0/orgs/{org}/dimensions/personas. |
dimensions_personas_post | POST /v0/orgs/{orgtag}/dimensions/personas | push | GET|POST /v0/orgs/{org}/dimensions/personas. |
dimensions_personas_from_document | POST /v0/orgs/{orgtag}/dimensions/personas-from-document | push | POST /v0/orgs/{org}/dimensions/personas-from-document with {source}: the personas the source’s documents’ PINNED extractions cite, written as the named persona axis (valueprofile.from_document). |
org_coverage_get | GET /v0/orgs/{orgtag}/coverage | read | GET /v0/orgs/{org}/coverage: the persona x job matrix — which cells hold tasks and verifiers, which a person marked as a job of that persona and are still gaps (valueprofile.coverage). |
ontology_get | GET /v0/orgs/{orgtag}/ontology | read | One graph’s snapshot: ?surface= names it (or the org’s only surface is resolved), ?legacy=1 addresses the pre-keying per-org graph. |
ontology_compile_post | POST /v0/orgs/{orgtag}/ontology/compile | push | Compile one surface’s graph from its corpus. |
ontology_pins_get | GET /v0/orgs/{orgtag}/ontology/pins | read | The pinned model draws of one surface’s ontology: every pin it has had (?surface=), or one by id (?pin=oxp_…), current or superseded — so a sandbox’s schema_pin can be read back to the draw it came from. |
ontology_restore_post | POST /v0/orgs/{orgtag}/ontology/restore | push | Restore one SURFACE’s revision; the answer names the surface. |
ontology_node_get | GET /v0/orgs/{orgtag}/ontology/nodes/{node_id:path} | read | One node of one graph, with the edges that touch it and the evidence behind both; the graph is addressed as ontology_get addresses it. |
surface_ontology_derive | POST /v0/orgs/{orgtag}/surfaces/{surface}/ontology/derive | push | POST /v0/orgs/{org}/surfaces/{surface}/ontology/derive — store the ontology of the version this surface serves NOW under its sfv-. |
surface_ontology_versions | GET /v0/orgs/{orgtag}/surfaces/{surface}/ontology/versions | read | GET …/surfaces/{surface}/ontology/versions — the switcher’s list: every stored version with the agent versions that expose it, and the current one (a pointer, read live) whether or not it was derived. |
surface_ontology_diff | GET /v0/orgs/{orgtag}/surfaces/{surface}/ontology/diff | read | GET …/surfaces/{surface}/ontology/diff?a=sfv-…&b=sfv-… — what moved between two stored versions: operations added / removed / renamed, relation edges moved, surface-half dimension values moved. |
surface_document_relations | POST /v0/orgs/{orgtag}/surfaces/{surface}/relations/document | push | {source?} — one model pass over the org’s document, written as the edges it states between this surface’s services, objects and data. |
dimension_value_parent_post | POST /v0/orgs/{orgtag}/dimensions/values/{value_id}/parent | push | A PERSON says which value this one belongs to. |
dimension_merge_spellings_post | POST /v0/orgs/{orgtag}/dimensions/merge-spellings | push | Fold a use case the org holds in two spellings into the first one written. |
dimension_value_hero_post | POST /v0/orgs/{orgtag}/dimensions/values/{value_id}/hero | push | Mark a use case as one of the org’s HERO use cases, or unmark it. |
dimension_value_decide_post_accept | POST /v0/orgs/{orgtag}/dimensions/values/{value_id}/accept | push | {by, reason?} — accept or dismiss a proposed axis (dx_…) or one of its values (dv_…). |
dimension_value_decide_post_dismiss | POST /v0/orgs/{orgtag}/dimensions/values/{value_id}/dismiss | push | {by, reason?} — accept or dismiss a proposed axis (dx_…) or one of its values (dv_…). |
ontology_revision_restore | POST /v0/orgs/{orgtag}/ontology/revisions/{rev}/restore | push | POST /v0/orgs/{org}/ontology/revisions/{rev}/restore — put back what an applied ontology diff changed: the agent’s layer slots as the revision found them. |
org_dimensions_get | GET /v0/orgs/{orgtag}/dimensions | read | Every axis this org’s representation carries, with its provenance. |
org_dimensions_derive | POST /v0/orgs/{orgtag}/dimensions/derive | push | {source} — one model pass over one source, written as dimension values. |
dims_post | POST /v0/orgs/{orgtag}/dimensions | admin | Declare a dimension: {field, values, label?, description?, added_by?}, answered with the stored record. |
dims_delete | DELETE /v0/dimensions/{did} | admin | No description. |
Sandboxes
88 tools. List only these with?chapters=sandboxes on the server URL. Routes: Sandboxes.
| Tool | Route | Scope | What it does |
|---|---|---|---|
list_environments · shaped | GET /v0/orgs/{orgtag}/surfaces | read | Every surface this org serves, with the environments it hosts. |
generate_tasks · shaped | POST /v0/orgs/{orgtag}/environments/{env_id}/tasks | push | Generate count more tasks on this environment and APPEND them; or, with {goal: {mode, n, voice?, max_turns?, usd_cap?, personas?, fallback?, from_sessions?}}, have the Litmus agent write n tasks that reproduce one failure mode, as a job, the rule writing only as its labelled fallback. |
held_tasks · shaped | GET /v0/orgs/{orgtag}/environments/{env_id}/tasks | read | This environment’s tasks, each with its verifier and what it decides, and family on a task a rule-written family wrote. |
hold_task · shaped | POST /v0/orgs/{orgtag}/environments/{env_id}/review/hold | push | A PERSON holds one task: {index, reason, grader, grader_kind?}. |
retire_tasks · shaped | POST /v0/orgs/{orgtag}/environments/{env_id}/tasks/retire | admin | Retire a range of this environment’s tasks, by index. |
restore_tasks · shaped | POST /v0/orgs/{orgtag}/environments/{env_id}/tasks/restore | admin | Put a retired range back. |
bundles | GET /v0/bundles | read | The gallery, as whoever is asking may see it. |
bundle_manifest | GET /v0/bundles/{bundle_id}/manifest | read | The bundle’s tool registry — exactly what an agent sees on tools/list. |
bundle_axes | GET /v0/bundles/{bundle_id}/axes | read | The admitted parameter axes for a bundle — the knobs a world can be designed with (deterministic; same machinery as profiles). |
apidoor_post | POST /v0/orgs/{orgtag}/apidoor | push | Compile and serve an org’s API door from evidence the workspace holds. |
apidoor_get | GET /v0/orgs/{orgtag}/apidoor | read | What API doors this org has: bundle id, the worlds served, and for each world the base URL a wrapper points its upstream at beside the gaps URL it must be handed with. |
sandbox_door_post | POST /v0/orgs/{orgtag}/sandboxes/{sid}/door | push | Mint a hosted door for this sandbox: its API mock on a public URL. |
sandbox_door_get | GET /v0/orgs/{orgtag}/sandboxes/{sid}/door | read | This sandbox’s hosted doors, oldest first, revoked ones included with their revoked_at: a door somebody just killed has to be seen dead. |
sandbox_door_delete | DELETE /v0/orgs/{orgtag}/sandboxes/{sid}/door/{tid} | push | Revoke one door. |
sandbox_door_reset | POST /v0/orgs/{orgtag}/sandboxes/{sid}/door/{tid}/reset | push | Put the door’s world back to the state it was drawn at — the same answer <door>/_litmus/reset gives the token holder. |
sandbox_door_tally | GET /v0/orgs/{orgtag}/sandboxes/{sid}/door/{tid}/tally | read | What this door has answered: counts by route template and status, unserved requests by method, how many the limit refused and how many resets. |
served_repo_post | POST /v0/orgs/{orgtag}/served-repo | push | Connect a customer’s own MCP server as a served_repo surface. |
served_repo_get | GET /v0/orgs/{orgtag}/served-repo | read | This org’s served repositories: pins and counts only, each with its state — connecting, connected or refused. |
served_repo_one | GET /v0/orgs/{orgtag}/served-repo/{rid} | read | One registration’s status read: the listing’s row for it — connecting with the stages finished and their seconds, connected with its counts, or refused with the redacted refusal. |
served_repo_read | GET /v0/orgs/{orgtag}/served-repo/{rid}/read | read | One served version’s client read: the route keys and field_codes the door is handed for it, read over its cached checkout (or the read stored at registration) — what a try’s door serves codes by, seen without a run. |
environment_task_turns_post | POST /v0/orgs/{orgtag}/environments/{env_id}/tasks/{index}/turns | push | POST …/environments/{env}/tasks/{index}/turns with {turns, by, note?}: scripted follow-up user turns on one task (tasktags.set_turns). |
surface_representation | GET /v0/orgs/{orgtag}/surfaces/{surface}/representation | read | This surface’s representation — what the BUILDER offers and what the ONTOLOGY PAGE shows. |
surface_progress | GET /v0/orgs/{orgtag}/surfaces/{surface}/progress | read | GET …/surfaces/{surface}/progress — where the agent started on this system and what it took to get to the last mile: every agent version with a landed run on the system or a declared version of it (variant_of), oldest first; per version, per golden path, met / not met / not decided / untested at the weakest tier of its verdicts, with the first version that met it; per known failure mode, the version at which it stopped reproducing; per version, what changed (the run’s label, the agent version’s notes, or the system’s declared change). |
surface_world_redraw | POST /v0/orgs/{orgtag}/surfaces/{surface}/world/redraw | push | POST /v0/orgs/{orgtag}/surfaces/{surface}/world/redraw. |
surface_rebuild_post | POST /v0/orgs/{orgtag}/surfaces/{surface}/rebuild | push | Rebuild a derived surface version in place under the current compiler. |
surface_world_revert | POST /v0/orgs/{orgtag}/surfaces/{surface}/world/revert | push | POST /v0/orgs/{orgtag}/surfaces/{surface}/world/revert. |
surface_supersede_post | POST /v0/orgs/{orgtag}/surfaces/{surface}/supersede | push | Mark one surface superseded by another this org serves. |
surface_unsupersede_post | POST /v0/orgs/{orgtag}/surfaces/{surface}/unsupersede | push | Lift a supersession: the surface is listed by default again. |
surface_retire_post | POST /v0/orgs/{orgtag}/surfaces/{surface}/retire | push | Take one surface, and every state of it, off this org’s pages. |
surface_restore_post | POST /v0/orgs/{orgtag}/surfaces/{surface}/restore | push | Put a retired surface back on this org’s pages. |
org_retired_get | GET /v0/orgs/{orgtag}/retired | read | THE OPERATOR VIEW: what this org has retired, and what it was. |
environment_revisions_list | GET /v0/orgs/{orgtag}/environments/{env_id}/revisions | read | GET /v0/orgs/{org}/environments/{env_id}/revisions — this sandbox’s revisions, newest first, at most afteringest.REVISIONS_LISTED, each as GET …/revisions/{rev} answers it, with n (every one it holds). |
environment_revision_get | GET /v0/orgs/{orgtag}/environments/{env_id}/revisions/{rev} | read | GET /v0/orgs/{org}/environments/{env_id}/revisions/{rev} — one revision as it stands: its kind and state, the audit’s before/after counts (rows that contradict each other, realism flags) with what came out worse and whether the re-draw held, what it kept, and the jobs it ran with each one’s cost — priced once ended, its estimate while running. |
environment_revision_restore | POST /v0/orgs/{orgtag}/environments/{env_id}/revisions/{rev}/restore | push | POST /v0/orgs/{org}/environments/{env_id}/revisions/{rev}/restore — put back what an applied act replaced: the draw, the tasks and the verifiers kept under rev. |
environments_generate | POST /v0/orgs/{orgtag}/environments/generate | push | Generate N environments for one selection of dimension values. |
environments_for_criteria | POST /v0/orgs/{orgtag}/environments/for-criteria | push | POST /v0/orgs/{org}/environments/for-criteria. |
environment_retire_post | POST /v0/orgs/{orgtag}/environments/{env_id}/retire | push | Take one DRAWN STATE off this org’s pages. |
environment_restore_post | POST /v0/orgs/{orgtag}/environments/{env_id}/restore | push | Put a retired drawn state back on this org’s pages. |
environment_surface_key_get | GET /v0/orgs/{orgtag}/environments/{env_id}/surface-key | read | GET /v0/orgs/{org}/environments/{env_id}/surface-key — the surface key run against this sandbox: how many of the fixture’s declared behaviours it reproduces (present), does not (absent), and could not be asked about (undecidable), each with the probes behind it, at tier deterministic. |
environment_tools_get | GET /v0/orgs/{orgtag}/environments/{env_id}/tools | read | GET /v0/orgs/{org}/environments/{env_id}/tools — the tools this sandbox serves, by name with a one-line description: what a no-data report’s “holds the rows” may name. |
environment_fidelity_get | GET /v0/orgs/{orgtag}/environments/{env_id}/fidelity | read | GET /v0/orgs/{org}/environments/{env_id}/fidelity — how much of the org’s real tool surface this sandbox serves: one row per tool of the code at the agent’s main, fields served / lacked / invented (each sourced code or sessions), enum values and relations lacked, unobserved for a tool the sessions never call; the fix list ranked by n_calls; terms.coverage and the line “serves X of N tool shapes by name (deterministic)”. |
environment_answer_shapes_post | POST /v0/orgs/{orgtag}/environments/{env_id}/fidelity/answer-shapes | push | POST /v0/orgs/{org}/environments/{env_id}/fidelity/answer-shapes — {dry_run?, model?}. |
environment_get | GET /v0/orgs/{orgtag}/environments/{env_id} | read | One environment and its realism block: how far its written requests sit from the real requests of what it was generated under — numbers, each with its tier and n, never a request. |
environment_task_checks_get | GET /v0/orgs/{orgtag}/environments/{env_id}/tasks/{index}/checks | read | WHAT RUNS FOR EACH CHECK ON ONE TASK, verbatim: the deterministic read as executed, or the judge’s full prompt as sent with the pinned model and the “asked twice” rule — and the last run’s verdicts on it, with the read’s result or both of the judge’s answers. |
environment_task_check_rehearse_post | POST /v0/orgs/{orgtag}/environments/{env_id}/tasks/{index}/checks/{n}/rehearse | push | RUN ONE CHECK ON ONE STORED TRIAL, AS A REHEARSAL: {trial}. |
environment_script_starter_get | GET /v0/orgs/{orgtag}/environments/{env_id}/scripts/starter | read | GET /v0/orgs/{org}/environments/{env}/scripts/starter → {script, from: {site, form}}: a .spec.ts written from the sign-up journey on this environment’s own household web — log in with the account’s own credential (a process.env line, never a value), fill the form, submit, expect the page the site answers with. |
environment_script_dry_run_post | POST /v0/orgs/{orgtag}/environments/{env_id}/scripts/dry-run | push | POST /v0/orgs/{org}/environments/{env}/scripts/dry-run {source} → the script parsed, compiled and replayed once in a real browser on this environment’s own world (drawn with its household web into a temp dir), its checks graded on the captured trace and on an empty one: {steps, checks, noop, facts, events, gate, compiled, parse: {refused}}. |
environment_scripts_get | GET /v0/orgs/{orgtag}/environments/{env_id}/scripts | read | GET /v0/orgs/{org}/environments/{env}/scripts → {scripts: [{task, name, title, sha256, checks_n, created_at}]}: the script tasks on this environment, never their source. |
environment_scripts_post | POST /v0/orgs/{orgtag}/environments/{env_id}/scripts | push | POST /v0/orgs/{org}/environments/{env}/scripts {source, name?} — a Playwright script made a task on this environment. |
environment_realism_get | GET /v0/orgs/{orgtag}/environments/{env_id}/realism | read | The human marks on this environment: the whole thing, and per task. |
environment_realism_post | POST /v0/orgs/{orgtag}/environments/{env_id}/realism | push | Mark this environment — or one task of it, or one table or column of its world — realistic / not / unsure. |
environment_world_refresh | POST /v0/orgs/{orgtag}/environments/{env_id}/world/refresh | push | POST /v0/orgs/{orgtag}/environments/{env_id}/world/refresh — re-snapshot. |
environment_world_table | GET /v0/orgs/{orgtag}/environments/{env_id}/world/table | read | GET /v0/orgs/{orgtag}/environments/{env_id}/world/table?table=[&limit=] |
environment_review_regenerate_post | POST /v0/orgs/{orgtag}/environments/{env_id}/review/regenerate | push | Re-author the HELD tasks with the failing reasons in the brief: {model?}. |
environment_revoice_post | POST /v0/orgs/{orgtag}/environments/{env_id}/revoice | push | Re-voice tasks: {voice, tasks: [index, …], model?, review?}. |
environment_rekey_post | POST /v0/orgs/{orgtag}/environments/{env_id}/rekey | push | Key this environment’s tasks to every criterion their keys decide: {}. |
environment_review_rereview_post | POST /v0/orgs/{orgtag}/environments/{env_id}/review/rereview | push | Re-review the HELD tasks on this environment: {indexes?, model?}. |
environment_review_correct_post | POST /v0/orgs/{orgtag}/environments/{env_id}/review/correct | push | A PERSON corrects one task’s expected fact: {index, fact, literal?, aliases?, fact_text?, note, grader, grader_kind?}. |
environment_regate_post | POST /v0/orgs/{orgtag}/environments/{env_id}/regate | push | Re-gate this environment’s standing tasks on the world it serves NOW. |
environment_reserve_post | POST /v0/orgs/{orgtag}/environments/{env_id}/reserve | push | POST /v0/orgs/{org}/environments/{env_id}/reserve — {surface: B}. |
environment_review_get | GET /v0/orgs/{orgtag}/environments/{env_id}/review | read | This environment’s review: five verdicts per task, and the counts. |
environment_review_post | POST /v0/orgs/{orgtag}/environments/{env_id}/review | push | Review every task on this environment: {model?}. |
environment_audit_get | GET /v0/orgs/{orgtag}/environments/{env_id}/audit | read | GET /v0/orgs/{orgtag}/environments/{env_id}/audit — the stored report. |
environment_audit_post | POST /v0/orgs/{orgtag}/environments/{env_id}/audit | push | POST /v0/orgs/{orgtag}/environments/{env_id}/audit — {model?}. |
worldplans_from_surface | POST /v0/orgs/{orgtag}/worldplans/from-surface | push | Generate tasks on a served surface: {bundle, count?, label?, model?}. |
worldplans_list | GET /v0/worldplans | read | One page of plans, newest first, with the count and the cursor. |
worldplans_get | GET /v0/worldplans/{wid} | read | No description. |
worldplans_patch | PATCH /v0/worldplans/{wid} | push | Edit a plan’s label or tasks — the primitives that make generated task sets auditable AND ownable: a person can rewrite a task, drop one, or trim an expectation, and the artifact is theirs from then on. |
worldplans_delete | DELETE /v0/worldplans/{wid} | push | No description. |
worldplan_harbor | GET /v0/worldplans/{wid}/harbor | read | One minted world as a Harbor dataset: a task directory per task, each with its own verifier. |
audit_sandboxes_list | GET /v0/orgs/{orgtag}/sandboxes | read | The org’s auditable sandboxes, each carrying the representation that produced it, plus the human realism roll-up over all of them. |
audit_sandboxes_post | POST /v0/orgs/{orgtag}/sandboxes | push | Mint one sandbox from a named representation. |
realism_packet_get | GET /v0/orgs/{orgtag}/realism/packet | read | GET /v0/orgs/{orgtag}/realism/packet?per=&limit=&grader=&audience= — the review packet: tasks for a person to mark realistic / not / unsure, each with that grader’s own current mark and the cell’s n and graders. |
realism_get | GET /v0/orgs/{orgtag}/realism | read | The per-hypothesis realism roll-up, and the marks behind it. |
realism_post | POST /v0/orgs/{orgtag}/realism | push | Mark one record, or a whole sandbox, “would pass as real” / “would not”. |
audit_sandbox_get | GET /v0/sandboxes/{sid} | read | One sandbox in full — every table’s rows, the held-out real records beside them, and the marks already given on it. |
audit_sandbox_delete | DELETE /v0/sandboxes/{sid} | push | Remove one sandbox. |
surfaces_post | POST /v0/surfaces | push | Traces from an unknown tool surface in; a gated, served mock out. |
worlds_post | POST /v0/worlds | push | No description. |
worlds_get | GET /v0/worlds | read | The served fleets, scoped by _org_scope like every listing. |
world_check | POST /v0/worlds/{bundle_id}/{world}/check | read | Run a verifier against a served world’s CURRENT state. |
world_reset | POST /v0/worlds/{bundle_id}/{world}/reset | push | POST /v0/worlds/{bundle_id}/{world}/reset — the served copy back to the pristine mint. |
world_state | GET /v0/worlds/{bundle_id}/{world}/state | read | GET /v0/worlds/{bundle_id}/{world}/state — the served world’s CURRENT state, every table, through the control plane and never through the tools under test: the independent read path for a runner grading a hosted world. |
world_peek | GET /v0/worlds/{bundle_id}/{world}/peek | read | GET /v0/worlds/{bundle_id}/{world}/peek — the served world, bounded. |
world_tools | GET /v0/worlds/{bundle_id}/{world}/tools | read | GET /v0/worlds/{bundle_id}/{world}/tools — the tool specs this world serves. |
world_call | POST /v0/worlds/{bundle_id}/{world}/call | push | POST /v0/worlds/{bundle_id}/{world}/call {tool, args} — call one tool. |
tokens_post | POST /v0/tokens | push | No description. |
tokens_delete | DELETE /v0/tokens/{tid} | push | DELETE /v0/tokens/{tid} — revoke one world connector token. |
Agents
38 tools. List only these with?chapters=agents on the server URL. Routes: Agents.
| Tool | Route | Scope | What it does |
|---|---|---|---|
list_agents · shaped | GET /v0/orgs/{orgtag}/agents | read | This org’s agents: the connected rows, then the built-in harnesses. |
connect_agent · shaped | POST /v0/orgs/{orgtag}/agents | push | Connect an agent: {kind, name, version?, command | url | repo fields, token_env?, env?, model?, notes?}. |
add_agent_version · shaped | POST /v0/orgs/{orgtag}/agents/{aid}/versions | push | Add a version of a repository agent from a ref: POST …/agents/{aid}/versions, body {ref, by?} — or of an http agent by its label: {version, url?, by?}, the endpoint only when it changed. |
onboard_version · shaped | POST /v0/orgs/{orgtag}/agents/{aid}/onboard | push | POST /v0/orgs/{org}/agents/{aid}/onboard — {version?, against?, surface?, tasks?, allow_cannot?, environments?}, and for a surface agent the change to its surface: {description_overrides?, added_tools?, removed_tools?, redeclared_tools?, origin?, by?} — an added tool is {name, description, input_schema, behaviour}, compiled deterministically from its declared behaviour; a redeclared tool is {name, behaviour, description?}, an EXISTING tool whose function is replaced by its declared one (e.g. |
track_server_ref · shaped | POST /v0/orgs/{orgtag}/agents/{aid}/server-refs | push | POST /v0/orgs/{org}/agents/{aid}/server-refs — {ref, behaviours?, by?, supersedes?}. |
set_main_version · shaped | POST /v0/orgs/{orgtag}/agents/{aid}/main | push | POST /v0/orgs/{org}/agents/{aid}/main — {version, by?}. |
version_changes · shaped | GET /v0/orgs/{orgtag}/agents/{aid}/versions/{version:path}/changes | read | GET /v0/orgs/{org}/agents/{aid}/versions/{version}/changes — what a tracked version of the server repository changes, by file, as it was read once and kept on the version record: {version, changes?: {base, head, files, names, areas: {tools, handlers, skills, registration, other}, by_area, statuses, additions, deletions, reader, tier, line}}. |
read_version_changes · shaped | POST /v0/orgs/{orgtag}/agents/{aid}/versions/{version:path}/changes | push | POST /v0/orgs/{org}/agents/{aid}/versions/{version}/changes — read the changed-file list between a tracked version’s fork point and its head through the org’s installed GitHub App (GET /repos/{o}/{r}/compare/ {base}...{head}, a token minted for that one repository with contents: read, never returned or logged) and keep it on the version record. |
touched_coverage · shaped | GET /v0/orgs/{orgtag}/agents/{aid}/touched-coverage | read | GET /v0/orgs/{org}/agents/{aid}/touched-coverage?version=&tools=&set=&group= — before any B run: which tools a surface change touches, how many of the set’s tasks call each, which production calls (when the Datadog grammar holds them), and the tasks to generate for the gap, priced. |
write_path_by_tool · shaped | GET /v0/orgs/{orgtag}/agents/{aid}/write-path | read | The agent’s write path by tool: GET /v0/orgs/{org}/agents/{aid}/write-path. |
count_write_path · shaped | POST /v0/orgs/{orgtag}/agents/{aid}/write-path | push | Count the agent’s write path by tool now: POST /v0/orgs/{org}/agents/{aid}/write-path. |
accept_write_path_mode · shaped | POST /v0/orgs/{orgtag}/agents/{aid}/write-path/accept | push | Record one proposal as a known failure mode: POST /v0/orgs/{org}/agents/{aid}/write-path/accept, {tool, by}. |
agents_first | POST /v0/orgs/{orgtag}/agents/first | push | The org’s first agent, written if it has none. |
agents_get | GET /v0/orgs/{orgtag}/agents/{aid} | read | One agent: its versions, the groups it ran, and where its traces are. |
agents_patch | PATCH /v0/orgs/{orgtag}/agents/{aid} | push | Change an agent, or pin a new version of it. |
agents_delete | DELETE /v0/orgs/{orgtag}/agents/{aid} | push | Disconnect an agent. |
agents_retire | POST /v0/orgs/{orgtag}/agents/{aid}/retire | push | Retire an agent, or restore one: POST …/agents/{aid}/retire, body {retired: bool, by?}. |
agents_probe | POST /v0/orgs/{orgtag}/agents/{aid}/probe | push | Ask the agent one fixture question, and store the shape of its answer. |
agents_smoke | POST /v0/orgs/{orgtag}/agents/{aid}/smoke | push | One trial of the agent on the fixture household: the connect step’s smoke. |
agents_discover | POST /v0/orgs/{orgtag}/agents/{aid}/discover | push | Ask a connected agent version a few tasks against a surface this org serves, and store which operations it used and who answered each. |
agents_discoveries | GET /v0/orgs/{orgtag}/agents/{aid}/discoveries | read | This agent’s discover runs, newest first. |
agents_primary_post | POST /v0/orgs/{orgtag}/agents/{aid}/primary | push | Make this agent the org’s primary: POST /v0/orgs/{org}/agents/{aid}/primary, body {by?}. |
agents_detect | POST /v0/orgs/{orgtag}/agents/{aid}/detect | push | Read the repository again and probe what it proposes. |
agents_scan | POST /v0/orgs/{orgtag}/agents/{aid}/scan | push | Scan for new versions of this agent now: POST /v0/orgs/{org}/agents/{aid}/scan, body {paths?, cadence?}. |
agents_server_repo | POST /v0/orgs/{orgtag}/agents/{aid}/server-repo | push | POST /v0/orgs/{org}/agents/{aid}/server-repo — {url, ref?, paths?, listing?, run?, setup?, upstream_env?}. |
agents_server_tools | GET /v0/orgs/{orgtag}/agents/{aid}/server-tools/{sha} | read | GET /v0/orgs/{org}/agents/{aid}/server-tools/{sha} — the tool definitions read from the server repository at one commit (main for main’s): {sha, reader, count, tools: [{name, description, input_schema, data_source?}]}, as a deterministic reader read them. |
agents_version_patch | PATCH /v0/orgs/{orgtag}/agents/{aid}/versions/{version} | push | Retire a version, or restore one: PATCH …/agents/{aid}/versions/{version}, body {retired: bool, by?}. |
agents_version_skills_post | POST /v0/orgs/{orgtag}/agents/{aid}/versions/{version:path}/skills | push | POST /v0/orgs/{org}/agents/{aid}/versions/{version}/skills — {ref, paths?}: register “main’s tools + this version’s skills”. |
agents_version_tool_texts_post | POST /v0/orgs/{orgtag}/agents/{aid}/versions/{version:path}/tool-texts | push | POST /v0/orgs/{org}/agents/{aid}/versions/{version}/tool-texts — {by, dry_run?}: carry a tracked version’s MCP tool texts onto the served API tools their data source names, as a STAND-IN beside a control. |
agent_env_list | GET /v0/orgs/{orgtag}/agent-env | admin | The names this org has set, who set each and when — never a value. |
agent_env_put | PUT /v0/orgs/{orgtag}/agent-env/{name} | admin | Set one value, {value}. |
agent_env_delete | DELETE /v0/orgs/{orgtag}/agent-env/{name} | admin | Remove one value. |
github_installations_post | POST /v0/orgs/{orgtag}/github/installations | push | Record a Litmus GitHub App installation on this org: {installation_id, account_login?}. |
github_installation_get | GET /v0/orgs/{orgtag}/github/installation | read | The installations this org references: {org, installations: [{installation_id, account, account_type}]}. |
github_installation_post | POST /v0/orgs/{orgtag}/github/installation | push | Reference, from this org, an installation an org the caller acts for already holds: {installation_id, from_org?}. |
github_repositories_get | GET /v0/orgs/{orgtag}/github/repositories | read | The repositories this org’s Litmus GitHub App installations cover: {org, installations: [{account, installation_id, repositories: [{full_name, clone_url, default_branch, private, pushed_at}], refusal}]}. |
github_refs_get | GET /v0/orgs/{orgtag}/github/refs | read | One covered repository’s branches and open pull requests: GET /v0/orgs/{org}/github/refs?repo=owner/name → {org, repo, account, installation_id, branches: [{name, sha}], pulls: [{number, title, head_ref, head_sha, base_ref}], pulls_refusal, refusal}. |
github_unrecorded_get | GET /v0/orgs/{orgtag}/github/unrecorded | read | Installations of the Litmus GitHub App that GitHub holds and no org has recorded, on the account ?owner= names: {org, owner, unrecorded: [{installation_id, account}]}. |
Runs
31 tools. List only these with?chapters=runs on the server URL. Routes: Runs.
| Tool | Route | Scope | What it does |
|---|---|---|---|
run_selection · shaped | POST /v0/orgs/{orgtag}/runs/start | push | Start a run of one minted world as a job: 202 + job id, poll it as a pipeline job. |
job_status · shaped | GET /v0/jobs/{job_id} | read | One job. |
compare_runs · shaped | GET /v0/orgs/{orgtag}/runs/compare | read | Two run groups side by side, per environment and task: ?a=<group>&b=<group>. |
runs_finish | POST /v0/orgs/{orgtag}/runs/finish | push | POST /v0/orgs/{org}/runs/finish {job, agent?} — land a stopped run’s finished trials as a run record. |
runs_groups_list | GET /v0/orgs/{orgtag}/runs/groups | read | This org’s run groups, newest first, each with its status and counts. |
runs_live | GET /v0/orgs/{orgtag}/runs/live | read | GET /v0/orgs/{org}/runs/live — every live run and group of this org, and those that ended in the last LIVE_RECENT_S, each as jobs_live answers it: the Runner’s Runs tab in one read. |
runs_providers_start | POST /v0/orgs/{orgtag}/runs/providers | push | POST …/runs/providers {environment, tasks?, pins?, trials, timeout_seconds?}. |
providers_get | GET /v0/orgs/{orgtag}/providers | read | GET …/providers: the pins a comparison can run, each with whether the workspace holds its provider’s key (by name) and whether its model has a price row. |
comparisons_list | GET /v0/orgs/{orgtag}/comparisons | read | GET …/comparisons: this org’s provider comparisons, newest first — id, when, the environment’s label, the pins by label, how many tasks, and the status their groups add up to. |
comparisons_get | GET /v0/orgs/{orgtag}/comparisons/{pcid} | read | GET …/comparisons/{id}: the pins side by side — per pin its status, trials by terminal state, met of valid, cost and cost per met trial, latency — and one row per task with a cell per pin (providercompare.read). |
runs_clean | GET /v0/orgs/{orgtag}/runs/clean | read | The kept clean run per dataset: for each environment, the most recent group the org’s PRIMARY agent ran that finished, landed every environment and has every trial valid. |
runs_last | GET /v0/orgs/{orgtag}/runs/last/{env} | read | A sandbox’s last landed run group, as its met / total. |
runs_group_get | GET /v0/orgs/{orgtag}/runs/group/{gid} | read | One group, rolled up per SUCCESS CRITERION — “how their surface did”. |
runs_group_versions | GET /v0/orgs/{orgtag}/runs/group/{gid}/versions | read | GET /v0/orgs/{org}/runs/group/{gid}/versions: the versions this group’s tasks can run on — every agent version of its agent and, for a surface agent, every surface version — each with runnable, how many tasks and environments it would run, and the refusal sentence runs/start {group, agent_version} would give. |
runs_group_recount_post | POST /v0/orgs/{orgtag}/runs/group/{gid}/recount | push | POST …/runs/group/{gid}/recount: count a landed group again against the LIVE review and keys. |
runs_group_recount_get | GET /v0/orgs/{orgtag}/runs/group/{gid}/recount | read | GET …/runs/group/{gid}/recount: every count of a group’s landed records, oldest first — the one each landed with, under its own stamp, then each re-count — with each count’s numbers and which one the pages read (read, the newest). |
runs_group_regrade_post | POST /v0/orgs/{orgtag}/runs/group/{gid}/regrade | push | POST …/runs/group/{gid}/regrade: re-grade a landed eval from its stored tries at the deterministic tier, with no agent and no judge. |
runs_group_regrade_get | GET /v0/orgs/{orgtag}/runs/group/{gid}/regrade | read | GET …/runs/group/{gid}/regrade: an eval’s verdict versions, read. |
runs_group_levels | GET /v0/orgs/{orgtag}/runs/group/{gid}/levels | read | GET /v0/orgs/{org}/runs/group/{gid}/levels: the group at three levels (the call works / the right call / the job got done), its failures grouped by a root cause a record names, and one improvement report per audience written by rule (runlevels). |
runs_post | POST /v0/orgs/{orgtag}/runs | push | Import one exported run bundle into this org. |
runs_list | GET /v0/orgs/{orgtag}/runs | read | This org’s imported runs, newest import first. |
runs_get | GET /v0/runs/{rid} | read | One imported run, whole: the bundle as litmus export wrote it, plus the org it was imported under (which record:runs:rid guards on). |
runs_publish | POST /v0/orgs/{orgtag}/runs/{rid}/publish | push | Publish one stored run’s transcripts to LangFuse. |
runs_trace_get | GET /v0/orgs/{orgtag}/runs/{rid}/trace | read | One trial of a stored run as a span tree, priced per span. |
runs_reanalyze | POST /v0/orgs/{orgtag}/runs/{rid}/reanalyze | push | Re-analyse one stored run from the directory already on disk. |
jobs_list | GET /v0/jobs | read | ?org= narrows to one customer, exactly as evidence does; ?org=* is every org, asked for on purpose (_org_scope). |
jobs_stop | POST /v0/jobs/{job_id}/stop | push | Stop a run or a run group: POST /v0/jobs/{id}/stop, body {by?}. |
jobs_cancel | POST /v0/jobs/{job_id}/cancel | push | Cancel a generation, a task mint, a derive or an ontology compile: POST /v0/jobs/{id}/cancel, body {by?}. |
jobs_live | GET /v0/jobs/{job_id}/live | read | GET /v0/jobs/{id}/live — a run’s or a group’s trials as they start and end, the in-flight trial’s tool calls so far, n of N and the estimate to finish. |
jobs_audit | GET /v0/jobs/{job_id}/audit | read | GET /v0/jobs/{id}/audit — the run’s audit log, oldest first: started, each trial’s start and end, stops and who asked, restarts and resumes, the end. |
jobs_calls | GET /v0/jobs/{job_id}/calls | read | GET /v0/jobs/{id}/calls — a run’s or a group’s tries so far, call by call, while it runs: per try, its task (task 07), its try number, each call’s tool and the class of its answer (data, empty, error:<class>, refused, pending), its egress forwarded and refused counts when its tools were a served version, its final answer’s class (answered, said_no_data, refused, none) and the names of the checks it failed; and tally, per pinned version, met, failed, infra and invalid so far. |
Findings and reports
30 tools. List only these with?chapters=findings on the server URL. Routes: Findings and reports.
| Tool | Route | Scope | What it does |
|---|---|---|---|
investigate_report · shaped | POST /v0/orgs/{orgtag}/failure-modes/investigate | push | {text, author, ticket, origin_source, session_ids?, trace_ids?, window?, tools?, title?, seen_at?, member?, stats?} — one report, investigated: pull what it names, record it as a failure mode with the trace cited, and name the sandbox its tasks go on. |
classify_failure_mode · shaped | PATCH /v0/orgs/{orgtag}/failure-modes/{mid} | push | PATCH …/failure-modes/{id} {behaviour?, bucket?, target?: {tool?, column?}, status?, by}: a person classifies a mode already recorded — failuremodes.to_classify lists the ones nobody has, and this is what writes. |
classify_reported_sessions · shaped | POST /v0/orgs/{orgtag}/failure-modes/{mid}/classify | push | POST …/failure-modes/{id}/classify {reader?: rule | model, holding?: [tool], model?, dry_run?, by}: the sessions the org’s feedback sheet reported, read against this false_no_data mode (modesessions). |
held_set_pickout · shaped | POST /v0/orgs/{orgtag}/held-sets/{gid}/pickout | push | POST /v0/orgs/{org}/held-sets/{gid}/pickout {dry_run?, by, seed?}: the model pick-out read of a held set. |
report_proposals · shaped | GET /v0/orgs/{orgtag}/failure-modes/proposals | read | ?origin=slack|console&state=proposed|confirmed|dismissed — this org’s proposals, oldest first, each with what its confirm did (the mode, the sandbox, the job, the task indexes once landed, the run link). |
report_to_set · shaped | POST /v0/orgs/{orgtag}/failure-modes/proposals/{pid}/confirm | push | {by, title?, behaviour?, bucket?, environment?, count?, target_tool?} — a member confirms the proposal: the failure mode is written through the failure-mode store (human tier, made_mode naming who), and its tasks (count, 6 by default, 5 to 8) are written through the existing generation route on the bucket’s sandbox — a write’s as a job, a no-data mode’s at once (201, stored_as, no job), read straight off that answer. |
held_set_quality · shaped | GET /v0/orgs/{orgtag}/held-sets/{gid}/quality | read | GET /v0/orgs/{org}/held-sets/{gid}/quality: one held set’s quality packet — the line, the ranked losses and one row per task with each check’s verdict and its reason, every term with its tier and n, an unmeasured term absent (heldquality). |
weekly_reports · shaped | GET /v0/orgs/{orgtag}/reports | read | GET /v0/orgs/{org}/reports: this org’s reports, newest first — title, kind, source, who made it and when, how often its link was opened, and whether the link answers now. |
write_weekly_report · shaped | POST /v0/orgs/{orgtag}/reports/weekly | push | POST /v0/orgs/{org}/reports/weekly with {by?}: write this org’s page for the previous complete ISO week now (Monday 00:00Z to Sunday 23:59:59Z), assembled by rule from stored records with no model call. |
org_rubric_get | GET /v0/orgs/{orgtag}/rubric | read | GET /v0/orgs/{org}/rubric?persona=<id>&use_case=<id>: one persona’s criteria for one job, trace checks and output checks apart (valueprofile.rubric). |
calibration_list | GET /v0/orgs/{orgtag}/calibration | read | GET /v0/orgs/{org}/calibration: every calibration set, summarised. |
calibration_post | POST /v0/orgs/{orgtag}/calibration | push | POST /v0/orgs/{org}/calibration: a set of judge verdicts beside a person’s labels (calibration.normalize names the body). |
calibration_get | GET /v0/orgs/{orgtag}/calibration/{cid} | read | GET /v0/orgs/{org}/calibration/{cid}: one set’s summary and its labelled items. |
calibration_patch | PATCH /v0/orgs/{orgtag}/calibration/{cid} | push | PATCH /v0/orgs/{org}/calibration/{cid} with {retired: {reason}, by} or {retired: null}: a set leaves the panel, and stays on the record, restorable. |
reports_post | POST /v0/orgs/{orgtag}/reports | push | POST /v0/orgs/{org}/reports with {kind: "run_group" | "packet" | "provider_comparison", id, title?, made_by?}: freeze what that run group’s page (or that sandbox’s realism, or that provider comparison as a member reads it) shows now into one report. |
weekly_delivery_get | GET /v0/orgs/{orgtag}/reports/weekly/delivery | read | GET /v0/orgs/{org}/reports/weekly/delivery: whether each week’s page is shared and its link posted to Slack — {on, channel, by, at} — and whether this org has a Slack install to post through. |
weekly_delivery_put | PUT /v0/orgs/{orgtag}/reports/weekly/delivery | push | PUT /v0/orgs/{org}/reports/weekly/delivery with {on, channel?, by?}: turn weekly delivery on (a Slack channel id is required when the org has a Slack install) or off. |
report_get | GET /v0/orgs/{orgtag}/reports/{report_id} | read | GET /v0/orgs/{org}/reports/{report_id}: one report as its reader will see it, with who made it, its source and its opens. |
report_share | POST /v0/orgs/{orgtag}/reports/{report_id}/share | push | POST /v0/orgs/{org}/reports/{report_id}/share with {on: bool, named?: bool}. |
failure_modes_list | GET /v0/orgs/{orgtag}/failure-modes | read | This org’s known failure modes, oldest first — the feedback notes that carry a mode block — with the vocabularies a form offers. |
failure_modes_post | POST /v0/orgs/{orgtag}/failure-modes | push | {modes: [{title, description?, example?, reported_by?, seen_at?, status?, origin?, behaviour?, target?, tasks?, ticket?, trace?}], author | by} or one such object — the console’s form, an operator’s import of a ticket export and the customer’s feedback hook write the same notes. |
failure_mode_proposals_post | POST /v0/orgs/{orgtag}/failure-modes/proposals | push | {report: rpt_…} (a report the Slack handler parked) or {text, author, ticket?, origin_source?, tools?, seen_at?, member?} — the report read into ONE proposed failure mode by rule (reportproposal), stored as a proposal in the org’s store. |
failure_mode_proposal_dismiss | POST /v0/orgs/{orgtag}/failure-modes/proposals/{pid}/dismiss | push | {by} — a member dismisses the proposal: nothing is written but the proposal’s state and who. |
failure_modes_rekey | POST /v0/orgs/{orgtag}/failure-modes/{mid}/rekey | push | POST …/failure-modes/{id}/rekey {environment, by, dry_run?}: a held false_no_data set’s answer key, re-written so it rejects a sprayed or padded list (failuremodes.rekey_false_no_data). |
org_scorers_get | GET /v0/orgs/{orgtag}/scorers | read | THE ORG’S SCORERS: every check kind in use on the org’s standing tasks — its name and tier, what runs verbatim (the detector’s read as executed, or the judge’s prompt as sent, with each value the task supplies named as a slot), its version digest, where it is used (checks, tasks, sandboxes) and its last verdicts (met / missed / no-verdict kinds over the stored trials of each task’s newest run). |
root_post | POST /v0/root | read | One sentence in; the path that was chosen and what each step returned. |
root_get | GET /v0/root | read | The paths this surface holds, and the record of what has been asked. |
thread_get | GET /v0/root/threads/{tid} | read | One conversation, every turn, with the answers that were kept. |
thread_patch | PATCH /v0/root/threads/{tid} | read | Rename a conversation. |
thread_delete | DELETE /v0/root/threads/{tid} | read | Delete a conversation and every turn in it. |
Exports
16 tools. List only these with?chapters=exports on the server URL. Routes: Exports.
| Tool | Route | Scope | What it does |
|---|---|---|---|
export_dataset · shaped | GET /v0/orgs/{orgtag}/datasets | read | The datasets a run can be started over — one per environment, by the name LangFuse holds it under. |
comment_on_pr · shaped | POST /v0/orgs/{orgtag}/runs/group/{gid}/pr-comment | push | Post a landed run’s result on its branch’s open pull request: {against?, console?, dry_run?} → {number, pr_url, comment_url, action, branch, verdict}. |
datasets_publish | POST /v0/orgs/{orgtag}/datasets/publish | push | Send every environment of this org to LangFuse as a dataset. |
deliveries_slack | POST /v0/orgs/{orgtag}/deliveries/slack | push | The bug finding as one Slack message: {group, channel?, against?, main?, thread_ts?, dry_run?, confirm?, weekly?} → the blocks, and on a real post {channel, ts, permalink}. |
deliveries_datadog | POST /v0/orgs/{orgtag}/deliveries/datadog | push | POST /v0/orgs/{orgtag}/deliveries/datadog {group, against?, main?, bug?, kinds?, dry_run?, confirm?, console?, format?} — one bug finding as Datadog payloads: an event, a notebook and the Experiments record, each citing the trace ids the compare carries and linking the finding’s page (?via=datadog). |
runs_reexport | POST /v0/orgs/{orgtag}/runs/{rid}/reexport | push | Re-export one stored run from the directory already on disk. |
org_export | GET /v0/orgs/{orgtag}/export | read | Every world of one org, one tier, one target. |
export_datadog | POST /v0/exports/datadog | read | POST /v0/exports/datadog {org, set, environment?} -> a zip of one set (ci-10 / smoke-20 / full) in Datadog’s dataset shape. |
export_datadog_experiment | POST /v0/exports/datadog-experiment | read | POST /v0/exports/datadog-experiment {org, group} -> a zip of one landed run group in Datadog Experiments’ shape. |
export_datadog_judges | POST /v0/exports/datadog-judges | read | POST /v0/exports/datadog-judges {org, environment?} -> a zip of judge definitions mirroring the dataset’s verifiers, in Datadog’s evaluation shape. |
org_archive_get | GET /v0/orgs/{orgtag}/archive | admin | The whole org as one tar.gz, streamed. |
org_mirror_post | POST /v0/orgs/{orgtag}/mirror | admin | POST /v0/orgs/{org}/mirror {to?, kinds?, since?, replace?, dry_run?} — start a round sending this org’s records to the workspace this one mirrors into. |
org_mirror_list | GET /v0/orgs/{orgtag}/mirror | admin | GET /v0/orgs/{org}/mirror[?since=&kinds=a,b] — what a round from this workspace would send, in order, with each member’s digest and size, and what it holds back with why. |
org_mirror_member | GET /v0/orgs/{orgtag}/mirror/{kind} | admin | GET /v0/orgs/{org}/mirror/{kind}?id= — one listed member’s bytes, as stored: what litmus-ai mirror streams from the source into the receiver’s door. |
org_mirror_in_held | GET /v0/orgs/{orgtag}/mirror-in | admin | GET /v0/orgs/{org}/mirror-in[?wait=1] — what this workspace holds of the org, as {"<kind>/<id>": sha256}, for a sender to diff against, and what it refused at which digest, so a refusal is named again and not re-sent. |
org_mirror_in | POST /v0/orgs/{orgtag}/mirror-in/{kind} | admin | POST /v0/orgs/{org}/mirror-in/{kind}?id=&sha256=[&bundle=][&replace=1] — admit ONE member of a mirror round. |
Organization
32 tools. List only these with?chapters=org on the server URL. Routes: Organization.
| Tool | Route | Scope | What it does |
|---|---|---|---|
name_version · shaped | PUT /v0/orgs/{orgtag}/version-labels | push | PUT /v0/orgs/{org}/version-labels with {version, label} — the version string a run group carries (its agent version or a surface version) and the team’s name for it, up to 40 characters; a null label clears it. |
delivery_get | GET /v0/orgs/{orgtag}/delivery | read | GET /v0/orgs/{org}/delivery: one row per channel in D3’s order — Slack, PR comment, Datadog, email, download — each {channel, on, state, reason, target, recommended, last_send}. |
delivery_log_get | GET /v0/orgs/{orgtag}/delivery/log | read | GET /v0/orgs/{org}/delivery/log?limit=: every send, newest first — who, when, the channel, the finding’s group and the external ref (permalink, Slack ts, Datadog event id). |
delivery_confirm_post | POST /v0/orgs/{orgtag}/delivery/confirm | push | POST /v0/orgs/{org}/delivery/confirm with {member, channel, group, target?}: a single-use, ten-minute confirm that this signed-in member sends this group’s finding over this channel (to target only, when named: the Slack channel an admin pressed Send for). |
delivery_put | PUT /v0/orgs/{orgtag}/delivery/{channel} | push | PUT /v0/orgs/{org}/delivery/{channel} with {on, target?, by?}: turn one channel on or off. |
delivery_preview_get | GET /v0/orgs/{orgtag}/delivery/{channel}/preview | read | GET /v0/orgs/{org}/delivery/{channel}/preview: the current finding as that channel would carry it — a dry run: nothing is sent, nothing is logged. |
delivery_opens_post | POST /v0/orgs/{orgtag}/delivery/opens | read | POST /v0/orgs/{org}/delivery/opens with {group, via, member?}: count one open. |
delivery_opens_get | GET /v0/orgs/{orgtag}/delivery/opens | read | GET /v0/orgs/{org}/delivery/opens?group=: the org’s opens per channel, for one finding or all — line is “opened from Slack 3 · PR 1 · …”. |
org_spend_get | GET /v0/orgs/{orgtag}/spend | read | GET /v0/orgs/{org}/spend — today’s budget and what is spent against it, where the budget comes from (the org’s own setting, the env var by name, or the default), the ledger by day for 30 days and by job (kind, started, dollars when priced, who or what asked: a person, a schedule, the pass), and what the budget or a person holds back, each with its estimate and its sentence. |
org_budget_put | PUT /v0/orgs/{orgtag}/budget | push | PUT /v0/orgs/{org}/budget with {usd_per_day} — the daily budget the after-ingest pass applies within, for this org; null clears it and the env var (by name), else the default, answers again. |
org_judge_get | GET /v0/orgs/{orgtag}/judge | read | GET /v0/orgs/{org}/judge — the model the org’s LLM-judged checks are decided by, and where it comes from (org, or default: the graders’ own pin); each judged scorer’s model and source (scorer, org, default); the stored overrides with who set them; and the exact model strings an admin may choose — the agents’ allow-list, narrowed to the provider the judge calls. |
org_judge_put | PUT /v0/orgs/{orgtag}/judge | admin | PUT /v0/orgs/{org}/judge with {model} — the model this org’s judged checks ask from the next run on; null returns them to the graders’ pin. |
org_confidence_get | GET /v0/orgs/{orgtag}/confidence | read | GET /v0/orgs/{org}/confidence — the thresholds the compare’s confidence is banded by (low, high), whether they are the org’s or the defaults (20 and 90), who set them, the bands in words and the formula. |
org_confidence_put | PUT /v0/orgs/{orgtag}/confidence | admin | PUT /v0/orgs/{org}/confidence with {low, high} — whole numbers from 0 to 100, low below high; both null returns the org to 20 and 90. |
version_labels_get | GET /v0/orgs/{orgtag}/version-labels | read | GET /v0/orgs/{org}/version-labels — {version: label} for every version this org has named. |
slack_install_get | GET /v0/orgs/{orgtag}/slack | read | The org’s Slack install: names and ids, never a token. |
slack_install_post | POST /v0/orgs/{orgtag}/slack | admin | {team_id, team_name?, by, bot_token?} — connect the org’s Slack workspace. |
slack_install_delete | DELETE /v0/orgs/{orgtag}/slack | admin | Disconnect: the install goes, and a token this workspace stored for it is deleted from the secret store. |
slack_oauth_post | POST /v0/orgs/{orgtag}/slack/oauth | admin | {code, redirect_uri, by} — finish Add to Slack. |
members_list | GET /v0/orgs/{orgtag}/members | admin | No description. |
members_post | POST /v0/orgs/{orgtag}/members | admin | No description. |
members_patch | PATCH /v0/members/{mid} | admin | No description. |
members_delete | DELETE /v0/members/{mid} | admin | No description. |
whoami | GET /v0/whoami | read | What the presented credential is. |
orgs_list | GET /v0/orgs | admin | GET /v0/orgs — the orgs this workspace holds, [{org, surfaces, created_at}]. |
org_purge_plan | GET /v0/orgs/{orgtag}/purge | admin | What deleting this org would remove, without removing it. |
org_status_get | GET /v0/orgs/{orgtag}/status | read | GET /v0/orgs/{org}/status[?format=md][&names=1]. |
org_delete | DELETE /v0/orgs/{orgtag} | admin | Delete every trace of an org from this workspace. |
org_rename | POST /v0/orgs/{orgtag}/rename | admin | POST /v0/orgs/{org}/rename {to} — every record the org owns moves to the tag to, as one journaled move. |
tokens_list | GET /v0/orgs/{orgtag}/tokens | admin | The org’s tokens: id, name, scope, prefix, created_by, created_at, expires_at, last_used_at, revoked_at. |
tokens_create | POST /v0/orgs/{orgtag}/tokens | admin | Mint a token for the org: name (what will hold it), scope in read|push|admin, and optionally expires_in_days or expires_at (unix). |
tokens_revoke | DELETE /v0/orgs/{orgtag}/tokens/{tid} | admin | Revoke one token. |