AG-UI Interface Contract

Generated from the canonical contract at docs/api_standards/cdc_event_contract.md (CONTRACT-CDC-001). Edit the source, then regenerate: python docs/portfolio/_build/generate_agui_contract.py.

Version: 2.8 Status: Active — single source of truth for the AI backend ↔︎ frontend interface Owner: CTO Consumers: KAP web frontend (web/ — AG-UI client, /api/ag-ui-chat proxy, useAguiEvents) Producers: KAP AI backend (ai/ — gateway + ADK runtime, dataadk default), KavApps kavai_server (extern/KavApps, DataADK), and agent-harness adapters (ai/servers/kawa/)

This document is the canonical interface contract. The Web Handbook’s “Web/AI Interface” chapter is the narrative companion and renders a copy of this contract on the documentation portal; the linter fails if the copy drifts. What this contract promises is the intersection verified against both backends (TEST-E2E-01); backend-specific behavior is marked explicitly.

1. Overview

This document defines the AG-UI event contract between the AI backends and the frontend chat surfaces. All events are delivered over Server-Sent Events (SSE) at POST /chat/agui/stream.

The frontend MUST handle all events listed below and MUST tolerate the absence of any event marked reserved or backend-specific. The backends guarantee the event ordering in §8 and the payload shapes defined here.

AI servers: one category, many roles

An AI server is anything that serves the five contract endpoints (/health, /systems, /systems/verified, /chat/agui/capabilities, POST /chat/agui/stream) with the semantics in this document. kavai_server, the KAP ai/ gateway, and the agent-harness adapters are all the same category of thing; which port each occupies is a deployment role, not a difference in kind:

  • The front-door slot is “whatever AI_SERVER_URL points at” — any conformant server can fill it. In a single-engine deployment that can be an adapter directly; in a multi-engine deployment it is a server that additionally routes (registry, e2e_verified gating).
  • Engine slots are conformant servers sitting behind another one.

Because an AI server presents exactly the surface it consumes, servers compose: a front door is simultaneously a server (to the frontend) and a client (of the engines behind it), and nothing upstream can tell the difference. Port assignments in other documents (:8080, :8082, …) are conventions for these roles, not properties of the servers themselves.

2. Transport

Property Value
Endpoint POST /chat/agui/stream
Content-Type (response) text/event-stream
Auth Authorization: Bearer <jwt_token> (or body jwt_token)
Format data: <JSON>\n\n per event

Request Body (ChatRequest)

{
  message: string;            // The user's message (required)
  threadId?: string;          // Conversation thread ID (UUID)
  runId?: string;             // Optional run ID (auto-generated if omitted)
  system_type?: "dataadk";    // KAP ai/ routes on it; kavai_server ignores it
                              // (single DataADK pipeline)
  state?: Record<string, any>;
  tools?: any[];
  context?: any[];
  workspace_context?: {       // Where the user is. v2.3 — see §2.1.
    workspace_slug?: string;       // "integrity" | "data-explorer"
    organization_id?: string;      // id or "all"
    campaign_id?: string;          // id or "all"
  };
  dataset_scope?: {           // Workspace scope. KAP ai/ honors it;
    organization_ids?: string[];   // kavai_server currently IGNORES it —
    dataset_slugs?: string[];      // do not rely on server-side scoping
    campaign_ids?: string[];       // against that backend.
    focus_image_ids?: string[];
    // Defaults that ground an ambiguous prompt without narrowing what may
    // be answered (v2.3):
    default_organization_id?: string;
    default_dataset_slug?: string;
    default_campaign_id?: string;
    // Enforcement switch (v2.3) — see §2.2. Absent or false means the scope
    // is advisory context only.
    strict?: boolean;
    enforce?: boolean;             // legacy alias for `strict`
  };
}

Browser → proxy layer: the KAP web app does not call the backend directly. The browser posts to /api/ag-ui-chat with { message, auth: { jwt_token }, system_type, workspace_context, dataset_scope, stream }; the proxy forwards to AI_SERVER_URL + /chat/agui/stream as above.

2.1 workspace_context — where the user is (v2.3)

The proxy must forward workspace_context to the backend. Before v2.3 the field was named as something the browser sends, but ChatRequest had no place to put it, so the proxy dropped it and every request reached the engine as organization_id: all. An engine that cannot see the user’s workspace must discover it, which is what produced dataset-catalogue calls — and catalogue surfaces — in answers about a single asset.

Backends should use it to ground ambiguous prompts. It is context, not authorization: RLS remains the only thing that decides what a caller may read.

Identifiers are not context. organization_id and campaign_id are ids, and no data tool accepts an id as a selector. A backend that surfaces workspace context to a model should resolve ids to the names and slugs its tools accept, so the model does not have to spend a discovery call translating.

2.2 dataset_scope is advisory unless strict (v2.3)

dataset_scope is a default, not a filter. A selected campaign grounds “show me the images”; it must not stop “list my datasets” being answered against everything the caller’s JWT can see.

Hard filtering happens only when the request carries strict: true (or the legacy enforce: true) and a non-empty dataset_slugs. In that mode the gateway validates outbound entity events against the scoped slug set: out-of-scope DATASET_LIST rows and IMAGE_GALLERY images are dropped, counts are kept consistent, and an event whose rows are all out of scope is dropped entirely rather than emitted hollow.

An absent scope, a malformed scope, a non-strict scope, or an empty slug list all mean context only. Producers must not treat the presence of dataset_scope as an authorization boundary.

3. Authentication Behavior (tested)

  • Valid JWT: request proceeds; RLS scopes all data access to the caller.
  • Missing or expired JWT reaching a backend: the stream opens normally and fails with a mid-stream RUN_ERROR — NOT an HTTP 401. Frontends must not rely on transport-level auth failures from the backends.
  • The KAP web proxy compensates: /api/ag-ui-chat pre-checks JWT expiry and fast-fails with HTTP 401 before contacting the backend; the browser’s token-refresh flow keys off that proxy 401.

4. Run Lifecycle Events

Every SSE stream follows this lifecycle:

RUN_STARTED → [events...] → RUN_FINISHED | RUN_ERROR

RUN_STARTED

{
  type: "RUN_STARTED",
  id: string,                 // Event UUID
  timestamp: string,          // ISO 8601
  runId: string,
  threadId: string
}

RUN_FINISHED

{
  type: "RUN_FINISHED",
  id: string,
  timestamp: string,
  runId: string,
  threadId: string,
  result?: { usage?: TokenUsage }   // kavai_server: result = { usage } only
}

RUN_ERROR

{
  type: "RUN_ERROR",
  id: string,
  timestamp: string,
  runId: string,
  threadId: string,
  error: { message: string, type: string, details?: any }
}

5. Text Message Events

Text streams as deltas within a message lifecycle:

TEXT_MESSAGE_START → TEXT_MESSAGE_CONTENT* → TEXT_MESSAGE_END

TEXT_MESSAGE_START

{ type: "TEXT_MESSAGE_START", id: string, timestamp: string,
  messageId: string, role: "assistant" }

TEXT_MESSAGE_CONTENT

{ type: "TEXT_MESSAGE_CONTENT", id: string, timestamp: string,
  messageId: string,
  delta: string,              // Non-empty text chunk
  data?: { response_summary?: any } }

TEXT_MESSAGE_END

{ type: "TEXT_MESSAGE_END", id: string, timestamp: string, messageId: string }

Note: for side-panel response types (markdown_report, analysis) the chat bubble may carry only a short status line (“Analysis complete. See the detailed report in the side panel.”) with the substance in a custom event.

6. Tool Call Events

TOOL_CALL_START → TOOL_CALL_ARGS → TOOL_CALL_RESULT → TOOL_CALL_END

TOOL_CALL_START

{ type: "TOOL_CALL_START", id: string, timestamp: string,
  toolCallId: string,
  toolCallName: string,       // e.g. "list_datasets", "execute_sql"
  parentMessageId?: string }

Active tool names (DataADK, both backends):

Tool Description
list_datasets Discover datasets accessible to the caller (RLS)
execute_sql Read-only SQL behind the AST firewall (SELECT/WITH only, LIMIT enforced)
smart_image_search Image retrieval by text query within a dataset

TOOL_CALL_ARGS / TOOL_CALL_RESULT / TOOL_CALL_END

{ type: "TOOL_CALL_ARGS", id: string, timestamp: string,
  toolCallId: string, delta: string,               // JSON string of args
  truncated?: true, contentLength?: number }       // present only when cut

{ type: "TOOL_CALL_RESULT", id: string, timestamp: string,
  messageId: string, toolCallId: string,
  content: string, role: "tool",                   // JSON string of result
  truncated?: true, contentLength?: number }       // present only when cut

{ type: "TOOL_CALL_END", id: string, timestamp: string, toolCallId: string }

Both payloads are display mirrors, not the model’s input. The agent receives the untruncated arguments and result directly from the harness; these events exist so the UI (and anyone reading an SSE transcript) can see what a tool was called with and what it returned. Server-side derivation of IMAGE_GALLERY / DATASET_LIST rows also runs on the full result, before any capping.

Emitters cap both to bound SSE volume — Kawa uses 32 000 chars for content and 2 000 for delta (_TOOL_RESULT_CAP and _TOOL_ARGS_CAP in ai/servers/kawa/runner.py). When a cap bites, the emitter must say so: append a ...[truncated N of M chars ...] marker to the text and set truncated: true with contentLength = the full pre-truncation size. The rule is the same for both events, and applies to any future mirrored payload — delta was capped without declaring it for long enough to matter, and the argument that reaches the cap in practice is a kap_execute_sql statement, cut mid-clause and read as the query that ran.

Uncapped text is byte-identical to what the tool emitted, so a complete result stays parseable as JSON. Consumers may ignore both fields; they must not assume the text is complete when truncated is present.

Because a cut drops the tail, KAP tool results put scalars (count, note, efficiency_note) before the rows array, so the surviving prefix always carries the row total.

7. Custom Events

Custom events use type: "CUSTOM" with a customType discriminator. The frontend switches on customType.

Emission matrix (tested 2026-07-07):

customType KAP ai/ kavai_server Frontend handling
IMAGE_GALLERY emits suppressed for markdown_report responses (see §7.2) required
DATASET_LIST emits emits required
MARKDOWN_REPORT emits emits (primary answer form — V3 reporter) required
IMAGE_ANOMALIES emits — required
THREE_D_OVERVIEW emits — required
THERMAL_ANALYSIS emits — optional
CDC_FULL_PAYLOAD reserved — tolerate absence
CDC_PROVENANCE reserved — tolerate absence (see §7.7)
IMAGE_ANALYSIS_RESULT emitted (v2.5, see §7.8) consumed (gallery overlay) tolerate absence (see §7.8)

7.3 DATASET_LIST

{
  type: "CUSTOM", customType: "DATASET_LIST", id: string, timestamp: string,
  data: {
    datasets: DatasetInfo[],
    metadata: { type: "dataset_list", total_datasets: number, message: string }
  }
}

DatasetInfo: { id, name, slug, description?, total_images, created_at? }

7.4 MARKDOWN_REPORT

{
  type: "CUSTOM", customType: "MARKDOWN_REPORT", id: string, timestamp: string,
  data: {
    markdown: string,         // may contain [[component:identifier]] markers (§7.2 B)
                              // and [<image_uuid>] reference lines (§7.4.1)
    title?: string,
    metadata?: Record<string, any>   // includes dataset_slug when resolved,
                              // dataset_count for a consolidated listing
                              // (§7.4.2), and, when
                              // KAWA_PDF_UPLOAD_ENABLED=1, the optional
                              // PDF fields below
  }
}

Optional PDF metadata fields (Kawa only, present only when the upload succeeded):

Field Type Meaning
pdf_report_id number pdf_reports.id; pass to GET /api/reports/{id}/download to mint a signed URL
pdf_storage_path string Bucket-relative path; stable permanent reference

Frontends MUST treat both fields as optional — they are absent when the upload was skipped, timed out, or the feature flag is off.

7.4.1 Inline image references (v2.2)

Motivation. KavApps#1037 introduced image embeds in MARKDOWN_REPORT markdown as ![filename|image_uuid](thumbnail_url) — an alt-text micro-format carrying a filename and a storage URL. That re-splits the responsibility settled in v2.1 (§7.1): backends emit image identities; presentation — thumbnails, signed URLs — is resolved by the frontend at render time. Embedding storage URLs in report markdown persists them in session state and message history, coupling stored transcripts to today’s storage layout, and the filename|uuid split forces every markdown consumer to know a bespoke convention. This amendment extends the §7.1 rule to MARKDOWN_REPORT: the backend passes only the image UUID.

Reference grammar. An image reference is a line containing exactly [<image_uuid>], where <image_uuid> is a lowercase RFC 4122 UUID — no filename, no URL. Multi-image example (consecutive lines form a run):

## 🖼️ Thermal Imagery
[aaaaaaaa-0000-0000-0000-000000000001]
[bbbbbbbb-0000-0000-0000-000000000002]

Backend obligations:

  • MUST emit a reference only for image UUIDs verified against retrieved rows (no guessed or synthesized IDs).
  • MUST NOT embed filenames, thumbnail_urls, or signed URLs for referenced images — the same deprecation §7.1 applies to IMAGE_GALLERY.
  • MUST set data.metadata.dataset_slug whenever the markdown contains references (frontend URL resolution requires it).

Frontend obligations:

  • MUST pre-parse reference tokens from the raw markdown before rendering (the renderer already does an equivalent pre-pass for image runs).
  • MUST batch-resolve metadata — one fetch per report, not one per image (KAP web: POST /api/images with image_ids, ≤50 per batch, RLS-scoped).
  • MUST render each resolved reference as a thumbnail with the filename as caption and data-image-id="<image_uuid>" on the rendered element.
  • Runs of ≥2 consecutive reference lines render as a gallery grid — the same run rule the renderer applies to form C (splitImageRuns); single references render inline.
  • Click-to-viewer is required (unlike form C): full-resolution viewing resolves through the batched SAS endpoint POST /api/datasets/<slug>/image-url (resolveGalleryImageUrls — cached, request-deduped); click target as in §7.1.
  • Unresolved references (deleted image, RLS-filtered, malformed ID, transient resolution failure): render the existing broken-image placeholder with the UUID as caption. MUST NOT drop the embed silently and MUST NOT fail the rest of the report.

Accepted trade-off (part of the contract). [<image_uuid>] is not a renderable image in standard markdown — CommonMark treats it as a shortcut reference and, with no definition, renders it as literal text. Resolution is therefore a rendering concern everywhere: any consumer of the stored markdown (chat exports, PDF generation, external viewers) must apply the same resolution pass, and unresolved output degrades to visible [uuid] text rather than an image. This is deliberate: agent output stays minimal, no storage URLs enter persisted transcripts, and presentation logic lives in one place.

Supersedes. This replaces the ![filename|uuid](thumbnail_url) alt-text injection described in KavApps#1037 (backend UUID injection + frontend parseAltText/data-image-id splitting). Frontends MAY keep parsing the superseded alt format while stored reports from its era exist. The emission matrix is unchanged — this is a payload rule inside MARKDOWN_REPORT, not a new event.

Adoption status (2026-08-01). The adoption gaps identified at proposal time are closed: POST /api/images returns per-image dataset_slug and is documented with the batch imageIds form of POST /api/datasets/{slug}/image-url in web/public/api-docs/swagger.yaml and the API inventory; verification exists in TEST-E2E-01 (ask-gallery-smoke contract D: multi-image run → grid, resolution, click-to-viewer), the engine-certification image-browsing row (canonical multi/single/mixed fixtures, unresolved-placeholder rule), and TEST-AIB-01 (signed-URL leakage guard on every collected MARKDOWN_REPORT). KavApps propagation is tracked in docs/plans/20260801_markdown_report_image_references_execution.md (WP-E).

7.4.2 Consolidated dataset listing (v2.7)

Motivation. An engine may answer “list my datasets” without emitting a separate DATASET_LIST event, folding the listing into the turn’s MARKDOWN_REPORT instead. KavApps’ DataADK does this by design — its pipeline emits exactly two custom events, MARKDOWN_REPORT and IMAGE_WITH_ANNOTATIONS, and smart_router never produces any other response_type. The listing is then carried by the §7.2 form-B marker [[dataset-list:all]], which the frontend expands into the dataset-list component.

Declared field. A consolidated listing SHOULD set:

Field Type Meaning
metadata.dataset_count number How many datasets the listing covers. > 0 marks the report as a dataset listing; 0/absent means it is not one.

Consumers MUST treat it as optional and MUST NOT infer identity from it: it is a count, not a set of identifiers.

What consolidation costs, stated plainly. dataset_count tells a consumer how many, never which. In this form no dataset id or slug crosses the wire at all — the frontend re-fetches the list itself — so a scope-restricted run cannot be checked for confinement from the stream. That is not a weaker check but an absent one: scope enforcement becomes unobservable, not merely unverified, and the certification scope row skips with that reason recorded rather than passing (ai/tests/features/ test_scope.py).

Closing that gap needs identifiers, not a larger count. An engine that wants its scope enforcement to be verifiable SHOULD additionally carry the listed datasets’ id/slug in metadata — the shape is deliberately left open here pending agreement with the AI team, since it is their emitter that would carry it (docs/proposals/20260902_analysis_turn_image_reference_two_contracts.md records the sibling question for analysis turns).

7.5 IMAGE_ANOMALIES — KAP ai/ only

{
  type: "CUSTOM", customType: "IMAGE_ANOMALIES", id: string, timestamp: string,
  data: { anomalies: Anomaly[], total_count: number, dataset_slug?: string }
}

Anomaly: { id, image_id, image_filename, category, parent_category, subcategory, bbox: [x, y, w, h] /* normalized 0-1 */, area }

7.6 THREE_D_OVERVIEW — KAP ai/ only

{
  type: "CUSTOM", customType: "THREE_D_OVERVIEW", id: string, timestamp: string,
  data: { images?: ImageData[], dataset_slug?: string, dataset_name?: string,
          gaussian_splats?: any, reference_lat?: number, reference_lon?: number,
          metadata?: Record<string, any> }
}

7.7 Reserved: CDC_FULL_PAYLOAD, CDC_PROVENANCE

Defined for the provenance/data-grid roadmap (legacy FR-CDC-014 lineage; no live FR yet). No production backend currently emits them. The frontend keeps its handlers (ProvenanceBar, metadata injection) and MUST render correctly when they never arrive. Shapes retained from v1.0:

CDC_FULL_PAYLOAD: data: { payload: Record<string, any>[], sql_query: string, row_count: number }
CDC_PROVENANCE:   data: { data_source, tables_queried, sql_query, capture_ids,
                          timestamp, row_count,
                          confidence: "high" | "medium" | "low" | "none",
                          citations: { source_table, entity_id, entity_label?, snippet? }[] }

7.8 IMAGE_ANALYSIS_RESULT (v2.5)

Motivation. Image-analysis answers (surface analysis today; thermal anomaly detection, OCR, quality assessment as they land) previously had no provider-neutral contract surface: a MARKDOWN_REPORT with [uuid] references proves an image was referenced, not analyzed, and the DataADK-specific IMAGE_WITH_ANNOTATIONS event was never part of this contract. This surface is the structured evidence of analysis — it carries the input UUIDs, a per-image outcome, and canonical findings — and is what the surface-analysis certification capability row asserts (docs/proposals/20260808_surface_analysis_certification_row.md). Canonical types: kavai_agent_contract.media (ImageAnalysisResult).

{
  type: "CUSTOM", customType: "IMAGE_ANALYSIS_RESULT", id: string, timestamp: string,
  data: {
    skill: { id: string, version: string },  // e.g. "image.surface-condition", "1.0.0"
    status: "ok" | "partial" | "failed",     // implied by the per-image statuses
    results: PerImageAnalysis[],             // exactly one entry per requested image UUID
    provenance?: AnalysisProvenance          // what produced this result (v2.5)
  }
}

PerImageAnalysis:

{
  image_id: string,          // lowercase RFC 4122 UUID — the requested image
  status: "ok" | "not_found" | "storage_error" | "fetch_failed" |
          "too_large" | "unsupported_media" | "decode_limit" | "executor_error",
  findings: Finding[],       // empty on non-ok status AND on a clean image
  reason?: string            // REQUIRED on non-ok status; absent on ok
}

Finding: { label: string, confidence: number /* 0..1 */, region?: \{ x, y, width, height } /* normalized 0..1, contained */ }

AnalysisProvenance (v2.5):

{
  attempt_id: string,            // identifies this execution
  backend: string,               // kind of executor: "llm", a detector service id
  model_id?: string,             // the specific model or checkpoint
  vocabulary_version?: string    // the label vocabulary the findings mean
}

Producer obligations:

  • MUST return exactly one PerImageAnalysis per requested image UUID — no dropped, duplicated, or synthesized entries. A failed acquisition is a stated per-image status, never a silent omission: a clean image is ok with zero findings; a failure is a non-ok status with a reason and no findings.
  • MUST carry image UUIDs only. The §7.4.1 leakage rule extends to this event in full: no filename, storage path, dataset slug, signed/SAS URL, credential, or raw bytes anywhere in the payload — including inside reason strings, skill options echoes, and provenance.
  • MUST emit geometry that is contained by the image, not merely bounded field-by-field: x + width ≤ 1 and y + height ≤ 1, within a tolerance for pixel→normalized division dust. Four fields in [0,1] is a weaker claim than a region inside the picture — x=0.9, width=0.5 passes every per-field check and draws a box whose right edge is at 1.4, which puts an overlay somewhere the finding is not. A producer whose model returns an oversized box trims the extent; it does not drop the finding.
  • SHOULD emit provenance. It is optional at v2.5 so adoption costs a producer nothing, and that is a migration state rather than the intent: a finding persisted as a durable annotation suggestion requires it (FR-AI-13), because “which model, which vocabulary, which attempt?” is what makes a bad backend’s output withdrawable rather than a manual audit.
  • Emitted ALONGSIDE the turn’s MARKDOWN_REPORT (which uses §7.4.1 [uuid] references for the same images); neither replaces the other.
  • Ordering per §8: after the pipeline completes, before RUN_FINISHED.

confidence is per-producer. The range is defined — [0,1], so consumers can sort and threshold — but the meaning is not comparable across backends: an LLM’s self-reported number, a detector’s calibrated objectness, and an open-vocabulary matcher’s similarity score are three different quantities wearing one name. Rank within one producer; do not read 0.8-here as 0.8-there. A display threshold is a property of a backend, not of this contract.

executor_error supersedes model_error (v2.5). The status is the one field a reviewer reads to answer “why did this image come back empty?”, and it was named for the only executor that existed when it was written. A detector service that is down, out of quota, or missing its weights is not a model error. Producers MUST emit executor_error; consumers MUST accept model_error from a producer built before v2.5 and treat it as the same outcome. The canonical types coerce it at validation, so code downstream of parsing sees one name.

Frontend obligations. Tolerate absence while adoption is in flight. The committed KAP web consumer is the gallery-viewer annotation overlay (plan docs/plans/20260808_image_skills_certification_execution.md, WP-7): findings render as overlays on the image resolved by UUID, using the same batched metadata/URL resolution as §7.4.1. Consumer state MUST be keyed by (image_id, skill.id) and not by image_id alone: two skills may analyze one image, and an image-keyed store silently replaces the first skill’s overlay with the second’s. Until WP-7 ships, KAP web MAY ignore the event.

Relationship to IMAGE_WITH_ANNOTATIONS. The IMAGE_WITH_ANNOTATIONS event (one event per analyzed image, analysis-target block) is an accepted transitional surface for surface analysis, alongside the canonical IMAGE_ANALYSIS_RESULT (v2.8, 2026-09-02). It was outside this contract until v2.8; it is now inside it, deliberately and temporarily.

Why it was brought in. Both surfaces answer the same question — did the engine identify its surface-analysis output in a form a consumer can act on — and both are emitted by engines that genuinely perform the analysis. DataADK emits the per-image form and is tested for it upstream (kavai_server’s test_multi_image_surface_analysis.py asserts N events for N images); argus, orion and kawa emit the canonical form. Leaving one of those outside the contract meant a conformant engine failed a certification row for choosing the surface its own frontend consumes, which measures migration progress rather than capability.

Producers MAY emit either surface; new producers SHOULD emit IMAGE_ANALYSIS_RESULT. Consumers MUST accept both for as long as this clause stands: a consumer that reads only one will lose analysis from half the engines.

Retirement is unchanged in intent and now has a stated condition: the transitional surface is removed from this contract, and from the certification row’s accepted set, once every frontend consuming surface analysis reads IMAGE_ANALYSIS_RESULT — at which point producers still emitting the old event fail the row rather than passing it. Recording the condition here is what keeps “transitional” from becoming permanent by default.

8. Event Ordering Guarantees

  1. RUN_STARTED is always the first event.
  2. RUN_FINISHED or RUN_ERROR is always the last event.
  3. Text message events are ordered: START → CONTENT* → END.
  4. Tool call events are nested within the run, before the final answer events.
  5. Custom events are emitted after the pipeline completes and before RUN_FINISHED.

9. Threads & Multi-Turn (tested)

Operation Method
Create thread Implicit: POST /chat/agui/stream with a new threadId
Resume thread Same endpoint with the existing threadId

Multi-turn context is server-side, keyed by threadId (session identity: app, JWT subject, threadId). Any client-sent conversation_history is ignored by kavai_server — do not rely on client-side history replay. Note the accuracy caveat: multi-turn answer quality is gated Q4 (see FR-APP-03 and the TEST-EVAL-01 registry notes).

10. Verification

This contract is enforced by tests, not review:

Surface Suite Run
Frontend ↔︎ kavai_server compatibility (incl. §7.2 A/B/C/D + §7.4.1 leakage guard) TEST-E2E-01 pixi run test-e2e-kavai-server
Event semantics per intent, handoff contracts TEST-CHT-02 (KavApps) pixi run pytest tests/integration
KAP ai/ live AG-UI semantics + leakage guards TEST-AIB-01 (test_agui_event_semantics.py) cd ai && pixi run pytest tests
Protocol compliance KavApps backend/api-tests/ upstream CI

Runtime self-description: GET /chat/agui/capabilities lists supported events per backend; GET /systems/verified lists E2E-verified engines.

11. Version History

Version Date Changes
2.6 2026-09-02 Accepted 2026-09-02 (owner decision, in session). IMAGE_WITH_ANNOTATIONS becomes an accepted transitional surface for surface analysis (§7.8) alongside the canonical IMAGE_ANALYSIS_RESULT, and KAP web commits to rendering both. It was explicitly outside this contract from v2.4; it is now inside it, deliberately and temporarily. The finding that prompted it, measured 2026-09-02 against KavApps main (a97ff172) served locally: asked to analyze one image, DataADK returns a substantive surface analysis and emits IMAGE_WITH_ANNOTATIONS alongside its MARKDOWN_REPORT — never IMAGE_ANALYSIS_RESULT. That matches the surface KavApps’ own committed test asserts (test_multi_image_surface_analysis.py: N events for N images). The capability is real — an rgb_surface_analyzer with a canonical vocabulary, multi-image cap and mandatory vocabulary gate — so requiring only the canonical event failed a conformant engine for emitting the surface its own frontend consumes. Which build is measured matters: the same question against the kap-stable lineage is declined as out of scope, because that build lacks the analyzer’s agent config; a cross-host result is meaningless unless it records the ref behind the URL. The row was therefore measuring migration progress, not capability, and failing a conformant engine for emitting the surface its own frontend consumes. Both surfaces answer the row’s question — did the engine identify its surface-analysis output in a form a consumer can act on. Consumers MUST accept both while this clause stands; producers MAY emit either, and new producers SHOULD emit the canonical form. Retirement gains a stated condition: the transitional surface leaves this contract, and the row’s accepted set, once every frontend consuming surface analysis reads IMAGE_ANALYSIS_RESULT — after which producers still emitting the old event fail the row rather than pass it. Recording the condition is what keeps “transitional” from becoming permanent by default. No canonical shape changes: §7.8’s IMAGE_ANALYSIS_RESULT definition, its statuses, UUID correlation and §7.4.1 leakage rule are untouched.
2.5 2026-08-09 Accepted 2026-08-09 (review: docs/reviews/20260809_image_analysis_proposals.md; decision records: docs/proposals/20260809_ai_annotation_suggestions.md, docs/proposals/20260809_pluggable_detection_backends.md). Four changes to §7.8, all consequences of a finding outliving the chat turn that produced it. Provenance (attempt_id, backend, model_id, vocabulary_version) is added as an optional payload field the canonical runner always populates: a result that says what was found but not what found it is fine for one turn and useless once a reviewer acts on it weeks later, when “a backend was mis-thresholded for a month, which findings came from it?” would otherwise be a manual audit. model_error becomes executor_error — the status was named for the only executor that existed, and a detector service that is down, out of quota, or missing weights is not a model error; producers emit the new name, consumers accept the old one, and the canonical types coerce so nothing downstream sees two names for one outcome. Geometry must be contained, not merely bounded field-by-field: x=0.9, width=0.5 passes four independent [0,1] checks and draws a box whose right edge is at 1.4, putting an overlay where the finding is not; producers trim an oversized extent rather than dropping the finding. confidence is declared per-producer — the range is contractual, the meaning is not comparable across backends. Consumer state is keyed by (image_id, skill.id): an image-keyed store silently replaces one skill’s overlay with another’s.
2.4 2026-08-08 Accepted 2026-08-08 (decision records: docs/proposals/20260808_uuid_only_image_analysis_skills.md and docs/proposals/20260808_surface_analysis_certification_row.md, both accepted 2026-08-08; plan: docs/plans/20260808_image_skills_certification_execution.md). Accepted with the consumer shipped (gallery-viewer annotation overlays) and the surface certified by the surface-analysis capability row + contract fixture; at acceptance no producer emitted it, engine adoption being FR-AI-09’s Q4 target deferred with the DataADK port — superseded 2026-08-09: argus, kawa and orion adopted the capability and declare the surface-analysis row, certified passing (docs/evaluation/ENGINE_CERTIFICATION.md); dataadk still does not declare, by the same owner decision. The canonical image-analysis surface: IMAGE_ANALYSIS_RESULT (§7.8), a provider-neutral custom event carrying the skill envelope (id/version), an overall ok\|partial\|failed status, and exactly one per-image entry per requested UUID with a typed per-image status — acquisition and model failures stated, never implied. The §7.4.1 leakage rule extends to the event in full (UUIDs only; no paths, slugs, signed URLs, or credentials anywhere in the payload). Canonical types and validators ship in kavai_agent_contract.media; the committed KAP web consumer is the gallery-viewer annotation overlay (plan WP-7, decision D2). DataADK’s IMAGE_WITH_ANNOTATIONS stays a KavApps-compatible emission during the transition and is retired once both frontends consume the canonical surface (decision D1, defaults accepted 2026-08-08). Emitted alongside MARKDOWN_REPORT; neither replaces the other.
2.3 2026-08-07 Proposed (decision record: docs/proposals/20260807_request_envelope_amendment.md). The request envelope reconciled with what is actually sent and honored. workspace_context added to ChatRequest (§2.1): it was named as a browser field with no place in the forwarded body, so the proxy dropped it and every request reached the engine as organization_id: all — engines then spent a discovery call working out where the user was, which surfaced as dataset catalogues inside single-asset answers. Backends should resolve ids to the names and slugs their tools accept, since an id is not usable context. dataset_scope gains the fields the client has been sending and the gateway has been reading (§2.2): default_organization_id, default_dataset_slug, default_campaign_id, and strict/enforce. strict is the switch that turns scope from a default into a filter, and was undocumented despite being the only field with enforcement consequences. States the advisory-by-default model already implemented in kavai.gateway.scope. No event shapes change; error.type remains free-form, so ENGINE_ERROR needs no amendment.
2.2 2026-08-01 Accepted 2026-08-01 (decision record: docs/proposals/20260801_markdown_report_image_references.md; discussion: KavApps#1037). Inline image references in MARKDOWN_REPORT (§7.2 form D, §7.4.1): backends embed images as bare [<image_uuid>] reference lines — no filename, no thumbnail or signed URL — extending the v2.1 “presentation is a frontend concern” rule from IMAGE_GALLERY to report markdown; metadata.dataset_slug required when references are present. The frontend pre-parses tokens, batch-resolves metadata (one fetch per report), renders thumbnails with filename captions and data-image-id, gallery-grids runs of ≥2, and resolves full-res via the batched SAS image-url endpoint; unresolved references render a placeholder captioned with the UUID, never dropped silently. Supersedes the ![filename\|uuid](thumbnail_url) alt-text format from KavApps#1037; form C (inline thumbnails) remains accepted transitional behavior. Stated trade-off: raw [uuid] does not render in plain CommonMark — every consumer of stored markdown must apply the resolution pass.
2.1 2026-07-18 Image presentation declared a frontend concern (§7.1): mandatory per-image fields are id + dataset_slug (resolvable per-image or via metadata fallback), with filename/type recommended; thumbnail_url is OPTIONAL and deprecated for emission — backends MAY include it during transition, frontends MUST NOT require it and SHOULD resolve presentation (thumbnails, signed URLs) from id + dataset_slug. Rationale: engines return image identities, not presentation; the frontend already owns presentation resolution (SAS image-url endpoint, gallery thumbnail resolution), so this keeps certified engines uniform and presentation logic in one place.
2.0 2026-07-07 Reconciled with tested reality (TEST-E2E-01): dual-backend scope (KAP ai/ + kavai_server), system_type: dataadk, real ChatRequest + proxy layer, auth behavior (mid-stream RUN_ERROR vs proxy 401), per-backend custom-event emission matrix, the three gallery contracts (event / marker / inline), CDC_PROVENANCE + CDC_FULL_PAYLOAD downgraded to reserved, thread semantics (server-side, history ignored), verification section. Declared single source of truth; handbook copy lint-enforced.
1.0 2026-04-03 Initial contract — Orion-era full event catalog, provenance, thread management