huskies/.huskies/specs/tech/LLM_CONTEXT_FROM_EVENTS.md

# LLM Context From Events

Design overview for making any LLM-driven chat persona (Timmy at the
gateway, Sally at a single sled, future personas) aware of huskies
events without the user having to re-narrate them.

## Goal

**Update the LLM's context non-intrusively when a state transition
happens.** No new LLM turn is fired; events are simply visible to the
LLM the next time the user (or anything else) does cause it to run.
The LLM should never need to be told what already happened inside
huskies.

## Guiding Principle

**Transports have nothing to do with LLMs.** A transport (Matrix bot,
web UI, CLI, future TUIs) is a pure courier — it relays user text in,
LLM text out, and never owns LLM-facing state. Anything the LLM needs
to know lives in huskies, behind a single `assemble_prompt_context`
helper that the transport calls. Adding a new transport must require
zero changes to the event-awareness path.

## Three things this doc is NOT

1. **Triggers** — `on StoryMerged{1122} do Rebuild`. These are
   deterministic subscribers; they should never invoke the LLM. Covered
   in a separate design.
2. **Proactive wake** — running an LLM turn *because* an event fired,
   without the user typing. Costs tokens, risks ramble. Explicitly out
   of scope here; a separate decision to make later.
3. **A transport feature** — this design assumes any transport that
   invokes the LLM uses the same context-assembly helper. Matrix bot,
   web UI, CLI all funnel through it.

## Why Past Attempts Have Failed

- **Buffer lived on the transport**, not on huskies. The current
  `BotContext.pending_pipeline_events` (`server/src/chat/transport/matrix/bot/context.rs:103-116`)
  is Matrix-only; web UI users see nothing of the kind, and the buffer
  dies with the bot process.
- **Process-local, RAM-only**. Server rebuild → buffer empty. Any
  events between the old binary's last user turn and the new binary's
  first are silently lost.
- **Unbounded `mpsc` channels drop under lag.** The server logs
  routinely show `[xxx-sub] Subscriber lagged, skipped N event(s)`.
  When the subscriber feeding the buffer falls behind, events vanish
  without being recorded.
- **No end-to-end test.** Nothing asserts "fire event E, send user
  message M, the LLM's prompt contains E."
- **No cross-process aggregation.** Events in a sled have no path to
  the gateway-side LLM context without bespoke plumbing per event type.

## Architecture at a Glance

```
┌────────────┐  ┌────────────┐  ┌────────────┐
│  Sled A    │  │  Sled B    │  │  Sled C    │
│ event_log/ │  │ event_log/ │  │ event_log/ │  ◄── source of truth
└─────┬──────┘  └─────┬──────┘  └─────┬──────┘     (CRDT-backed)
      │               │               │
      └───────────────┼───────────────┘
                      ▼
              ┌────────────────────┐
              │ Gateway aggregator │  ◄── tail-merges all sled logs
              │  event_view/       │     into a single ordered stream
              └─────────┬──────────┘
                        │
                        ▼
            ┌────────────────────────┐
            │ Per-LLM-session state  │  ◄── scope filter +
            │  sessions/<id>/        │     high-water mark per stream
            └─────────┬──────────────┘
                      │
                      ▼
            ┌────────────────────────┐
            │ assemble_prompt_context│  ◄── single helper used by
            │   (session_id) -> Str  │     every transport before
            └─────────┬──────────────┘     each LLM turn
                      │
       ┌──────────────┼──────────────┐
       ▼              ▼              ▼
   Matrix bot      Web UI         CLI / TUI
```

## Event Model — Reuse What Already Exists

There is no need to invent a parallel event taxonomy. Huskies already
has a complete typed enum and a single broadcast bus:

- `server/src/pipeline_state/transition.rs` defines `PipelineEvent`
  with **30 variants** covering every state-machine transition
  (`DepsMet`, `GatesStarted/Passed/Failed`, `QaSkipped`,
  `MergeSucceeded/Failed/FailedFinal`, `Accepted`, `Block/Unblock`,
  `Abandon`, `Supersede`, `ReviewHold/Cleared`, `Reject`, `Triage`,
  `Close`, `Demote`, `Freeze/Unfreeze`, `MergemasterAttempted`,
  `FixupRequested`, `ReQueuedForQa`, `MergeAborted`,
  `HotfixRequested`, `MergeRetryStarted`).
- The same module defines `ExecutionEvent` with 7 variants for agent
  lifecycle (`SpawnRequested`, `SpawnedSuccessfully`, `Heartbeat`,
  `HitRateLimit`, `Exited`, `Stopped`, `Reset`).
- Every transition fires a `TransitionFired` event on a single internal
  bus. Ten subscribers already consume it (audit-log,
  worktree-create-sub, worktree-cleanup-sub, merge-failure-sub,
  merge-block-sub, done-archive-sub, content-gc, cost-rollup-sub,
  stage-notification-sub, event-triggers).

**The LLM context injector is just the 11th subscriber on the same
bus.** It writes typed events into the per-sled CRDT event log
described below; everything downstream reuses the existing taxonomy.

Each persisted entry carries:

```
struct LoggedEvent {
    id: EventId,                  // monotonic per sled
    sled_id: SledId,
    timestamp: UnixSeconds,
    transition: TransitionFired,  // story_id + from + to + PipelineEvent
                                  // (or ExecutionEvent — see open question)
}
```

The few events that genuinely don't fit the pipeline state machine
(e.g. `ProjectAdopted`, `Rebuilt`, `GatewayHealthChanged`) live in a
small, separately-enumerated `InfraEvent` enum, but the same log and
the same subscriber pattern still apply.

## Session Model

An LLM session is a first-class CRDT entity:

```
struct LlmSession {
    id: SessionId,
    persona: Persona,        // "Timmy", "Sally", ...
    scope: ScopeFilter,      // { sleds: All } | { sleds: Set<SledId> }
    high_water: BTreeMap<SledId, EventId>,   // per-stream
    created: UnixSeconds,
}
```

The session id is what the transport carries; it's not the Matrix room,
not the web socket id. A given Matrix room may map to one session; a
web UI tab may map to another. Multiple transports for the same human
can share a session if you want — that's a separate UX call.

## Prompt Assembly Contract

Every transport calls one helper before invoking the LLM:

```
fn assemble_prompt_context(session_id: SessionId) -> String
```

Behavior:

1. Read the session's scope filter and high-water marks.
2. Fetch events from the gateway aggregator that match the scope and
   are newer than the high-water marks.
3. Render them as a single `<system-reminder>` block, ordered by sled
   then timestamp.
4. Advance the high-water marks to the latest event seen, atomically
   with the LLM-turn-start CRDT op (so a crash mid-turn doesn't double-
   inject).
5. Return the rendered block (empty string if no new events).

The transport prepends the result to the user's prompt and invokes the
LLM as usual.

## Persistence & Reliability Rules

- **Event log is CRDT-backed.** Survives sled restart.
- **High-water marks are CRDT-backed.** Survives gateway restart.
- **Aggregator uses bounded queues with drop-oldest semantics**, and
  every drop logs `[event-agg] dropped N events for session <id>; client
  must re-fetch from <high-water>`. The aggregator never silently
  swallows events — if the queue is full, the session gets a sentinel
  event `EventStreamGap { from, to }` so the LLM can see it missed
  context.
- **End-to-end test required**: `fire(Event::StoryMerged{1122}) → user
  sends "what's going on?" → assembled prompt contains "1122 merged"`.

## Multi-Persona Scoping

The same machinery serves both Timmy and Sally:

| Persona | Scope filter                       | Notes                          |
|---------|------------------------------------|--------------------------------|
| Timmy   | `{ sleds: All }`                   | Gateway-wide; aware of every sled |
| Sally   | `{ sleds: { huskies-server } }`    | Single-sled; sled-local events only |
| Manny   | `{ sleds: { huskies, ketflix } }`  | Hypothetical; subset           |

Sally never has to know Timmy exists, and vice versa. Their sessions
advance their own high-water marks against the same underlying log.

## Decisions

| Decision               | Choice                              | Alternative                                  |
|------------------------|-------------------------------------|----------------------------------------------|
| Event publication      | Each sled owns its log              | Single global log: cross-sled bottleneck     |
| Aggregation            | Gateway tail-merges                 | Each session pulls from each sled directly: N×M fanout |
| Buffer location        | CRDT-persisted                      | In-process: lost on rebuild (current bug)    |
| Event identity         | Typed enum                          | Strings: structured-log creep, no compile-time safety |
| Drop semantics         | Drop-oldest + `EventStreamGap`      | Silent drop (current bug): LLM lies confidently |
| Session ↔ transport    | Session is separate from transport  | One per transport: web tab + Matrix get different views |
| Proactive LLM wake     | OUT OF SCOPE                        | Wake on every event: cost + ramble           |

## Open Questions

1. **Session lifecycle**. How are sessions created and garbage-
   collected? Created on first transport message? GC'd after N days
   idle?
2. **Event retention**. How long are events kept in the log? Forever
   feels wrong; "since last terminal session turn" feels right but
   needs care for multi-session readers.
3. **Multi-transport same session**. Should one human's Matrix and web
   UI share a session by default, or always be separate?
4. **Render budget**. If 500 events accumulated between turns, do we
   render all 500 or summarize? A `summarize_events` fallback path is
   probably worth designing in from the start.
5. **Aggregator placement when there is no gateway**. A standalone
   single-sled install has no gateway — does the sled itself host the
   aggregator? (Probably yes; trivially "aggregates" its own log.)

## Phasing

- **Phase 0 (now):** this design doc.
- **Phase 1:** typed `Event` enum + per-sled CRDT-backed event log;
  one publisher subscribes to existing pipeline transitions and writes
  `StoryStaged` / `StoryMerged` / `StoryMergeFailed`.
- **Phase 2:** `LlmSession` CRDT entity + `assemble_prompt_context`
  helper, wired into the Matrix bot's `handle_message` (replaces the
  existing `pending_pipeline_events` Vec). End-to-end test covering the
  fire-event → user-turn → prompt-contains-event contract.
- **Phase 3:** Gateway aggregator over multiple sleds; Timmy's session
  scoped to `All`. Sally's session scoped to a single sled.
- **Phase 4:** Web UI and any other transports migrated onto
  `assemble_prompt_context`; the Matrix-specific Vec deleted.
- **Phase 5:** Bounded queues + `EventStreamGap` sentinel; observability
  for `assemble_prompt_context` runs (events injected, gaps observed).

Each phase ships independently. Phase 2 alone delivers the user-facing
fix: Timmy sees what merged when you next say anything, without you
needing to re-narrate.