huskies/.huskies/specs/tech/LLM_CONTEXT_FROM_EVENTS.md

LLM Context From Events

Design overview for making any LLM-driven chat persona (Timmy at the gateway, Sally at a single sled, future personas) aware of huskies events without the user having to re-narrate them.

Goal

Update the LLM's context non-intrusively when a state transition happens. No new LLM turn is fired; events are simply visible to the LLM the next time the user (or anything else) does cause it to run. The LLM should never need to be told what already happened inside huskies.

Guiding Principle

Transports have nothing to do with LLMs. A transport (Matrix bot, web UI, CLI, future TUIs) is a pure courier — it relays user text in, LLM text out, and never owns LLM-facing state. Anything the LLM needs to know lives in huskies, behind a single assemble_prompt_context helper that the transport calls. Adding a new transport must require zero changes to the event-awareness path.

Three things this doc is NOT

1. Triggers — on StoryMerged{1122} do Rebuild. These are deterministic subscribers; they should never invoke the LLM. Covered in a separate design.
2. Proactive wake — running an LLM turn because an event fired, without the user typing. Costs tokens, risks ramble. Explicitly out of scope here; a separate decision to make later.
3. A transport feature — this design assumes any transport that invokes the LLM uses the same context-assembly helper. Matrix bot, web UI, CLI all funnel through it.

Why Past Attempts Have Failed

• Buffer lived on the transport, not on huskies. The current BotContext.pending_pipeline_events (server/src/chat/transport/matrix/bot/context.rs:103-116) is Matrix-only; web UI users see nothing of the kind, and the buffer dies with the bot process.
• Process-local, RAM-only. Server rebuild → buffer empty. Any events between the old binary's last user turn and the new binary's first are silently lost.
• Unbounded mpsc channels drop under lag. The server logs routinely show [xxx-sub] Subscriber lagged, skipped N event(s). When the subscriber feeding the buffer falls behind, events vanish without being recorded.
• No end-to-end test. Nothing asserts "fire event E, send user message M, the LLM's prompt contains E."
• No cross-process aggregation. Events in a sled have no path to the gateway-side LLM context without bespoke plumbing per event type.

Architecture at a Glance

┌────────────┐  ┌────────────┐  ┌────────────┐
│  Sled A    │  │  Sled B    │  │  Sled C    │
│ event_log/ │  │ event_log/ │  │ event_log/ │  ◄── source of truth
└─────┬──────┘  └─────┬──────┘  └─────┬──────┘     (CRDT-backed)
      │               │               │
      └───────────────┼───────────────┘
                      ▼
              ┌────────────────────┐
              │ Gateway aggregator │  ◄── tail-merges all sled logs
              │  event_view/       │     into a single ordered stream
              └─────────┬──────────┘
                        │
                        ▼
            ┌────────────────────────┐
            │ Per-LLM-session state  │  ◄── scope filter +
            │  sessions/<id>/        │     high-water mark per stream
            └─────────┬──────────────┘
                      │
                      ▼
            ┌────────────────────────┐
            │ assemble_prompt_context│  ◄── single helper used by
            │   (session_id) -> Str  │     every transport before
            └─────────┬──────────────┘     each LLM turn
                      │
       ┌──────────────┼──────────────┐
       ▼              ▼              ▼
   Matrix bot      Web UI         CLI / TUI

Event Model — Reuse What Already Exists

There is no need to invent a parallel event taxonomy. Huskies already has a complete typed enum and a single broadcast bus:

• server/src/pipeline_state/transition.rs defines PipelineEvent with 30 variants covering every state-machine transition (DepsMet, GatesStarted/Passed/Failed, QaSkipped, MergeSucceeded/Failed/FailedFinal, Accepted, Block/Unblock, Abandon, Supersede, ReviewHold/Cleared, Reject, Triage, Close, Demote, Freeze/Unfreeze, MergemasterAttempted, FixupRequested, ReQueuedForQa, MergeAborted, HotfixRequested, MergeRetryStarted).
• The same module defines ExecutionEvent with 7 variants for agent lifecycle (SpawnRequested, SpawnedSuccessfully, Heartbeat, HitRateLimit, Exited, Stopped, Reset).
• Every transition fires a TransitionFired event on a single internal bus. Ten subscribers already consume it (audit-log, worktree-create-sub, worktree-cleanup-sub, merge-failure-sub, merge-block-sub, done-archive-sub, content-gc, cost-rollup-sub, stage-notification-sub, event-triggers).

The LLM context injector is just the 11th subscriber on the same bus. It writes typed events into the per-sled CRDT event log described below; everything downstream reuses the existing taxonomy.

Each persisted entry carries:

struct LoggedEvent {
    id: EventId,                  // monotonic per sled
    sled_id: SledId,
    timestamp: UnixSeconds,
    transition: TransitionFired,  // story_id + from + to + PipelineEvent
                                  // (or ExecutionEvent — see open question)
}

The few events that genuinely don't fit the pipeline state machine (e.g. ProjectAdopted, Rebuilt, GatewayHealthChanged) live in a small, separately-enumerated InfraEvent enum, but the same log and the same subscriber pattern still apply.

Session Model

An LLM session is a first-class CRDT entity:

struct LlmSession {
    id: SessionId,
    persona: Persona,        // "Timmy", "Sally", ...
    scope: ScopeFilter,      // { sleds: All } | { sleds: Set<SledId> }
    high_water: BTreeMap<SledId, EventId>,   // per-stream
    created: UnixSeconds,
}

The session id is what the transport carries; it's not the Matrix room, not the web socket id. A given Matrix room may map to one session; a web UI tab may map to another. Multiple transports for the same human can share a session if you want — that's a separate UX call.

Prompt Assembly Contract

Every transport calls one helper before invoking the LLM:

fn assemble_prompt_context(session_id: SessionId) -> String

Behavior:

1. Read the session's scope filter and high-water marks.
2. Fetch events from the gateway aggregator that match the scope and are newer than the high-water marks.
3. Render them as a single <system-reminder> block, ordered by sled then timestamp.
4. Advance the high-water marks to the latest event seen, atomically with the LLM-turn-start CRDT op (so a crash mid-turn doesn't double- inject).
5. Return the rendered block (empty string if no new events).

The transport prepends the result to the user's prompt and invokes the LLM as usual.

Persistence & Reliability Rules

• Event log is CRDT-backed. Survives sled restart.
• High-water marks are CRDT-backed. Survives gateway restart.
• Aggregator uses bounded queues with drop-oldest semantics, and every drop logs [event-agg] dropped N events for session <id>; client must re-fetch from <high-water>. The aggregator never silently swallows events — if the queue is full, the session gets a sentinel event EventStreamGap { from, to } so the LLM can see it missed context.
• End-to-end test required: fire(Event::StoryMerged{1122}) → user sends "what's going on?" → assembled prompt contains "1122 merged".

Multi-Persona Scoping

The same machinery serves both Timmy and Sally:

Persona	Scope filter	Notes
Timmy	{ sleds: All }	Gateway-wide; aware of every sled
Sally	{ sleds: { huskies-server } }	Single-sled; sled-local events only
Manny	{ sleds: { huskies, ketflix } }	Hypothetical; subset

Sally never has to know Timmy exists, and vice versa. Their sessions advance their own high-water marks against the same underlying log.

Decisions

Decision	Choice	Alternative
Event publication	Each sled owns its log	Single global log: cross-sled bottleneck
Aggregation	Gateway tail-merges	Each session pulls from each sled directly: N×M fanout
Buffer location	CRDT-persisted	In-process: lost on rebuild (current bug)
Event identity	Typed enum	Strings: structured-log creep, no compile-time safety
Drop semantics	Drop-oldest + EventStreamGap	Silent drop (current bug): LLM lies confidently
Session ↔ transport	Session is separate from transport	One per transport: web tab + Matrix get different views
Proactive LLM wake	OUT OF SCOPE	Wake on every event: cost + ramble

Open Questions

1. Session lifecycle. How are sessions created and garbage- collected? Created on first transport message? GC'd after N days idle?
2. Event retention. How long are events kept in the log? Forever feels wrong; "since last terminal session turn" feels right but needs care for multi-session readers.
3. Multi-transport same session. Should one human's Matrix and web UI share a session by default, or always be separate?
4. Render budget. If 500 events accumulated between turns, do we render all 500 or summarize? A summarize_events fallback path is probably worth designing in from the start.
5. Aggregator placement when there is no gateway. A standalone single-sled install has no gateway — does the sled itself host the aggregator? (Probably yes; trivially "aggregates" its own log.)

Phasing

• Phase 0 (now): this design doc.
• Phase 1: typed Event enum + per-sled CRDT-backed event log; one publisher subscribes to existing pipeline transitions and writes StoryStaged / StoryMerged / StoryMergeFailed.
• Phase 2: LlmSession CRDT entity + assemble_prompt_context helper, wired into the Matrix bot's handle_message (replaces the existing pending_pipeline_events Vec). End-to-end test covering the fire-event → user-turn → prompt-contains-event contract.
• Phase 3: Gateway aggregator over multiple sleds; Timmy's session scoped to All. Sally's session scoped to a single sled.
• Phase 4: Web UI and any other transports migrated onto assemble_prompt_context; the Matrix-specific Vec deleted.
• Phase 5: Bounded queues + EventStreamGap sentinel; observability for assemble_prompt_context runs (events injected, gaps observed).

Each phase ships independently. Phase 2 alone delivers the user-facing fix: Timmy sees what merged when you next say anything, without you needing to re-narrate.