Adding show story mcp
This commit is contained in:
Timmy
@@ -0,0 +1,245 @@
|
||||
# LLM Context From Events
|
||||
|
||||
Design overview for making any LLM-driven chat persona (Timmy at the
|
||||
gateway, Sally at a single sled, future personas) aware of huskies
|
||||
events without the user having to re-narrate them.
|
||||
|
||||
## Goal
|
||||
|
||||
**Update the LLM's context non-intrusively when a state transition
|
||||
happens.** No new LLM turn is fired; events are simply visible to the
|
||||
LLM the next time the user (or anything else) does cause it to run.
|
||||
The LLM should never need to be told what already happened inside
|
||||
huskies.
|
||||
|
||||
## Guiding Principle
|
||||
|
||||
**Transports have nothing to do with LLMs.** A transport (Matrix bot,
|
||||
web UI, CLI, future TUIs) is a pure courier — it relays user text in,
|
||||
LLM text out, and never owns LLM-facing state. Anything the LLM needs
|
||||
to know lives in huskies, behind a single `assemble_prompt_context`
|
||||
helper that the transport calls. Adding a new transport must require
|
||||
zero changes to the event-awareness path.
|
||||
|
||||
## Three things this doc is NOT
|
||||
|
||||
1. **Triggers** — `on StoryMerged{1122} do Rebuild`. These are
|
||||
deterministic subscribers; they should never invoke the LLM. Covered
|
||||
in a separate design.
|
||||
2. **Proactive wake** — running an LLM turn *because* an event fired,
|
||||
without the user typing. Costs tokens, risks ramble. Explicitly out
|
||||
of scope here; a separate decision to make later.
|
||||
3. **A transport feature** — this design assumes any transport that
|
||||
invokes the LLM uses the same context-assembly helper. Matrix bot,
|
||||
web UI, CLI all funnel through it.
|
||||
|
||||
## Why Past Attempts Have Failed
|
||||
|
||||
- **Buffer lived on the transport**, not on huskies. The current
|
||||
`BotContext.pending_pipeline_events` (`server/src/chat/transport/matrix/bot/context.rs:103-116`)
|
||||
is Matrix-only; web UI users see nothing of the kind, and the buffer
|
||||
dies with the bot process.
|
||||
- **Process-local, RAM-only**. Server rebuild → buffer empty. Any
|
||||
events between the old binary's last user turn and the new binary's
|
||||
first are silently lost.
|
||||
- **Unbounded `mpsc` channels drop under lag.** The server logs
|
||||
routinely show `[xxx-sub] Subscriber lagged, skipped N event(s)`.
|
||||
When the subscriber feeding the buffer falls behind, events vanish
|
||||
without being recorded.
|
||||
- **No end-to-end test.** Nothing asserts "fire event E, send user
|
||||
message M, the LLM's prompt contains E."
|
||||
- **No cross-process aggregation.** Events in a sled have no path to
|
||||
the gateway-side LLM context without bespoke plumbing per event type.
|
||||
|
||||
## Architecture at a Glance
|
||||
|
||||
```
|
||||
┌────────────┐ ┌────────────┐ ┌────────────┐
|
||||
│ Sled A │ │ Sled B │ │ Sled C │
|
||||
│ event_log/ │ │ event_log/ │ │ event_log/ │ ◄── source of truth
|
||||
└─────┬──────┘ └─────┬──────┘ └─────┬──────┘ (CRDT-backed)
|
||||
│ │ │
|
||||
└───────────────┼───────────────┘
|
||||
▼
|
||||
┌────────────────────┐
|
||||
│ Gateway aggregator │ ◄── tail-merges all sled logs
|
||||
│ event_view/ │ into a single ordered stream
|
||||
└─────────┬──────────┘
|
||||
│
|
||||
▼
|
||||
┌────────────────────────┐
|
||||
│ Per-LLM-session state │ ◄── scope filter +
|
||||
│ sessions/<id>/ │ high-water mark per stream
|
||||
└─────────┬──────────────┘
|
||||
│
|
||||
▼
|
||||
┌────────────────────────┐
|
||||
│ assemble_prompt_context│ ◄── single helper used by
|
||||
│ (session_id) -> Str │ every transport before
|
||||
└─────────┬──────────────┘ each LLM turn
|
||||
│
|
||||
┌──────────────┼──────────────┐
|
||||
▼ ▼ ▼
|
||||
Matrix bot Web UI CLI / TUI
|
||||
```
|
||||
|
||||
## Event Model — Reuse What Already Exists
|
||||
|
||||
There is no need to invent a parallel event taxonomy. Huskies already
|
||||
has a complete typed enum and a single broadcast bus:
|
||||
|
||||
- `server/src/pipeline_state/transition.rs` defines `PipelineEvent`
|
||||
with **30 variants** covering every state-machine transition
|
||||
(`DepsMet`, `GatesStarted/Passed/Failed`, `QaSkipped`,
|
||||
`MergeSucceeded/Failed/FailedFinal`, `Accepted`, `Block/Unblock`,
|
||||
`Abandon`, `Supersede`, `ReviewHold/Cleared`, `Reject`, `Triage`,
|
||||
`Close`, `Demote`, `Freeze/Unfreeze`, `MergemasterAttempted`,
|
||||
`FixupRequested`, `ReQueuedForQa`, `MergeAborted`,
|
||||
`HotfixRequested`, `MergeRetryStarted`).
|
||||
- The same module defines `ExecutionEvent` with 7 variants for agent
|
||||
lifecycle (`SpawnRequested`, `SpawnedSuccessfully`, `Heartbeat`,
|
||||
`HitRateLimit`, `Exited`, `Stopped`, `Reset`).
|
||||
- Every transition fires a `TransitionFired` event on a single internal
|
||||
bus. Ten subscribers already consume it (audit-log,
|
||||
worktree-create-sub, worktree-cleanup-sub, merge-failure-sub,
|
||||
merge-block-sub, done-archive-sub, content-gc, cost-rollup-sub,
|
||||
stage-notification-sub, event-triggers).
|
||||
|
||||
**The LLM context injector is just the 11th subscriber on the same
|
||||
bus.** It writes typed events into the per-sled CRDT event log
|
||||
described below; everything downstream reuses the existing taxonomy.
|
||||
|
||||
Each persisted entry carries:
|
||||
|
||||
```
|
||||
struct LoggedEvent {
|
||||
id: EventId, // monotonic per sled
|
||||
sled_id: SledId,
|
||||
timestamp: UnixSeconds,
|
||||
transition: TransitionFired, // story_id + from + to + PipelineEvent
|
||||
// (or ExecutionEvent — see open question)
|
||||
}
|
||||
```
|
||||
|
||||
The few events that genuinely don't fit the pipeline state machine
|
||||
(e.g. `ProjectAdopted`, `Rebuilt`, `GatewayHealthChanged`) live in a
|
||||
small, separately-enumerated `InfraEvent` enum, but the same log and
|
||||
the same subscriber pattern still apply.
|
||||
|
||||
## Session Model
|
||||
|
||||
An LLM session is a first-class CRDT entity:
|
||||
|
||||
```
|
||||
struct LlmSession {
|
||||
id: SessionId,
|
||||
persona: Persona, // "Timmy", "Sally", ...
|
||||
scope: ScopeFilter, // { sleds: All } | { sleds: Set<SledId> }
|
||||
high_water: BTreeMap<SledId, EventId>, // per-stream
|
||||
created: UnixSeconds,
|
||||
}
|
||||
```
|
||||
|
||||
The session id is what the transport carries; it's not the Matrix room,
|
||||
not the web socket id. A given Matrix room may map to one session; a
|
||||
web UI tab may map to another. Multiple transports for the same human
|
||||
can share a session if you want — that's a separate UX call.
|
||||
|
||||
## Prompt Assembly Contract
|
||||
|
||||
Every transport calls one helper before invoking the LLM:
|
||||
|
||||
```
|
||||
fn assemble_prompt_context(session_id: SessionId) -> String
|
||||
```
|
||||
|
||||
Behavior:
|
||||
|
||||
1. Read the session's scope filter and high-water marks.
|
||||
2. Fetch events from the gateway aggregator that match the scope and
|
||||
are newer than the high-water marks.
|
||||
3. Render them as a single `<system-reminder>` block, ordered by sled
|
||||
then timestamp.
|
||||
4. Advance the high-water marks to the latest event seen, atomically
|
||||
with the LLM-turn-start CRDT op (so a crash mid-turn doesn't double-
|
||||
inject).
|
||||
5. Return the rendered block (empty string if no new events).
|
||||
|
||||
The transport prepends the result to the user's prompt and invokes the
|
||||
LLM as usual.
|
||||
|
||||
## Persistence & Reliability Rules
|
||||
|
||||
- **Event log is CRDT-backed.** Survives sled restart.
|
||||
- **High-water marks are CRDT-backed.** Survives gateway restart.
|
||||
- **Aggregator uses bounded queues with drop-oldest semantics**, and
|
||||
every drop logs `[event-agg] dropped N events for session <id>; client
|
||||
must re-fetch from <high-water>`. The aggregator never silently
|
||||
swallows events — if the queue is full, the session gets a sentinel
|
||||
event `EventStreamGap { from, to }` so the LLM can see it missed
|
||||
context.
|
||||
- **End-to-end test required**: `fire(Event::StoryMerged{1122}) → user
|
||||
sends "what's going on?" → assembled prompt contains "1122 merged"`.
|
||||
|
||||
## Multi-Persona Scoping
|
||||
|
||||
The same machinery serves both Timmy and Sally:
|
||||
|
||||
| Persona | Scope filter | Notes |
|
||||
|---------|------------------------------------|--------------------------------|
|
||||
| Timmy | `{ sleds: All }` | Gateway-wide; aware of every sled |
|
||||
| Sally | `{ sleds: { huskies-server } }` | Single-sled; sled-local events only |
|
||||
| Manny | `{ sleds: { huskies, ketflix } }` | Hypothetical; subset |
|
||||
|
||||
Sally never has to know Timmy exists, and vice versa. Their sessions
|
||||
advance their own high-water marks against the same underlying log.
|
||||
|
||||
## Decisions
|
||||
|
||||
| Decision | Choice | Alternative |
|
||||
|------------------------|-------------------------------------|----------------------------------------------|
|
||||
| Event publication | Each sled owns its log | Single global log: cross-sled bottleneck |
|
||||
| Aggregation | Gateway tail-merges | Each session pulls from each sled directly: N×M fanout |
|
||||
| Buffer location | CRDT-persisted | In-process: lost on rebuild (current bug) |
|
||||
| Event identity | Typed enum | Strings: structured-log creep, no compile-time safety |
|
||||
| Drop semantics | Drop-oldest + `EventStreamGap` | Silent drop (current bug): LLM lies confidently |
|
||||
| Session ↔ transport | Session is separate from transport | One per transport: web tab + Matrix get different views |
|
||||
| Proactive LLM wake | OUT OF SCOPE | Wake on every event: cost + ramble |
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. **Session lifecycle**. How are sessions created and garbage-
|
||||
collected? Created on first transport message? GC'd after N days
|
||||
idle?
|
||||
2. **Event retention**. How long are events kept in the log? Forever
|
||||
feels wrong; "since last terminal session turn" feels right but
|
||||
needs care for multi-session readers.
|
||||
3. **Multi-transport same session**. Should one human's Matrix and web
|
||||
UI share a session by default, or always be separate?
|
||||
4. **Render budget**. If 500 events accumulated between turns, do we
|
||||
render all 500 or summarize? A `summarize_events` fallback path is
|
||||
probably worth designing in from the start.
|
||||
5. **Aggregator placement when there is no gateway**. A standalone
|
||||
single-sled install has no gateway — does the sled itself host the
|
||||
aggregator? (Probably yes; trivially "aggregates" its own log.)
|
||||
|
||||
## Phasing
|
||||
|
||||
- **Phase 0 (now):** this design doc.
|
||||
- **Phase 1:** typed `Event` enum + per-sled CRDT-backed event log;
|
||||
one publisher subscribes to existing pipeline transitions and writes
|
||||
`StoryStaged` / `StoryMerged` / `StoryMergeFailed`.
|
||||
- **Phase 2:** `LlmSession` CRDT entity + `assemble_prompt_context`
|
||||
helper, wired into the Matrix bot's `handle_message` (replaces the
|
||||
existing `pending_pipeline_events` Vec). End-to-end test covering the
|
||||
fire-event → user-turn → prompt-contains-event contract.
|
||||
- **Phase 3:** Gateway aggregator over multiple sleds; Timmy's session
|
||||
scoped to `All`. Sally's session scoped to a single sled.
|
||||
- **Phase 4:** Web UI and any other transports migrated onto
|
||||
`assemble_prompt_context`; the Matrix-specific Vec deleted.
|
||||
- **Phase 5:** Bounded queues + `EventStreamGap` sentinel; observability
|
||||
for `assemble_prompt_context` runs (events injected, gaps observed).
|
||||
|
||||
Each phase ships independently. Phase 2 alone delivers the user-facing
|
||||
fix: Timmy sees what merged when you next say anything, without you
|
||||
needing to re-narrate.
|
||||
Reference in New Issue
Block a user