From ce9bdbbb9d687566ae7b0795b9f6750ba7e83d15 Mon Sep 17 00:00:00 2001 From: Dave Date: Wed, 18 Mar 2026 10:48:55 +0000 Subject: [PATCH] story-kit: done 266_story_matrix_bot_structured_conversation_history --- ...rix_bot_structured_conversation_history.md | 60 +++++++++++++++++++ 1 file changed, 60 insertions(+) create mode 100644 .story_kit/work/5_done/266_story_matrix_bot_structured_conversation_history.md diff --git a/.story_kit/work/5_done/266_story_matrix_bot_structured_conversation_history.md b/.story_kit/work/5_done/266_story_matrix_bot_structured_conversation_history.md new file mode 100644 index 0000000..c3c96c7 --- /dev/null +++ b/.story_kit/work/5_done/266_story_matrix_bot_structured_conversation_history.md @@ -0,0 +1,60 @@ +--- +name: "Matrix bot structured conversation history" +agent: coder-opus +--- + +# Story 266: Matrix bot structured conversation history + +## User Story + +As a user chatting with the Matrix bot, I want it to remember and own its prior responses naturally, so that conversations feel like talking to one continuous entity rather than a new instance each message. + +## Acceptance Criteria + +- [ ] Conversation history is passed as structured API messages (user/assistant turns) rather than a flattened text prefix +- [ ] Claude recognises its prior responses as its own, maintaining consistent personality across a conversation +- [ ] Per-room history survives server restarts (persisted to disk or database) +- [ ] Rolling window trimming still applies to keep context bounded +- [ ] Multi-user rooms still attribute messages to the correct sender + +## Investigation Notes (2026-03-18) + +The current implementation attempts session resumption via `--resume ` but it's not working: + +### Code path: how session resumption is supposed to work + +1. `server/src/matrix/bot.rs:671-676` — `handle_message()` reads `conv.session_id` from the per-room `RoomConversation` to get the resume ID. +2. `server/src/matrix/bot.rs:717` — passes `resume_session_id` to `provider.chat_stream()`. +3. `server/src/llm/providers/claude_code.rs:57` — `chat_stream()` stores it as `resume_id`. +4. `server/src/llm/providers/claude_code.rs:170-173` — if `resume_session_id` is `Some`, appends `--resume ` to the `claude -p` command. +5. `server/src/llm/providers/claude_code.rs:348` — `process_json_event()` looks for `json["session_id"]` in each streamed NDJSON event and sends it via a oneshot channel (`sid_tx`). +6. `server/src/llm/providers/claude_code.rs:122` — after the PTY exits, `sid_rx.await.ok()` captures the session ID (or `None` if never sent). +7. `server/src/matrix/bot.rs:785-787` — stores `new_session_id` back into `conv.session_id` and persists via `save_history()`. + +### What's broken + +- **No session_id captured:** `.story_kit/matrix_history.json` contains conversation entries but no `session_id`. `RoomConversation.session_id` is always `None`. +- **Root cause:** `claude -p --output-format stream-json` may not emit a `session_id` in its NDJSON events, or the parser at step 5 isn't matching the actual event shape. The oneshot channel never fires. +- **Effect:** Every message spawns a fresh Claude Code process with no `--resume` flag. Each turn is a blank slate. +- **History persistence works fine** — serialization round-trips correctly (test at `bot.rs:1335-1339`). The problem is purely that `--resume` is never invoked. + +### Debugging steps + +1. Run `claude -p "hello" --output-format stream-json --verbose 2>/dev/null` manually and inspect the NDJSON for a `session_id` field. Check what event type carries it and whether the key name matches what `process_json_event()` expects. +2. If `session_id` is present but nested differently (e.g. inside an `event` wrapper), fix the JSON path at `claude_code.rs:348`. +3. If `-p` mode doesn't emit `session_id` at all, consider an alternative: pass conversation history as a structured prompt prefix, or switch to the Claude API directly. + +### Previous attempt failed (2026-03-18) + +A sonnet coder attempted this story but did NOT fix the root cause. It rewrote the `chat_stream()` call in `bot.rs` to look identical to what was already there — it never investigated why `session_id` isn't being captured. The merge auto-resolver then jammed the duplicate call inside the `tokio::select!` permission loop, producing mismatched braces. The broken merge was reverted. + +**What the coder must actually do:** + +1. **Do NOT rewrite the `chat_stream()` call or the `tokio::select!` loop in `bot.rs`.** That code is correct and handles permission forwarding (story 275). Do not touch it. +2. **The bug is in `claude_code.rs`, not `bot.rs`.** The `process_json_event()` function at line ~348 looks for `json["session_id"]` but it's likely never finding it. Start by running step 1 above to see what the actual NDJSON output looks like. +3. **If `claude -p` doesn't emit `session_id` at all**, the `--resume` approach won't work. In that case, the fix is to pass conversation history as a prompt prefix (prepend prior turns to the user message) or use `--continue` instead of `--resume`, or call the Claude API directly instead of shelling out to the CLI. +4. **Rebase onto current master before starting.** Master has changed significantly (spike 92, story 275 permission handling, gitignore changes). + +## Out of Scope + +- TBD