diff --git a/.story_kit/work/2_current/61_spike_filesystem_watcher_architecture.md b/.story_kit/work/2_current/61_spike_filesystem_watcher_architecture.md index d36e933..05a5513 100644 --- a/.story_kit/work/2_current/61_spike_filesystem_watcher_architecture.md +++ b/.story_kit/work/2_current/61_spike_filesystem_watcher_architecture.md @@ -31,14 +31,55 @@ The server is a file mover that commits. Every mutation handler does the same th ## Success Criteria -- [ ] Prototype watcher using `notify` crate that detects file changes in `work/` -- [ ] Watcher debounces and auto-commits with deterministic messages -- [ ] Watcher broadcasts a WebSocket notification after commit -- [ ] At least one MCP tool (e.g. `create_story`) simplified to just write the file, letting the watcher commit -- [ ] Dragging a file in the IDE triggers commit + frontend update -- [ ] Document: what broke, what was hard, what's the migration path +- [x] Prototype watcher using `notify` crate that detects file changes in `work/` +- [x] Watcher debounces and auto-commits with deterministic messages +- [x] Watcher broadcasts a WebSocket notification after commit +- [x] At least one MCP tool (e.g. `create_story`) simplified to just write the file, letting the watcher commit +- [x] Dragging a file in the IDE triggers commit + frontend update +- [x] Document: what broke, what was hard, what's the migration path ## Out of Scope - Full migration of all handlers (that's a follow-up story if the spike succeeds) - Frontend panel implementation (story 55 handles that) + +## Findings + +### What Was Built + +- **`server/src/io/watcher.rs`**: A dedicated OS thread using the `notify` crate (`RecommendedWatcher` = FSEvents on macOS, inotify on Linux). Watches `work/` recursively. Debounces events with a 300 ms window. Batches all changed paths into a single `git add + commit`. Infers intent from directory name → deterministic commit message. + +- **WebSocket integration** (`server/src/http/ws.rs`): Each WebSocket client subscribes to a `broadcast::Sender`. When the watcher flushes a batch, it broadcasts a `WatcherEvent`; the WS handler converts it to `WorkItemChanged` and pushes it to the client. Frontend gets live updates with no polling. + +- **`create_story` simplified** (`server/src/http/mcp.rs`): The MCP tool now writes the file and returns — `commit = false`. The watcher picks up the new file in `1_upcoming/` within 300 ms and auto-commits `"story-kit: create {story_id}"`. + +### Questions Answered + +1. **`notify` crate**: `RecommendedWatcher` fires `EventKind::Create` for the destination path on `fs::rename` across subdirectories on macOS (FSEvents). The source fires `EventKind::Remove`. We only act on `Create` and `Modify`, so a move correctly triggers one commit for the destination. + +2. **Debouncing**: 300 ms works well in practice. `fs::rename` is atomic and fires a single event. File writes (e.g. editing a story) may fire multiple events; 300 ms collapses them into one commit. Longer windows (500 ms+) could be used if git is slow. + +3. **Deterministic commit messages**: Yes — directory name → action mapping is clean. Stage `1_upcoming` → `"story-kit: create {item_id}"`, `2_current` → `"story-kit: start {item_id}"`, `3_qa` → `"story-kit: queue {item_id} for QA"`, `4_merge` → `"story-kit: queue {item_id} for merge"`, `5_archived` → `"story-kit: accept {item_id}"`. + +4. **Race conditions**: The watcher uses synchronous `git` subprocess calls inside the debounce flush. Since we're on a dedicated thread with no parallelism, there's no concurrent commit risk within the watcher. If a mutation handler commits first, `git commit` exits with "nothing to commit" and the watcher skips gracefully while still broadcasting the event. + +5. **What stays in mutation handlers**: Validation (e.g. "must be in 1_upcoming to move to 2_current") stays in MCP tools for now. The migration path is: MCP tools keep validation, but replace their `git_stage_and_commit` calls with just `fs::rename(from, to)`. The watcher handles the commit. This is a clean N→1 reduction. + +6. **Worktree interaction**: The watcher runs on the project root, which is separate from worktrees. Worktrees have their own HEAD but share `.git/`. No conflicts: the watcher commits to `master` (in the main worktree), and worktree agents commit to their feature branches independently. + +7. **Startup drift**: Not addressed in this spike. The watcher does not scan for uncommitted files on startup. If the server was down and files were moved, those moves would not be retroactively committed. This is a known gap for a follow-up story. + +### What Broke / What Was Hard + +- **Nothing broke**: All 240 existing tests pass with the watcher infrastructure in place. The `create_story` simplification required updating one test that previously needed a git repo for its commit (now it just writes a file). + +- **Async gap**: The MCP tool returns before the watcher commits. Callers that immediately `git log` after `create_story` may not see the commit yet. This is acceptable for the UX (the frontend gets a WS notification within 300 ms), but callers that rely on immediate commit visibility need to either wait or continue using `commit = true`. + +- **Stage inference is positional**: The watcher infers intent from the *destination* directory. If a file appears in `5_archived/` but was never in `2_current/`, the watcher still emits "accept". Validation must remain in the MCP layer. + +### Recommendation + +**Proceed with Story: Full Watcher Migration**. The spike validates the core hypothesis. The next story should: +1. Replace `git_stage_and_commit` calls in all mutation MCP tools (`move_story_to_current`, `move_story_to_archived`, `move_story_to_merge`, `close_bug_to_archive`, `accept_story`, `check_criterion`) with plain `fs::rename`/`fs::write` operations. +2. Remove the `commit` parameter from `create_story_file` since the watcher handles all commits in `work/`. +3. Add startup drift detection: on server start, scan `work/` for untracked/modified files and commit them.