---
name: "MCP/HTTP capability to write a CRDT tombstone (delete op) for a story, to clear it from in-memory state"
---

# Story 521: MCP/HTTP capability to write a CRDT tombstone (delete op) for a story, to clear it from in-memory state

## User Story

**Note:** content stuffed into user_story per bug 509 workaround.

---

## Context

Today (2026-04-09) we discovered the hard way that **there is no way to remove a story from huskies's running in-memory state without restarting the server process**. The state machines that keep stories alive include:

1. The persisted CRDT op log (`crdt_ops` table) — direct sqlite DELETE works
2. The in-memory CRDT view (`CRDT_STATE` global in `server/src/crdt_state.rs`) — **no eviction API**
3. The in-memory content store (`CONTENT_STORE` in `server/src/db/mod.rs:46`) — has `delete_content()` but no MCP / HTTP exposure
4. The shadow `pipeline_items` table — direct sqlite DELETE works
5. Filesystem shadows under `.huskies/work/` — `find -delete` works
6. `timers.json` — direct file edit works

If a story gets into a bad state (split-brain, ghost row, runaway timer respawning it), we can scrub all the *persistent* layers (1, 4, 5, 6) but the *in-memory* layers (2, 3) keep regenerating it because some periodic code reads in-memory state and writes new ops based on what it sees. The only way to clear in-memory state today is `docker restart huskies`, which is heavy and disrupts the matrix bot, web UI, and any in-flight agents.

We need a **scoped, surgical capability** to write a CRDT tombstone op for a single story_id, which:
- Marks the in-memory item as `is_deleted = true`
- Persists the tombstone op to `crdt_ops` so future replays don't resurrect the story
- Removes the story from `CONTENT_STORE` 
- Cleans up any pending `timers.json` entries for the story
- Cancels any running agents on the story

…and exposes it as an MCP tool (e.g. `mcp__huskies__purge_story`) and ideally an HTTP endpoint, so an operator can "kill it with fire" without restarting the server.

## Real user story

As a huskies operator, I want a single MCP/HTTP call that completely removes a story from every layer of state — persistent AND in-memory — so I never have to restart the entire server just to clean up one stuck story.

## Observed 2026-04-09

We spent the last hour of this session whack-a-moling stories 503 and 478. Even after:
- `DELETE FROM pipeline_items WHERE id LIKE '503%'` ✓
- `DELETE FROM crdt_ops WHERE op_json LIKE '%503_bug_depends_on%'` ✓
- `mcp stop_agent + remove_worktree` for the running coders ✓
- `find .huskies/work -name '503_*' -delete` ✓
- emptying `timers.json` (multiple times — kept getting re-populated) ✓

…503 kept reappearing in `current` with new agents being spawned. The root cause: the in-memory `CRDT_STATE` (loaded from `crdt_ops` at startup at 18:19) still had 503 and 478 as live items, and a periodic code path was reading `crdt_state::read_all_items()`, seeing them as live, and triggering the auto-assign / rate-limit-retry chain.

Final resolution: `docker restart huskies` to wipe the in-memory state. Worked, but it's a sledgehammer.

## Implementation note

The bft-json-crdt library appears to support per-item delete via the `is_deleted: bool` field on each CRDT item (visible in the persisted op JSON we inspected today). Writing a delete op should look something like:

```rust
crdt_state::apply_and_persist(&mut state, |s| {
    s.crdt.doc.items[idx].delete()  // or whatever the BFT JSON CRDT delete API is
})
```

The op gets signed, applied to the in-memory state (marking the item deleted), and persisted to crdt_ops via the existing channel. Then `read_all_items()` should filter out `is_deleted: true` entries (it may already do this — verify in `extract_item_view`).

## Why this is distinct from bug 514 (delete_story full cleanup)

Bug 514 is about making the existing `delete_story` MCP tool do a full cleanup across all the layers we know about. **This** story is specifically about acquiring the *capability* to write a CRDT tombstone — without that, bug 514 can't be implemented correctly because it has no way to clear in-memory state. So 521 is a prerequisite for 514.

It's also a prerequisite for properly handling the fix for bug 510 (split-brain shadows) — when the reconcile pass detects a stale story, it needs a way to actually evict it. That eviction is what this story provides.

## Acceptance Criteria

- [ ] A new MCP tool (e.g. `mcp__huskies__purge_story`) is registered and callable
- [ ] The tool takes a story_id and returns a structured result indicating which layers were cleared (CRDT op, content store, timers, agents, worktree, filesystem)
- [ ] The tool writes a signed CRDT tombstone op (is_deleted: true) for the item, applies it to the in-memory CRDT, and persists it to crdt_ops
- [ ] After the tool runs, `read_all_items()` does NOT return the purged story (verify the filter handles is_deleted)
- [ ] After the tool runs, `read_content(story_id)` returns None (CONTENT_STORE entry is removed)
- [ ] After the tool runs, `timers.json` has no entries for the story
- [ ] After the tool runs, no agents are running on the story (stop_agent is called for any active ones)
- [ ] After the tool runs, the worktree at `.huskies/worktrees/{story_id}/` is removed
- [ ] After the tool runs, the filesystem shadow at `.huskies/work/*/{story_id}.md` is removed
- [ ] Idempotent: calling purge_story twice on the same story_id is safe and doesn't error
- [ ] Bug 514 (delete_story full cleanup) is updated to use this purge capability internally
- [ ] Regression test: insert a story via the normal write path, call purge_story, restart the server, verify the story is still gone (i.e. the tombstone persisted correctly)

## Out of Scope

- TBD
