story-kit: accept 298_story_bot_htop_command_with_live_updating_process_dashboard
This commit is contained in:
Dave
@@ -1,54 +0,0 @@
|
||||
---
|
||||
name: "Track per-agent token usage for cost visibility and optimisation"
|
||||
---
|
||||
|
||||
# Story 296: Track per-agent token usage for cost visibility and optimisation
|
||||
|
||||
## User Story
|
||||
|
||||
As a project owner, I want to see how many tokens each agent consumes per story, so that I can identify expensive operations and optimise token usage across the pipeline.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] Implement per-agent token tracking that captures input tokens, output tokens, and cache tokens for each agent run
|
||||
- [ ] Token usage is recorded per story and per agent (e.g. coder-1 on story 293 used X tokens)
|
||||
- [ ] Running totals are visible — either via MCP tool, web UI, or both
|
||||
- [ ] Historical token usage is persisted so it survives server restarts (e.g. in story files or a separate log)
|
||||
- [ ] Data is structured to support later analysis (e.g. which agent types are most expensive, which stories cost the most)
|
||||
|
||||
## Research Notes
|
||||
|
||||
Claude Code's JSON stream already emits all the data we need. No external library required.
|
||||
|
||||
**Data available in the `result` event at end of each agent session:**
|
||||
```json
|
||||
{
|
||||
"type": "result",
|
||||
"total_cost_usd": 1.57,
|
||||
"usage": {
|
||||
"input_tokens": 7,
|
||||
"output_tokens": 475,
|
||||
"cache_creation_input_tokens": 185020,
|
||||
"cache_read_input_tokens": 810585
|
||||
},
|
||||
"modelUsage": {
|
||||
"claude-opus-4-6[1m]": {
|
||||
"inputTokens": 7,
|
||||
"outputTokens": 475,
|
||||
"cacheReadInputTokens": 810585,
|
||||
"cacheCreationInputTokens": 185020,
|
||||
"costUSD": 1.57
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Where to hook in:**
|
||||
- `server/src/llm/providers/claude_code.rs` — `process_json_event()` already parses the JSON stream but currently ignores usage data from the `result` event
|
||||
- Parse `usage` + `total_cost_usd` from the `result` event and pipe it to the agent completion handler in `server/src/agents/pool.rs`
|
||||
|
||||
**No external libraries needed** — Anthropic SDK, LiteLLM, Helicone, Langfuse etc. are all overkill since we have direct access to Claude Code's output stream.
|
||||
|
||||
## Out of Scope
|
||||
|
||||
- TBD
|
||||
@@ -1,20 +0,0 @@
|
||||
---
|
||||
name: "Improve bot status command formatting"
|
||||
---
|
||||
|
||||
# Story 297: Improve bot status command formatting
|
||||
|
||||
## User Story
|
||||
|
||||
As a user reading the bot's status output in Matrix, I want to see clean story numbers and titles (not filenames), with agent assignments shown inline, so that the output is easy to scan at a glance.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] Status output shows story number and title (e.g. '293 — Register all bot commands') not the full filename stem
|
||||
- [ ] Each story shows which agent is working on it if one is assigned (e.g. 'coder-1 (sonnet)')
|
||||
- [ ] Stories with no agent assigned show no agent info rather than cluttering the output
|
||||
- [ ] Output is compact and scannable in a Matrix chat window
|
||||
|
||||
## Out of Scope
|
||||
|
||||
- TBD
|
||||
@@ -1,25 +0,0 @@
|
||||
---
|
||||
name: "Bot htop command with live-updating process dashboard"
|
||||
---
|
||||
|
||||
# Story 298: Bot htop command with live-updating process dashboard
|
||||
|
||||
## User Story
|
||||
|
||||
As a project owner in a Matrix room, I want to type "{bot_name} htop" and see a live-updating dashboard of system load and agent processes, so that I can monitor resource usage without needing terminal access.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] '{bot_name} htop' sends an initial status message showing load average and per-agent process info (CPU, memory, story assignment)
|
||||
- [ ] Message is edited every 5 seconds with updated stats
|
||||
- [ ] Only shows processes related to the project (agent PIDs and their child process trees)
|
||||
- [ ] '{bot_name} htop stop' stops the live updating and sends a final 'monitoring stopped' edit
|
||||
- [ ] Works regardless of what language/toolchain the agents are using (monitors by PID tree, not by process name)
|
||||
- [ ] Uses Matrix message editing (replacement events) to update in place
|
||||
- [ ] Only one htop session per room at a time — a second '{bot_name} htop' stops the existing session and starts a fresh one
|
||||
- [ ] Auto-stops after 5 minutes by default to prevent runaway editing
|
||||
- [ ] Optional timeout override: '{bot_name} htop 10m' to set a custom duration
|
||||
|
||||
## Out of Scope
|
||||
|
||||
- TBD
|
||||
@@ -1,22 +0,0 @@
|
||||
---
|
||||
name: "Bot git status command shows working tree and branch info"
|
||||
---
|
||||
|
||||
# Story 299: Bot git status command shows working tree and branch info
|
||||
|
||||
## User Story
|
||||
|
||||
As a project owner in a Matrix room, I want to type "{bot_name} git" and see the current git status (branch, uncommitted changes, how far ahead/behind remote), so that I can check the repo state without terminal access.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] '{bot_name} git' displays current branch name
|
||||
- [ ] Shows count of uncommitted changes (staged and unstaged) with filenames
|
||||
- [ ] Shows how many commits ahead/behind the remote branch
|
||||
- [ ] Output is formatted compactly for Matrix chat
|
||||
- [ ] Registered in the command registry in commands.rs so it appears in help output
|
||||
- [ ] Handled at bot level without LLM invocation
|
||||
|
||||
## Out of Scope
|
||||
|
||||
- TBD
|
||||
@@ -1,21 +0,0 @@
|
||||
---
|
||||
name: "Show token cost badge on pipeline board work items"
|
||||
---
|
||||
|
||||
# Story 300: Show token cost badge on pipeline board work items
|
||||
|
||||
## User Story
|
||||
|
||||
As a project owner viewing the pipeline board, I want to see the total token cost for each work item displayed as a badge, so that I can quickly spot expensive stories at a glance.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] Each work item on the pipeline board shows its total cost in USD as a small badge
|
||||
- [ ] Cost is fetched from the token_usage.jsonl data via a new API endpoint
|
||||
- [ ] Items with no recorded usage show no badge (not $0.00)
|
||||
- [ ] Cost updates when the pipeline refreshes (e.g. after an agent completes)
|
||||
- [ ] Expanded work item detail panel shows per-agent cost breakdown (coder, QA, mergemaster) with token counts
|
||||
|
||||
## Out of Scope
|
||||
|
||||
- TBD
|
||||
@@ -1,21 +0,0 @@
|
||||
---
|
||||
name: "Dedicated token usage page in web UI"
|
||||
---
|
||||
|
||||
# Story 301: Dedicated token usage page in web UI
|
||||
|
||||
## User Story
|
||||
|
||||
As a project owner, I want a dedicated token usage page in the web UI that shows per-story and per-agent cost breakdowns with totals, so that I can analyse where tokens are being spent and identify optimisation opportunities.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] New page/panel accessible from the main navigation
|
||||
- [ ] Shows a table of all recorded agent sessions with story, agent name, model, token counts, and cost
|
||||
- [ ] Sortable by cost, story, agent, or date
|
||||
- [ ] Shows summary totals: total cost, cost by agent type (coder vs QA vs mergemaster), cost by model (opus vs sonnet)
|
||||
- [ ] Data loads from the token_usage.jsonl log via API endpoint
|
||||
|
||||
## Out of Scope
|
||||
|
||||
- TBD
|
||||
@@ -1,22 +0,0 @@
|
||||
---
|
||||
name: "Bot cost command shows total and per-story token spend"
|
||||
---
|
||||
|
||||
# Story 302: Bot cost command shows total and per-story token spend
|
||||
|
||||
## User Story
|
||||
|
||||
As a project owner in a Matrix room, I want to type "{bot_name} cost" to see total token spend and the top most expensive stories, so that I can check burn rate from my phone.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] '{bot_name} cost' shows total spend for the last 24 hours
|
||||
- [ ] Shows top 5 most expensive stories from the last 24 hours with their costs
|
||||
- [ ] Shows cost breakdown by agent type (coder, QA, mergemaster) for the last 24 hours
|
||||
- [ ] Also shows an all-time total for context
|
||||
- [ ] Registered in the command registry so it appears in help output
|
||||
- [ ] Handled at bot level without LLM invocation
|
||||
|
||||
## Out of Scope
|
||||
|
||||
- TBD
|
||||
@@ -1,21 +0,0 @@
|
||||
---
|
||||
name: "Bot cost command with story filter for detailed breakdown"
|
||||
---
|
||||
|
||||
# Story 303: Bot cost command with story filter for detailed breakdown
|
||||
|
||||
## User Story
|
||||
|
||||
As a project owner in a Matrix room, I want to type "{bot_name} cost 293" to see a detailed token breakdown for a specific story, so that I can understand where the tokens went on an expensive item.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] '{bot_name} cost {story_number}' shows all agent sessions for that story
|
||||
- [ ] Each session shows agent name, model, input/output/cache tokens, and cost in USD
|
||||
- [ ] Shows total cost for the story at the bottom
|
||||
- [ ] Registered in the command registry (can share the 'cost' command with args parsing)
|
||||
- [ ] Returns a friendly message if no usage data exists for the story
|
||||
|
||||
## Out of Scope
|
||||
|
||||
- TBD
|
||||
@@ -1,23 +0,0 @@
|
||||
---
|
||||
name: "MCP tool to move stories between pipeline stages"
|
||||
---
|
||||
|
||||
# Story 304: MCP tool to move stories between pipeline stages
|
||||
|
||||
## User Story
|
||||
|
||||
As a bot operator (Timmy), I want an MCP tool that moves stories between pipeline stages, so that I don't need shell mv permissions to manage the pipeline.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] New MCP tool 'move_story' accepts story_id and target_stage (e.g. 'backlog', 'current', 'qa', 'merge', 'done')
|
||||
- [ ] Validates the story exists before moving
|
||||
- [ ] Handles the file move between stage directories
|
||||
- [ ] Returns a confirmation message with the old and new stage
|
||||
- [ ] Works for stories, bugs, spikes, and refactors
|
||||
- [ ] Replaces the need for shell mv commands to move story files
|
||||
- [ ] Tool description tells bots to prefer specific tools (accept_story, move_story_to_merge, request_qa) when available, and use move_story only for arbitrary moves that lack a dedicated tool (e.g. moving to backlog, moving ghost stories back to current)
|
||||
|
||||
## Out of Scope
|
||||
|
||||
- TBD
|
||||
@@ -1,21 +0,0 @@
|
||||
---
|
||||
name: "Bot show command displays story text in chat"
|
||||
---
|
||||
|
||||
# Story 305: Bot show command displays story text in chat
|
||||
|
||||
## User Story
|
||||
|
||||
As a project owner in a Matrix room, I want to type "{bot_name} show {story_number}" and see the full story text displayed in chat, so that I can review story details without accessing the file system.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] '{bot_name} show {number}' finds the story/bug/spike by number across all pipeline stages and displays its full markdown content
|
||||
- [ ] Output is formatted for readability in Matrix
|
||||
- [ ] Returns a friendly message if no story with that number exists
|
||||
- [ ] Registered in the command registry so it appears in help output
|
||||
- [ ] Handled at bot level without LLM invocation
|
||||
|
||||
## Out of Scope
|
||||
|
||||
- TBD
|
||||
@@ -1,26 +0,0 @@
|
||||
---
|
||||
name: "Replace manual_qa boolean with configurable qa mode field"
|
||||
---
|
||||
|
||||
# Story 306: Replace manual_qa boolean with configurable qa mode field
|
||||
|
||||
## User Story
|
||||
|
||||
As a project owner, I want to configure QA mode per-story and set a project-wide default, so that I can choose between human review, server-only gate checks, or full agent QA on a per-story basis.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] Replace manual_qa: true/false front matter field with qa: human|server|agent
|
||||
- [ ] qa: server — skip the QA agent entirely, rely on server's automated gate checks (clippy + tests + coverage). If gates pass, advance straight to merge
|
||||
- [ ] qa: agent — current behavior, spin up a QA agent (Claude session) to review code and run gates
|
||||
- [ ] qa: human — hold in QA for human approval after server gates pass (current manual_qa: true behavior)
|
||||
- [ ] Default qa mode is configurable in project.toml (e.g. default_qa = "server")
|
||||
- [ ] Set the initial default in project.toml to "server"
|
||||
- [ ] Per-story front matter qa field overrides the project default
|
||||
- [ ] Backwards compatible: existing stories without a qa field use the project default
|
||||
- [ ] Remove the old manual_qa field handling and replace with the new qa field throughout pool.rs, story_metadata.rs, and any other references
|
||||
- [ ] Update bot.toml.example and project.toml documentation to reflect the new field
|
||||
|
||||
## Out of Scope
|
||||
|
||||
- TBD
|
||||
Reference in New Issue
Block a user