Story 53: Add QA agent role with request_qa MCP tool
- Add `qa` agent entry to `.story_kit/project.toml` with a detailed prompt covering code quality scan, test verification, manual testing support, and structured report generation - Add `move_story_to_qa` function in `agents.rs` that moves a work item from `work/2_current/` to `work/3_qa/` and auto-commits (idempotent) - Add `request_qa` MCP tool in `mcp.rs` that moves the story to `work/3_qa/` and starts the QA agent on the existing worktree - Add unit tests for `move_story_to_qa` (moves, idempotent, error cases) - Update `tools_list_returns_all_tools` test to expect 27 tools Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -75,6 +75,72 @@ max_budget_usd = 5.00
|
||||
prompt = "You are working in a git worktree on story {{story_id}}. Read CLAUDE.md first, then .story_kit/README.md to understand the dev process. Pick up the story from .story_kit/work/ - move it to work/2_current/ if needed. Follow the SDTW process through implementation and verification (Steps 1-3). The worktree and feature branch already exist - do not create them. Check .mcp.json for MCP tools. Do NOT accept the story or merge - commit your work and stop. If the user asks to review your changes, tell them to run: cd \"{{worktree_path}}\" && git difftool {{base_branch}}...HEAD\n\nIMPORTANT: When all your work is committed, call report_completion as your FINAL action: report_completion(story_id='{{story_id}}', agent_name='{{agent_name}}', summary='<brief summary of what you implemented>'). The server will run cargo clippy and tests automatically to verify your work."
|
||||
system_prompt = "You are a full-stack engineer working autonomously in a git worktree. Follow the Story-Driven Test Workflow strictly. Run cargo clippy and biome checks before considering work complete. Commit all your work before finishing - use a descriptive commit message. Do not accept stories, move them to archived, or merge to master - a human will do that. Do not coordinate with other agents - focus on your assigned story. ALWAYS call report_completion as your absolute final action after committing."
|
||||
|
||||
[[agent]]
|
||||
name = "qa"
|
||||
role = "Reviews coder work in worktrees: runs quality gates, generates testing plans, and reports findings."
|
||||
model = "sonnet"
|
||||
max_turns = 40
|
||||
max_budget_usd = 4.00
|
||||
prompt = """You are the QA agent for story {{story_id}}. Your job is to review the coder's work in the worktree and produce a structured QA report.
|
||||
|
||||
Read CLAUDE.md first, then .story_kit/README.md to understand the dev process.
|
||||
|
||||
## Your Workflow
|
||||
|
||||
### 1. Code Quality Scan
|
||||
- Run `git diff master...HEAD --stat` to see what files changed
|
||||
- Run `git diff master...HEAD` to review the actual changes for obvious coding mistakes (unused imports, dead code, unhandled errors, hardcoded values)
|
||||
- Run `cargo clippy --all-targets --all-features` and note any warnings
|
||||
- If a `frontend/` directory exists:
|
||||
- Run `pnpm run build` and note any TypeScript errors
|
||||
- Run `npx @biomejs/biome check src/` and note any linting issues
|
||||
|
||||
### 2. Test Verification
|
||||
- Run `cargo test` and verify all tests pass
|
||||
- If `frontend/` exists: run `pnpm test --run` and verify all frontend tests pass
|
||||
- Review test quality: look for tests that are trivial or don't assert meaningful behavior
|
||||
|
||||
### 3. Manual Testing Support
|
||||
- Build the server: run `cargo build` and note success/failure
|
||||
- If build succeeds: find a free port (try 3010-3020) and attempt to start the server
|
||||
- Generate a testing plan including:
|
||||
- URL to visit in the browser
|
||||
- Things to check in the UI
|
||||
- curl commands to exercise relevant API endpoints
|
||||
- Kill the test server when done: `pkill -f story-kit-server || true`
|
||||
|
||||
### 4. Produce Structured Report
|
||||
Call report_completion as your FINAL action with a summary in this format:
|
||||
|
||||
```
|
||||
## QA Report for {{story_id}}
|
||||
|
||||
### Code Quality
|
||||
- clippy: PASS/FAIL (details)
|
||||
- TypeScript build: PASS/FAIL/SKIP (details)
|
||||
- Biome lint: PASS/FAIL/SKIP (details)
|
||||
- Code review findings: (list any issues found, or "None")
|
||||
|
||||
### Test Verification
|
||||
- cargo test: PASS/FAIL (N tests)
|
||||
- pnpm test: PASS/FAIL/SKIP (N tests)
|
||||
- Test quality issues: (list any trivial/weak tests, or "None")
|
||||
|
||||
### Manual Testing Plan
|
||||
- Server URL: http://localhost:PORT (or "Build failed")
|
||||
- Pages to visit: (list)
|
||||
- Things to check: (list)
|
||||
- curl commands: (list)
|
||||
|
||||
### Overall: PASS/FAIL
|
||||
```
|
||||
|
||||
## Rules
|
||||
- Do NOT modify any code — read-only review only
|
||||
- If the server fails to start, still provide the testing plan with curl commands
|
||||
- Call report_completion as your FINAL action"""
|
||||
system_prompt = "You are a QA agent. Your job is read-only: review code quality, run tests, try to start the server, and produce a structured QA report. Do not modify code. Call report_completion as your final action."
|
||||
|
||||
[[agent]]
|
||||
name = "mergemaster"
|
||||
role = "Merges completed coder work into master, runs quality gates, archives stories, and cleans up worktrees."
|
||||
|
||||
@@ -943,6 +943,42 @@ pub fn move_story_to_merge(project_root: &Path, story_id: &str) -> Result<(), St
|
||||
)
|
||||
}
|
||||
|
||||
/// Move a story/bug from `work/2_current/` to `work/3_qa/` and auto-commit.
|
||||
///
|
||||
/// This stages a work item for QA review before merging to master.
|
||||
/// Idempotent: if already in `3_qa/`, returns Ok without committing.
|
||||
pub fn move_story_to_qa(project_root: &Path, story_id: &str) -> Result<(), String> {
|
||||
let sk = project_root.join(".story_kit").join("work");
|
||||
let current_path = sk.join("2_current").join(format!("{story_id}.md"));
|
||||
let qa_dir = sk.join("3_qa");
|
||||
let qa_path = qa_dir.join(format!("{story_id}.md"));
|
||||
|
||||
if qa_path.exists() {
|
||||
// Already in 3_qa/ — idempotent, nothing to do.
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
if !current_path.exists() {
|
||||
return Err(format!(
|
||||
"Work item '{story_id}' not found in work/2_current/. Cannot move to 3_qa/."
|
||||
));
|
||||
}
|
||||
|
||||
std::fs::create_dir_all(&qa_dir)
|
||||
.map_err(|e| format!("Failed to create work/3_qa/ directory: {e}"))?;
|
||||
std::fs::rename(¤t_path, &qa_path)
|
||||
.map_err(|e| format!("Failed to move '{story_id}' to 3_qa/: {e}"))?;
|
||||
|
||||
eprintln!("[lifecycle] Moved '{story_id}' from work/2_current/ to work/3_qa/");
|
||||
|
||||
let msg = format!("story-kit: queue {story_id} for QA");
|
||||
git_stage_and_commit(
|
||||
project_root,
|
||||
&[qa_path.as_path(), current_path.as_path()],
|
||||
&msg,
|
||||
)
|
||||
}
|
||||
|
||||
/// Move a bug from `work/2_current/` or `work/1_upcoming/` to `work/5_archived/` and auto-commit.
|
||||
///
|
||||
/// * If the bug is in `2_current/`, it is moved to `5_archived/` and committed.
|
||||
@@ -1901,6 +1937,75 @@ mod tests {
|
||||
assert!(result.unwrap_err().contains("not found in work/2_current/"));
|
||||
}
|
||||
|
||||
// ── move_story_to_qa tests ────────────────────────────────────────────────
|
||||
|
||||
#[test]
|
||||
fn move_story_to_qa_moves_file_and_commits() {
|
||||
use std::fs;
|
||||
use tempfile::tempdir;
|
||||
|
||||
let tmp = tempdir().unwrap();
|
||||
let repo = tmp.path();
|
||||
init_git_repo(repo);
|
||||
|
||||
let current_dir = repo.join(".story_kit/work/2_current");
|
||||
fs::create_dir_all(¤t_dir).unwrap();
|
||||
let story_file = current_dir.join("30_story_qa_test.md");
|
||||
fs::write(&story_file, "---\nname: QA Test\ntest_plan: approved\n---\n").unwrap();
|
||||
|
||||
Command::new("git")
|
||||
.args(["add", "."])
|
||||
.current_dir(repo)
|
||||
.output()
|
||||
.unwrap();
|
||||
Command::new("git")
|
||||
.args(["commit", "-m", "add story"])
|
||||
.current_dir(repo)
|
||||
.output()
|
||||
.unwrap();
|
||||
|
||||
move_story_to_qa(repo, "30_story_qa_test").unwrap();
|
||||
|
||||
let qa_path = repo.join(".story_kit/work/3_qa/30_story_qa_test.md");
|
||||
assert!(!story_file.exists(), "2_current file should be gone");
|
||||
assert!(qa_path.exists(), "3_qa file should exist");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn move_story_to_qa_idempotent_when_already_in_qa() {
|
||||
use std::fs;
|
||||
use tempfile::tempdir;
|
||||
|
||||
let tmp = tempdir().unwrap();
|
||||
let repo = tmp.path();
|
||||
init_git_repo(repo);
|
||||
|
||||
let qa_dir = repo.join(".story_kit/work/3_qa");
|
||||
fs::create_dir_all(&qa_dir).unwrap();
|
||||
fs::write(
|
||||
qa_dir.join("31_story_test.md"),
|
||||
"---\nname: Test\ntest_plan: approved\n---\n",
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
// Should succeed without error even though there's nothing to move
|
||||
move_story_to_qa(repo, "31_story_test").unwrap();
|
||||
assert!(qa_dir.join("31_story_test.md").exists());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn move_story_to_qa_errors_when_not_in_current() {
|
||||
use tempfile::tempdir;
|
||||
|
||||
let tmp = tempdir().unwrap();
|
||||
let repo = tmp.path();
|
||||
init_git_repo(repo);
|
||||
|
||||
let result = move_story_to_qa(repo, "99_nonexistent");
|
||||
assert!(result.is_err());
|
||||
assert!(result.unwrap_err().contains("not found in work/2_current/"));
|
||||
}
|
||||
|
||||
// ── move_story_to_archived with 4_merge source ────────────────────────────
|
||||
|
||||
#[test]
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
use crate::agents::{close_bug_to_archive, move_story_to_archived, move_story_to_merge};
|
||||
use crate::agents::{close_bug_to_archive, move_story_to_archived, move_story_to_merge, move_story_to_qa};
|
||||
use crate::config::ProjectConfig;
|
||||
use crate::http::context::AppContext;
|
||||
use crate::http::settings::get_editor_command_from_store;
|
||||
@@ -742,6 +742,24 @@ fn handle_tools_list(id: Option<Value>) -> JsonRpcResponse {
|
||||
},
|
||||
"required": ["story_id"]
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "request_qa",
|
||||
"description": "Trigger QA review of a completed story worktree: moves the item from work/2_current/ to work/3_qa/ and starts the qa agent to run quality gates, tests, and generate a manual testing plan.",
|
||||
"inputSchema": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"story_id": {
|
||||
"type": "string",
|
||||
"description": "Story identifier (e.g. '53_story_qa_agent_role')"
|
||||
},
|
||||
"agent_name": {
|
||||
"type": "string",
|
||||
"description": "Agent name to use for QA (defaults to 'qa')"
|
||||
}
|
||||
},
|
||||
"required": ["story_id"]
|
||||
}
|
||||
}
|
||||
]
|
||||
}),
|
||||
@@ -797,6 +815,8 @@ async fn handle_tools_call(
|
||||
// Mergemaster tools
|
||||
"merge_agent_work" => tool_merge_agent_work(&args, ctx).await,
|
||||
"move_story_to_merge" => tool_move_story_to_merge(&args, ctx),
|
||||
// QA tools
|
||||
"request_qa" => tool_request_qa(&args, ctx).await,
|
||||
_ => Err(format!("Unknown tool: {tool_name}")),
|
||||
};
|
||||
|
||||
@@ -1360,6 +1380,42 @@ fn tool_move_story_to_merge(args: &Value, ctx: &AppContext) -> Result<String, St
|
||||
))
|
||||
}
|
||||
|
||||
// ── QA tool implementations ───────────────────────────────────────
|
||||
|
||||
async fn tool_request_qa(args: &Value, ctx: &AppContext) -> Result<String, String> {
|
||||
let story_id = args
|
||||
.get("story_id")
|
||||
.and_then(|v| v.as_str())
|
||||
.ok_or("Missing required argument: story_id")?;
|
||||
let agent_name = args
|
||||
.get("agent_name")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("qa");
|
||||
|
||||
let project_root = ctx.agents.get_project_root(&ctx.state)?;
|
||||
|
||||
// Move story from work/2_current/ to work/3_qa/
|
||||
move_story_to_qa(&project_root, story_id)?;
|
||||
|
||||
// Start the QA agent on the story worktree
|
||||
let info = ctx
|
||||
.agents
|
||||
.start_agent(&project_root, story_id, Some(agent_name))
|
||||
.await?;
|
||||
|
||||
serde_json::to_string_pretty(&json!({
|
||||
"story_id": info.story_id,
|
||||
"agent_name": info.agent_name,
|
||||
"status": info.status.to_string(),
|
||||
"worktree_path": info.worktree_path,
|
||||
"message": format!(
|
||||
"Story '{story_id}' moved to work/3_qa/ and QA agent '{}' started.",
|
||||
info.agent_name
|
||||
),
|
||||
}))
|
||||
.map_err(|e| format!("Serialization error: {e}"))
|
||||
}
|
||||
|
||||
/// Run `git log <base>..HEAD --oneline` in the worktree and return the commit
|
||||
/// summaries, or `None` if git is unavailable or there are no new commits.
|
||||
async fn get_worktree_commits(worktree_path: &str, base_branch: &str) -> Option<Vec<String>> {
|
||||
@@ -1519,7 +1575,8 @@ mod tests {
|
||||
assert!(names.contains(&"close_bug"));
|
||||
assert!(names.contains(&"merge_agent_work"));
|
||||
assert!(names.contains(&"move_story_to_merge"));
|
||||
assert_eq!(tools.len(), 26);
|
||||
assert!(names.contains(&"request_qa"));
|
||||
assert_eq!(tools.len(), 27);
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
||||
Reference in New Issue
Block a user