Tutorials · Extensions

Build the Test Triage extension

Package a focused Bun test runner with a failure-triage specialist, then prove required parameters, runtime tool resolution, approval behavior, and evidence-backed diagnosis.

Pattern under test

autohand.test-triage demonstrates that an extension agent can name a tool contributed by the same package. The agent receives the tool only after the complete registry snapshot is loaded and filtered.

autohand.test-triage/
  autohand.extension.json
  README.md
  tools/run-focused-test.json
  agents/failure-triage.md

Create the manifest

{
  "$schema": "https://raw.githubusercontent.com/autohandai/code-extensions/main/schema/autohand.extension.schema.json",
  "schemaVersion": 1,
  "extensionApi": 1,
  "id": "autohand.test-triage",
  "name": "Test Triage",
  "version": "1.0.0",
  "description": "Run a focused test and delegate evidence-based failure triage.",
  "license": "Apache-2.0",
  "repository": "https://github.com/autohandai/code-extensions",
  "contributes": {
    "tools": ["tools/run-focused-test.json"],
    "agents": ["agents/failure-triage.md"]
  }
}

Add the required-parameter tool

{
  "name": "run_focused_test",
  "description": "Run one focused test file with Bun",
  "parameters": {
    "type": "object",
    "properties": {
      "file": {
        "type": "string",
        "description": "Repository-relative test file"
      }
    },
    "required": ["file"]
  },
  "handler": "bun test {{file}}",
  "source": "user"
}

The handler cannot render without file. Requiring it in the JSON Schema makes omission a tool-input error instead of producing a broad bun test run by accident.

Keep the boundary focused. Do not replace file with a free-form command or args string. This extension promises one test file, not arbitrary shell execution.

Add the failure-triage agent

---
description: Reproduce and triage focused test failures before proposing a fix
tools: read_file, fff_grep, run_focused_test
---
Start from the exact failing test and error. Reproduce it, trace the real
production path, distinguish product failures from environment noise, and
propose the smallest contract-preserving correction. Do not weaken assertions
merely to make a test pass.

Notice that the specialist is not given broad mutation tools. Its job is diagnosis. A parent agent can decide whether to implement a proposed fix after reviewing the evidence.

Validate registry resolution

autohand extensions validate ./autohand.test-triage --json
autohand extensions install ./autohand.test-triage --link
autohand extensions show autohand.test-triage --json
autohand extensions doctor

Inspection should show run_focused_test and failure-triage. If the tool conflicts with a built-in, standalone tool, or another extension, the package contributes nothing and diagnostics identify the conflict. The agent's allowlist cannot resurrect the rejected tool.

Create a controlled failing test

Use a disposable Bun project with one test whose expected value is intentionally wrong. Record the exact file path and expected failure text before involving the agent.

import { expect, test } from "bun:test";
import { normalizeName } from "../src/normalize-name";

test("normalizes surrounding whitespace", () => {
  expect(normalizeName("  Ada  ")).toBe("Ada Lovelace");
});

Ask Autohand:

Delegate to failure-triage. Reproduce tests/normalize-name.test.ts,
read the production function, distinguish a bad expectation from a product bug,
and return evidence. Do not edit files.

Verify approval and denial

bun test runs through the same shell authorization path as an equivalent built-in command action. Test both outcomes:

  1. Approve once and confirm only the named test file runs.
  2. Inspect the output the specialist cites.
  3. Deny a second invocation and confirm it reports that reproduction was blocked.
  4. Ensure no fallback tool expands into the full test suite without your approval.

Test runtime removal

/extensions disable autohand.test-triage
/extensions show autohand.test-triage
/extensions enable autohand.test-triage
/extensions remove autohand.test-triage --yes

After disable, both the agent definition and its tool disappear from the active snapshot. After enable they return together. Removal of a linked installation must leave the authored directory untouched.