Lab 5.2: Skill Discovery Debugging¶

Goal¶

Take a skill that should be useful but does not reliably load, diagnose why discovery fails, and rewrite it so the agent can find it from natural user phrasing.

In Lab 5.1, you learned how to write skills. In this lab, you learn the more important skill: debugging discoverability.

What You'll Learn¶

Why vague descriptions cause skills to disappear
How to test trigger phrases without guessing
How to rewrite a skill description like a search index
How to separate a discovery problem from a content problem

Prerequisites¶

You completed Lab 5.1 or have at least one skill in .opencode/skills/
You can run opencode from a project directory
You understand that a skill's description is what the agent uses to decide whether to load it

The Broken Skill¶

You are given this skill:

---
name: api-safety
description: Helpful API advice.
---

## API Safety Checklist

When reviewing or writing an API endpoint, check:

1. Validate all incoming request data.
2. Return clear error responses.
3. Do not leak secrets or stack traces.
4. Use the right HTTP status code.
5. Add tests for success and failure paths.

## Example Review Output

- Missing validation for `email`
- Returns `500` for a user input error; should return `400`
- Error message exposes internal database field name

The content is useful. The problem is the description.

Helpful API advice is too broad. It does not say when to use the skill, what user phrasing should trigger it, or what job the skill performs.

Part 1: Create the Broken Skill¶

In a practice project, create the skill folder:

mkdir -p .opencode/skills/api-safety

Create .opencode/skills/api-safety/SKILL.md with the broken skill above.

Check that the folder name and frontmatter name match exactly:

.opencode/skills/api-safety/SKILL.md
name: api-safety

If these do not match, you have a naming problem, not a discovery problem.

Part 2: Run a Baseline Discovery Test¶

Start OpenCode in the project root:

opencode

Ask three natural questions that should make an API safety checklist useful:

Can you review this API endpoint for security and error handling issues?

I am adding a POST endpoint. What should I check before shipping it?

Look at this route handler and tell me if validation or status codes are wrong.

Watch whether the agent loads api-safety.

If the skill does not load, do not immediately edit the body. The checklist is not the problem yet. The agent is failing before it ever reads the checklist.

Part 3: Diagnose the Description¶

Open the skill and inspect only this line:

description: Helpful API advice.

Ask yourself four questions:

Does it name the task? No. It says "advice," not "review," "write," or "audit."
Does it include trigger phrases a user might type? No. It does not mention "endpoint," "route handler," "validation," or "status code."
Does it say what the skill provides? Barely. It does not mention a checklist.
Does it define when not to use the skill? No. It could match almost anything API-related.

This is the core debugging move: treat the description as a search query, not a title.

Part 4: Rewrite the Description¶

Replace the description with this stronger version:

description: Use when the user asks to write, review, or audit an API endpoint, route handler, or HTTP request flow. Provides a checklist for validation, error responses, status codes, secret leakage, and API tests.

Why this works:

Task verbs: write, review, audit
Domain nouns: API endpoint, route handler, HTTP request flow
Checklist terms: validation, error responses, status codes, secret leakage, API tests
Clear scope: API safety, not every API question

Your updated frontmatter should look like this:

---
name: api-safety
description: Use when the user asks to write, review, or audit an API endpoint, route handler, or HTTP request flow. Provides a checklist for validation, error responses, status codes, secret leakage, and API tests.
---

Part 5: Retest with the Same Prompts¶

Restart OpenCode so it picks up the updated skill, then ask the same three questions from Part 2.

Record the result in a simple table:

Prompt	Loaded Before?	Loaded After?
Review endpoint for security and error handling	Yes/No	Yes/No
Adding a POST endpoint	Yes/No	Yes/No
Validation or status codes are wrong	Yes/No	Yes/No

You are looking for improvement, not magic. If one prompt still misses, inspect the words in that prompt and decide whether the description should include them.

Part 6: Write One More Description Variant¶

Now write your own improved description. Keep it under 1024 characters.

Use this formula:

Use when the user asks to [task verbs] [domain object]. Provides [specific artifact/checklist/template] for [important subtopics].

Example:

description: Use when the user asks to review, harden, or design an API endpoint or route handler. Provides an API safety checklist covering input validation, error shape, HTTP status codes, sensitive data leaks, auth boundaries, and test cases.

Compare your version to the provided version. Which one better matches how you would ask for help?

Part 7: Debug Your Own Skill¶

Pick one skill from Lab 5.1 and run the same process:

Write three natural prompts that should trigger it.
Test whether it loads.
If it misses, inspect the description.
Add missing task verbs, domain nouns, and output nouns.
Retest with the same prompts.

Do not rewrite the whole skill unless the body is actually wrong. In most discovery failures, the body is fine and the description is weak.

Common Discovery Bugs¶

Symptom	Likely Cause	Fix
Skill never loads	Description is too vague	Add task verbs and domain nouns
Wrong skill loads	Description is too broad	Narrow the scope and add exclusions
Skill loads for unrelated prompts	Description sounds like a general helper	Replace generic words with specific triggers
Skill exists but is invisible	Folder name and `name` do not match	Make them identical
Skill loads but output is poor	Body lacks examples or checklist	Improve the content after frontmatter

Submission¶

Create a short lab note with:

# Lab 5.2: Skill Discovery Debugging

## Broken Skill
- Skill name: api-safety
- Original description: Helpful API advice.

## Diagnosis
[Explain why the original description failed.]

## Revised Description
[Paste your final description.]

## Test Results
| Prompt | Loaded Before? | Loaded After? | Notes |
|--------|----------------|---------------|-------|
| ... | ... | ... | ... |

## My Own Skill Debug
- Skill name: [your skill]
- What changed: [one sentence]
- Result: [one sentence]

Checkpoints¶

[ ] Created the broken api-safety skill
[ ] Tested three natural trigger prompts before editing
[ ] Diagnosed the description, not just the skill body
[ ] Rewrote the description with task verbs and domain nouns
[ ] Retested the exact same prompts
[ ] Debugged one of your own skills from Lab 5.1
[ ] Submitted a lab note with before/after results