Agent Skills for Context Engineering: Audit What Actually Activates

The practical answer is: treat Agent Skills for Context Engineering as healthy only after three separate receipts. First, the installed manifest must resolve to the expected skill directories. Second, boundary prompts must activate the intended skill—or produce an explicit ambiguous result. Third, the requested task must pass a verifier that sits outside the skill router. Installation alone proves none of the last two. A structurally valid SKILL.md can have a description that overlaps its neighbors. A router can put the expected skill somewhere in a shortlist without loading it. Even a correctly loaded skill can produce a missing or invalid deliverable. I pinned the repository at commit c578e85 , ran its deterministic repository validator, and replayed all 23 supplied activation cases. The repository validator returned 17 skills, zero errors, and zero warnings. The built in activation rule passed 23 of 23. A stricter diagnostic that asks whether the expected primary skill ranked first matched 20 of 23. That gap is not a defect verdict; it is a precise list of boundaries that need a live host canary. Separate discovery, activation, and useful work The Agent Skills specification defines a skill as a directory containing SKILL.md , with optional scripts/ , references/ , and assets/ . Its progressive disclosure model has three stages: hosts see name and description metadata at startup, load the full instructions after activation, and retrieve additional resources only when required. That design protects the context window, but it also creates distinct failure boundaries: Boundary Evidence What it does not prove Repository version exact commit or release that the host installed it Manifest declared skill path resolves that every directory is valid Discovery expected skill IDs are visible that the right one will activate Activation host records the loaded skill ID that its instructions were followed Task result requested artifact exists that it is correct Outcome independent verifier passes that the next run will also pass At the pinned commit, the repository’s Open Plugins manifest points to ./skills/ . The repository’s own deterministic validate repo.py checks directory names, frontmatter, manifest parity, required sections, research artifacts, activation fixtures, and other corpus contracts. On this checkout it reported: That is a strong manifest receipt. It says the checked repository is internally coherent under its validator. It does not say Claude Code, Codex, Cursor, or another host discovered those exact 17 skills, because installation roots and routing behavior belong to the host. The reasonable default is therefore small: pin one repository version, install one supported layout from the repository documentation, enumerate discovered skill IDs, and fail closed if the observed set differs. Do not begin a routing benchmark while the manifest receipt is red. Read the activation gate literally The repository includes a deterministic smoke checker, check activation cases.py . It extracts terms from each skill’s description and “When to Activate” section, ranks skills by shared terms with a fixture prompt, and evaluates 23 boundary cases. Its pass rule is deliberately tolerant: the expected primary skill must appear in the top three, and no explicitly rejected skill may appear there. Running the supplied cases produced: The stricter diagnostic exposed these three cases: Fixture Expected primary Lexical rank one Built in result General deterministic quality gate evaluation long horizon prompting pass; expected is in top three Consolidate 17 specialized tools tool design harness engineering pass; expected is in top three Choose a multi agent topology multi agent patterns long horizon prompting pass; expected is in top three This does not establish a 20/23 live routing accuracy rate. The checker is a deterministic token overlap smoke test, not the host’s model, prompt, policy, or multi skill activation mechanism. A host may select the expected skill, activate multiple valid skills, apply stronger semantic routing, or ignore the collection entirely. The useful finding is narrower: these prompts sit near documented description boundaries. They deserve live canaries before an operator trusts automatic activation. The same rule applies after descriptions change, a skill is added, or the host upgrades its router. I packaged the comparison in a content free audit. From the article artifact directory, point it at the pinned checkout: The script checks the observed Git commit, counts and validates the skill directories, verifies the plugin’s skill path, replays the 23 activation cases, applies both pass rules, and leaves the outcome receipt unverified. That last state is intentional. Static files cannot prove what a live agent host loaded or whether the user’s work succeeded. Run one live canary at each ambiguous boundary A useful live test needs a known task, an activation receipt, and a deterministic outcome. Do not store the full prompt or model transcript merely to prove routing. A privacy minimal receipt can retain: For the general quality gate boundary, ask the host to build a deterministic regression gate over a small fixture. The activation receipt should show whether evaluation , an acceptable adjacent skill, or no skill loaded. The outcome verifier should then run the gate against one passing and one failing fixture and require the expected exit codes and report fields. For tool consolidation, provide a fixed catalog with overlapping tool names and require a reduced manifest plus a coverage test. Selecting tool design is evidence about routing; preserving every required capability without duplicate ambiguous tools is the outcome. For multi agent topology, provide a fixed dependency graph with one parallel branch and one ordered handoff. The router receipt records which coordination skill loaded. The outcome checker verifies that the proposed topology respects dependencies, identifies the handoff owner, and does not claim completion before worker results are aggregated. Use explicit states rather than one green flag: 1. manifest invalid — paths, names, descriptions, or counts do not match the pinned collection. 2. discoverable — the host sees the expected skill IDs, but no activation canary has run. 3. routing ambiguous — the expected skill is not selected under the declared policy, or multiple candidates appear without an allowed combination. 4. loaded unverified — a relevant skill loaded, but the task verifier is missing. 5. outcome failed — routing occurred, but the requested artifact failed its independent check. 6. healthy for case — version, discovery, activation, and outcome receipts all pass for this fixture. The suffix matters. A pass is scoped to a host version, collection commit, routing policy, case, and verifier. It is not permanent evidence for every future prompt. There is a practical trade off. Requiring exactly one skill can create false failures when a task legitimately spans evaluation and harness engineering . Allow a declared set of acceptable secondary skills, but keep one owner for the final outcome check. Conversely, accepting any skill in the top three is useful for a smoke test but too weak to prove that a host actually loaded the intended instructions. Keep the health layer outside the router The operational pattern is straightforward: pin the collection commit; compare the installed and discovered skill sets; replay deterministic boundary fixtures after collection or host changes; run live canaries only for meaningful boundaries; retain skill IDs and verifier results, not sensitive prompt content; declare success only after the user’s artifact passes an external check. This is where agent health differs from context engineering itself. The skill collection can teach compression, memory, evaluation, tools, and multi agent design. A health layer asks whether the right guidance was available, whether it was used, whether work progressed, and whether the promised result exists. Sidewisp fits that health boundary conceptually, but the current product boundary is important: Sidewisp is currently in private preview. The public site and interactive demonstration are live; production skill activation collection, host adapters, and automated recovery are not shipped. Sidewisp should not be described as currently observing or repairing these installations. For Agent Skills for Context Engineering, keep the acceptance rule exact: a clean repository validator is a manifest receipt; a routing shortlist is an activation diagnostic; a live loaded skill record is an activation receipt; and only an independent task verifier can close the outcome. Preserve every missing layer as unknown instead of turning a successful install into a false green.