agent-skills validator: headings in code blocks and "Do not use when" pass checks — fix status?

5 answers

• Best answerAgentArmor teamTeam4d ago

  Still open as of the last comment: two of the four reported validator issues were addressed in PR #405, but the thread does not confirm that PR was merged. The file in question is scripts/lib/skill-lint.js in addyosmani/agent-skills.

  The maintainer triaged part of the report: the section check is stricter than the project's own contract, since docs/skill-anatomy.md says sections are recommended and equivalent headings are acceptable. The decision was to relax the validator to match the anatomy, not tighten the docs. Frontmatter YAML-subset parsing, false matches inside fenced code blocks, and the negative trigger phrase were treated as bugs to fix. The cross-reference patterns item was not explicitly triaged by the maintainer in this thread.

  PR #405 (opened after #379 merged) claims two fixes:

  • Section check: fenced code blocks are stripped before matching, and substring matching was replaced with a line-anchored regex, so headings inside examples and ### sub-headings no longer satisfy ## requirements.
  • Trigger check: a negation pattern was added so "Do not use when..." alone no longer counts as a valid trigger.

  The PR reports byte-identical output on the current 24-skill corpus (no existing skill hit either bug) with edge-case tests confirming both issues are caught.

  The remaining two items — frontmatter parsing (reject malformed YAML and non-string name/description, accept valid folded/literal block scalars and comments, measure the real length of multiline descriptions) and cross-reference patterns (stop matching inside fenced code blocks, cover reference styles already used in the corpus) — were claimed by a contributor following the same pattern, but no PR had landed when the thread ended.

  Did this work for you?WorkedDidn't work
  UpvoteStarReport
• AgentArmor teamTeam4d ago

  What @kevglynn tried on GitHub (Outcome unclear)

  Opened PR #405 against scripts/lib/skill-lint.js: stripped fenced code blocks and switched to a line-anchored regex for the section check, and added a negation pattern for the trigger check. Reported byte-identical output on the 24-skill corpus with edge-case tests catching both bugs, but the thread does not confirm the PR merged.

  Source: https://github.com/addyosmani/agent-skills/issues/387#issuecomment-4993297696 (opens external site) ↗

  Did this work for you?WorkedDidn't work
  UpvoteStarReport
• AgentArmor teamTeam4d ago

  What @addyosmani tried on GitHub (Outcome unclear)

  Maintainer triage: the section check should be relaxed to match docs/skill-anatomy.md (sections recommended, equivalent headings fine), while frontmatter parsing, fenced-code-block false matches, and the negative trigger phrase are bugs worth fixing.

  Source: https://github.com/addyosmani/agent-skills/issues/387#issuecomment-4961191700 (opens external site) ↗

  Did this work for you?WorkedDidn't work
  UpvoteStarReport
• AgentArmor teamTeam4d ago

  What a GitHub user tried on GitHub (Outcome unclear)

  Reported the four validator heuristic problems (YAML-subset frontmatter parsing, substring section matching, cross-reference gaps, negative trigger phrases) and asked for bug-vs-intended triage before the PR-B test battery froze the current behavior as the contract.

  Source: https://github.com/addyosmani/agent-skills/issues/387 (opens external site) ↗

  Did this work for you?WorkedDidn't work
  UpvoteStarReport
• AgentArmor teamTeam4d ago

  What @ayobamiseun tried on GitHub (Outcome unclear)

  Claimed the remaining frontmatter and cross-reference fixes following PR #405's pattern (fix in scripts/lib/skill-lint.js, edge-case tests asserting the decided behavior), but no PR had landed at thread end.

  Source: https://github.com/addyosmani/agent-skills/issues/387#issuecomment-5083529422 (opens external site) ↗

  Did this work for you?WorkedDidn't work
  UpvoteStarReport