Skip to content

AIEVALS-226: split canonical skill assets - #500

Open
Himnish wants to merge 7 commits into
harness:mainfrom
Himnish:AIEVALS-226-canonical-skill
Open

Himnish wants to merge 7 commits into
harness:mainfrom
Himnish:AIEVALS-226-canonical-skill

Conversation

@Himnish

@Himnish Himnish commented Nov 18, 2026

Copy link
Copy Markdown

Scope

  • Embeds the canonical modules/aievals/skill/SKILL.md source.
  • Adds deterministic skill placement and the TypeScript generator/hash formatter.
  • Keeps the generated UI artifact out of the CLI PR.

Dependencies

Standalone; the runtime wiring PR consumes the exported skill placement API.

Verification

  • go test ./modules/aievals/...
  • git diff --check

Explicit exclusions

No local-demo, source bootstrap, report persistence, metric advisor, JAX-RS, generated UI artifact, or unrelated docs.

@mohitsuman

Copy link
Copy Markdown
Collaborator

cosmos review

@augmentcode augmentcode Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deep Code Review Agent🐛

Review completed with 9 suggestions.

Reviewed commit: 92f0e70

Comment thread modules/aievals/skill.go Resolved
Comment thread modules/aievals/skill.go Resolved
Comment thread modules/aievals/skill.go Resolved
Comment thread modules/aievals/skillmodule.go Resolved
Comment thread modules/aievals/skillmodule.go Resolved
Comment thread modules/aievals/skill/SKILL.md Resolved
Comment thread modules/aievals/skill/SKILL.md Resolved
Comment thread modules/aievals/skill/SKILL.md Resolved
Comment thread modules/aievals/cmd/skillgen/main.go Resolved
@Himnish

Himnish commented Nov 18, 2026

Copy link
Copy Markdown
Author

cosmos review

@Himnish

Himnish commented Nov 18, 2026

Copy link
Copy Markdown
Author

Superseded by Himnish/cli#1 in native stack #6. All replacement layers passed GitHub CI. This branch and discussion are retained for reference.

@mohitsuman

Copy link
Copy Markdown
Collaborator

cosmos review

@augmentcode augmentcode Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deep Code Review Agent🐛

Review completed with 0 suggestions.

Reviewed commit: 3f0293c

GitHub could not attach the following findings inline to reviewed commit 3f0293c622c1fd3f5bf4db55bdb6e504adf07ef:

modules/aievals/skillmodule.go:24 — FormatSkillModule replaces tabs and newlines in the actual -skill path with spaces before putting it in the regeneration command. skillgen can read the original filename successfully, but rerunning the printed command can fail or read a different file.

Severity: medium


🤖 Was this useful? React with 👍 or 👎

modules/aievals/skill_test.go:57 — TestPlaceSkillSkipsSymlinkedDetectedDirectory only covers a symlink that detectSkillDir rejects before it calls safeDetectedSkillPath. The test would still pass if validation of nested destination directories were removed, so it doesn't protect against a regression that writes the skill outside the repository.

Severity: medium


🤖 Was this useful? React with 👍 or 👎

@Himnish

Himnish commented Nov 18, 2026

Copy link
Copy Markdown
Author

Addressed both findings in the review summary in 1f37762:

  • The regeneration command now shell-escapes the exact -skill path, keeping the generated header on one line. The test checks the argument Bash actually passes for a path with a tab, quote, backslash, and trailing newline.
  • The destination test now checks nested .cursor/skills and .cursor/skills/aievals symlinks as well as .cursor itself. Each case verifies stdout fallback and that the outside directory remains untouched.

The skill package race tests passed. The canonical SKILL.md is unchanged.

@mohitsuman

Copy link
Copy Markdown
Collaborator

cosmos grammar-review


// Command skillgen copies the canonical AI Evals skill into a TypeScript module.
// It has no dependencies beyond the standard library and this module.
package main

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It isn't wired into the harness binary, Who runs it, and where? It's weird to have a main file inside the module here.. I see if it is for creating a skill inside .cursor/ or .claude/, I suggest that it can be a command inside the cli to place that skill there...

@Himnish Himnish Nov 19, 2026 •

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

skillgen is just what I use to regenerate the PlatformUI copy from SKILL.md. Users don't run it. The command in #501 writes the embedded skill to the coding agent's skill directory or prints it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants