Currently the benchmark loads only the single file pointed to by target.skill. Skills that use a folder structure (e.g. SKILL/SKILL.md + SKILL/references/*.md) give models instructions to "read `references/setup.md`" but no way to actually do so — the reference content never reaches the model.
Proposed change: If target.skill points to a file inside a directory that also contains other .md files, concatenate them all into the skill context passed to each model. The simplest implementation: if target.skill is SKILL/SKILL.md and SKILL/ contains other files, treat the whole SKILL/ folder as the skill document.
Alternatively, add a target.skillDir config field that explicitly points at the folder root, and the loader recursively reads all .md files under it.
Why: Makes skills with reference docs work out of the box without requiring agentic mode or manual inlining.
Currently the benchmark loads only the single file pointed to by
target.skill. Skills that use a folder structure (e.g.SKILL/SKILL.md+SKILL/references/*.md) give models instructions to "read `references/setup.md`" but no way to actually do so — the reference content never reaches the model.Proposed change: If
target.skillpoints to a file inside a directory that also contains other.mdfiles, concatenate them all into the skill context passed to each model. The simplest implementation: iftarget.skillisSKILL/SKILL.mdandSKILL/contains other files, treat the wholeSKILL/folder as the skill document.Alternatively, add a
target.skillDirconfig field that explicitly points at the folder root, and the loader recursively reads all.mdfiles under it.Why: Makes skills with reference docs work out of the box without requiring
agenticmode or manual inlining.