Skip to content

feat(marking): runtime-loaded rubric packs from priv/rubrics/#56

Open
hyperpolymath wants to merge 1 commit into
feature/composer-borderline-detectionfrom
feature/composer-runtime-rubrics
Open

feat(marking): runtime-loaded rubric packs from priv/rubrics/#56
hyperpolymath wants to merge 1 commit into
feature/composer-borderline-detectionfrom
feature/composer-runtime-rubrics

Conversation

@hyperpolymath

Copy link
Copy Markdown
Owner

Stacked PR (5 of 5 follow-ups). Base = `feature/composer-borderline-detection` (PR #55). Merge order: #51 -> #52 -> #53 -> #54 -> #55 -> this.

Summary

Tutors can add a new exam rubric by dropping an `.exs` file into `priv/rubrics/` and clicking "Reload rubrics" in the composer UI — no recompile, no restart. Built-in rubrics stay as compile-time fallbacks; disk-discovered packs can shadow them on `id` collision (so `priv/rubrics/ou_3_part.exs` overrides the built-in OU pack without a code change).

What ships

  • `RubricRegistry` refactor
    • `@builtin_packs` (`ou_3_part`, `single_essay`) remain compile-time fallbacks
    • `all/0` returns disk packs ++ built-ins, with disk shadowing on `id` collision
    • First read triggers a lazy disk scan, cached in `:persistent_term`
    • `reload/0` re-scans and returns `%{loaded: N, skipped: [...]}`
    • `rubric_dir/0` configurable via `Application.get_env(:etma_handler, :rubric_dir)`, falling back to `:code.priv_dir(:etma_handler) <> "/rubrics"` (`nil` when the app isn't loaded — keeps standalone module tests viable)
  • Robustness: files that fail to evaluate / return a non-RubricPack / have an empty `id` are skipped with a reason — a single broken file does not take the whole registry down.
  • LiveView: "Reload rubrics" button next to the rubric selector. Result surfaced via `put_flash` with loaded count and per-file skip reasons.
  • `priv/rubrics/`:
    • `README.adoc` documents the file shape and the shadowing rule
    • `example_extended.exs` is a working three-component sample (method / analysis / conclusion) demonstrating the loading pipeline

Authoring a new rubric

Drop a file like:

```elixir
alias EtmaHandler.Marking.RubricPack

%RubricPack{
id: "my_course_t1",
title: "My course, TMA 1",
components: [%{id: "essay", code: "a", title: "Essay", short: "essay", good: [{"...", "..."}], improve: [{"...", "..."}]}],
default_questions: [%{"id" => "q1", "label" => "Essay", "mark" => "", "max" => "100", "component_id" => "essay"}]
}
```

into `priv/rubrics/`, click Reload rubrics, pick the new rubric in the selector. Done.

Tests

  • 50 total, 5 new runtime cases (`async: false`, isolated via per-test tmp dir + `:persistent_term` erase on exit)
    • `reload/0` picks up a freshly written `.exs`
    • Disk pack with the same id shadows the built-in
    • Malformed files are skipped with a reason (syntax error, wrong shape, missing id)
    • Built-ins remain available when the rubric directory is empty
    • `reload/0` with `nil` rubric_dir is a no-op (not a crash)

Test plan

  • 50/50 module + registry + runtime tests pass via standalone elixirc + ExUnit
  • LiveView syntax-checks
  • In browser: `mix phx.server` -> /exam-feedback-composer -> "Example: extended essay…" shows in the selector
  • Copy `example_extended.exs` to e.g. `my_test.exs`, edit the `id` and a label, hit Reload -> new rubric appears
  • Write a syntactically broken `.exs` file in `priv/rubrics/` and Reload -> flash shows the skip reason; built-ins still work

🤖 Generated with Claude Code

Tutors can add a new exam rubric by dropping an .exs file into
priv/rubrics/ and clicking "Reload rubrics" in the composer UI — no
recompile, no restart.

RubricRegistry refactor
  - Built-in @ou_3_part and @single_essay remain compile-time fallbacks
  - all/0 returns disk-discovered packs ++ built-ins, with disk packs
    shadowing a built-in of the same id (drop in ou_3_part.exs to
    override the built-in OU pack without recompiling)
  - First read triggers a lazy disk scan, cached in :persistent_term
  - reload/0 re-scans and returns %{loaded: N, skipped: [...]}
  - rubric_dir/0 is configurable via Application env, falling back to
    :code.priv_dir(:etma_handler) <> "/rubrics" (nil when the app
    isn't loaded — relevant for standalone module tests)

Robustness
  - Files that fail to evaluate, return a non-RubricPack value, or have
    an empty id are skipped with a reason (logged out, not raised). A
    single broken file does not take the whole registry down.

LiveView
  - "Reload rubrics" button next to the rubric selector
  - Reload result surfaced via put_flash with the loaded count and any
    per-file skip reasons

priv/rubrics/
  - README.adoc documents the file shape and the shadowing rule
  - example_extended.exs is a working three-component sample (method /
    analysis / conclusion) that demonstrates the loading pipeline

Tests (50 total, 5 new runtime cases — async: false, isolated via per-
test tmp dir and persistent_term erase on exit)
  - reload picks up a freshly written .exs file
  - disk pack with the same id shadows the built-in
  - malformed files are skipped with a reason, not raised (covers syntax
    error, wrong shape, missing id)
  - built-ins remain available when the rubric directory is empty
  - reload with nil rubric_dir is a no-op (not a crash)

Verified locally via standalone elixirc + ExUnit (50/0).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant