# Assurance accuracy v2 — spec (Michael's COM 100 review)

Source: Michael Miller annotated the COM 100 assurance report. Read-only re-audit of the
real accelerator dev shell **221272** (`DEV-2026Summer-AssessmentAccelerator-COM100`)
confirmed the report was built from the wrong shell (151110, now 403) and miscounted.
Verified facts: Papers = choose 1 of 10 (syllabus "of your choice" + a Sign Up assignment);
Quizzes carry `rules.drop_lowest: 2` and syllabus "we only count 10 of the 12 quizzes";
assignment groups are weight=0 (points-based).

Coder: Sonnet. Reviewer/tester: Opus (parent). PR-only, coder != reviewer, full pytest,
release. Every dynamic value HTML-escaped; house style APA caps + no em/en dashes.

## Workstreams

### 1. Inventory: honor drop rules  [`scripts/assurance_inventory.py`]
Assignment groups expose `rules` = `{"drop_lowest": k, "drop_highest": k, "never_drop": [ids]}`.
Today the weight map (`_weighted_group_weight_map` / `_points_only_weight_map`) sums ALL
counted items as the group denominator, ignoring drops. Fix, in BOTH weighted and points-only paths:

- For each group with `drop_lowest`/`drop_highest`, remove the k lowest (resp. highest)
  **point_possible** counted items from the group's counted-points denominator, honoring
  `never_drop` (those ids are never removed).
- Deterministic tiebreak: among equal points, drop the lowest assignment `id` first, so the
  result is stable without student scores. Document that WHICH items a student actually drops
  depends on their scores; at audit time we model the group's max achievable points (top n-k).
- The k dropped item slots get `grade_weight_pct = 0` and a new item field
  `dropped_by_rule: true`.
- New per-group meta in output: `counted_after_drops` (int), `dropped_count` (int).
- COM 100 check: 12 quizzes @10, drop_lowest 2 -> group max 100 pts, 10 counted, each retained
  quiz weight computed against 100 not 120.

### 2. Inventory: flag likely choice-groups (flag only, never silently recompute)  [same file]
1-of-N choice is NOT in the API. Do NOT guess the count. Add a detector that sets a group-level
flag when a group looks like "choose one of N":
- Signal: >= 3 counted siblings with identical `points_possible`, AND any sibling name or
  description (or a sibling in the same group) matches
  `/sign[\s-]?up|of your choice|choose (one|1)|your choice|pick (one|1)/i`.
- Output: group meta `possible_choice_group: true` + `choice_signal: "<matched phrase>"`.
- Counts/weights are UNCHANGED (still counts all N). The flag surfaces as an assumption/caveat
  so an author or ID verifies. COM 100 Papers (10 @50, a "Sign Up" sibling) must trigger it.

### 3. Course level detection + Bloom calibration  [NEW `scripts/course_level.py`]
- `detect_level(course_code: str) -> dict`: parse first `\d+` in the code, floor to hundred,
  clamp to 100..700 (>=700 -> 700; no digits -> level None). Return
  `{"level": int|None, "band": "100"|..|"700"|None, "label": "100-level"|"graduate"|None}`.
  500/600/700 label "graduate".
- `bloom_expectation(level: int|None) -> dict`: the Bloom band a course at this level is
  *expected* to emphasize, so low Bloom is not flagged as a defect below expectation.
  100-200: `{"floor": "Remember", "expected_ceiling": "Apply", "note": "recall and understand
  are appropriate at intro level"}`. 300-400: floor Apply, ceiling Analyze. 500+: floor Analyze,
  ceiling Create. None -> empty/no calibration.
- Pure functions, no I/O. Unit tests: "COM 100"->100, "PHI 101"->100, "SOC 334"->300,
  "LAW 524"->500(graduate), "ARS101"->100 (no space), "Foundations" (no digit)->None,
  "OGL 720"->700.

### 4. Report renderer  [`scripts/assurance_report_html.py`]
a. **Verdict matrix colors** (`_verdict_svg`). Remove the axis-color collision: today GOAL uses
   green `#2f7d32` and the course dot uses maroon `#8c1d40`, which clash with green=Security /
   maroon=Design used everywhere else. New: GOAL marker + "Real Thinking + Proven" label -> gold
   `#ffc627` (dot outlined so it reads on white: gold fill, `#b98900` stroke). Course dot ->
   neutral ink `#12100e`. "Proven but Shallow" / "Real Thinking Not Proven" / "Neither" quadrant
   labels stay muted `#8f877a` (drop the maroon on "Proven but Shallow"). Apply the SAME change to
   the brief's `_verdict_svg` in `scripts/assurance_brief_html.py` so both matrices match.
b. **§01 "secured" definition** (`_HOW_TO_READ`). Remove "peer review" (it does not verify
   authorship and the detector does not detect it). Reword: secured = a real control that confirms
   who did the work, a proctored or LockDown-locked exam; name proctoring a last resort. Keep it
   identity-verification only. Do NOT add process-evidence (decision logs / revision history /
   reflection) as *securing* here -- that is the unresolved model question; it stays a design move.
c. **Course-level badge + no low-Bloom-defect at intro.** `render()` reads
   `scorecard["course"].get("level_label")` (authoring sets it via course_level). When present,
   show it in the masthead sub line (e.g. "Assessment Assurance Report | 100-level"). No built-in
   text may frame low Bloom as a defect; that framing is authored and must respect level (see 5).

### 5. SKILL.md authoring guidance  [`skills/assurance-audit/SKILL.md`]
- **Counts**: before stating any assessment count, check `rules` (drop_lowest/highest) and the
  `possible_choice_group` flag and the assignment text; report the group's post-drop counted
  points, and never state N items as N deliverables when the group is a choose-1-of-N menu.
- **Course level**: set `course.level`/`level_label` from `course_level.detect_level`; calibrate
  the Bloom read to `bloom_expectation`. At <=200 level, recall/understand is expected -- do not
  frame it as a design gap; say so.
- **Recommendations** (Michael): (i) ground every move in the course's actual items; (ii) bind
  TILT annotated examples in the SAME sentence as any verify/critique/improve-the-AI move;
  (iii) drop AI-resistant moves defeated by current multimodal AI (cohort/classmate-reference,
  screenshot-readable prompts) -- do not present them as AI-resistant; (iv) do not recommend
  removing recall wholesale, especially at intro level; keep formative recall.
- Update the §secured wording to match 4b.

## Out of scope (deferred, per user)
- Playbook/recommendation separation (revisit only if relevance work fails).
- Motivation as a third axis; process-evidence-as-securing model change. Leave a one-line note
  in the report/SKILL that these are open, do not implement.

## Acceptance
- New/updated pytest per workstream; full `pytest tests/ -q` green.
- Opus reviews the diff (not the summary), regenerates COM 100 from 221272 corrected data, PRs,
  releases (both manifests + VERSION + CHANGELOG + tag).
