# Assurance audit: 80% AI / 20% regex (AI-led securing + conclusions)

User directive: stop anchoring securing on what regex can find (a LockDown flag). A full
assurance audit means AI reads the assessments and JUDGES; regex only supplies deterministic
facts. Split the engine 80/20: **AI does analysis and conclusions** (is an item secured, is it
substitutable, the tier, recommendations); **regex/field-checks do deterministic facts only**
(points, weights, drop rules, published/counted, presence of a LockDown flag or policy text).

This subsumes the process-evidence-as-securing fix Michael flagged: once securing is AI-judged
from content, process evidence (drafts, decision logs, revision history, reflection) is
securable like any other authorship control, without a bespoke detector.

## The invariant being flipped

`scripts/assurance_audit.py::_classify_lane` today: `lane1` (secured) iff
`proctoring_signal is True OR peer_reviews is True`. Oral/process-trail keyword hits are
nominated as `lane1_candidate` but **never promoted** to `lane1` ("they never auto-promote").
That refusal is the regex-first bias, and `peer_reviews` as securing contradicts Michael (peer
review does not verify authorship).

## Target model: two kinds of securing, AI-classified, deterministic arithmetic

Secured = there is a real basis to trust this student did the work. Two kinds:
- **Identity/condition control** (proctoring, LockDown) -- a deterministic API fact; regex confirms presence.
- **Authorship evidence** (milestoned drafts, decision/process logs, revision history, reflection, an AI-use appendix graded for judgment, a defensible oral/process account) -- NOT an API flag; AI reads the assignment/rubric and judges it, citing the evidence.

Proctoring is one subtype of securing, named the **last-resort** one (weak vs. current AI,
unscalable, an access barrier). It is not the definition of secured.

The arithmetic stays deterministic (the 20%): coverage % = secured grade-weight / total, tier
from `ASSURED_COVERAGE_THRESHOLD` + anchor count, weights/drops from the inventory. Only the
**classification feeding it** becomes AI (the 80%). Cross-course comparability now rests on
consistent AI judgment, so every secured/unsecured call must cite its evidence.

## Workstreams

### 1. `_classify_lane` + `score_course` accept AI securing  [`scripts/assurance_audit.py`]
- Add param `ai_securing: dict[int, dict] | None = None` to `score_course`, shape
  `{assessment_id: {"secured": bool, "kind": "identity"|"process"|"unsecured", "evidence": str}}`.
- `_classify_lane(record, ai_securing_entry=None)`:
  - `lane1` iff `proctoring_signal is True` (deterministic identity fact) OR
    `ai_securing_entry and ai_securing_entry["secured"] is True` (AI-judged, identity or process).
  - **Remove `peer_reviews` as an auto-lane1.** Keep `peer_reviews` as a recorded fact the AI may
    weigh, never an automatic secure. Update the Criterion A invariant docstring.
  - Keyword nominees: when `ai_securing` is absent, still emit the `lane_review` advisory (the
    deterministic-only fallback path is unchanged except peer removal). When `ai_securing` is
    present, the AI entry is authoritative; no advisory needed for a classified item.
- Deterministic-only fallback (no `ai_securing`): `lane1` iff `proctoring_signal is True` only.
  This is the conservative floor when no AI pass ran; document it.
- Coverage/tier math unchanged; it consumes the (now AI-augmented) lane classification.

### 2. Two-pass workflow  [`skills/assurance-audit/SKILL.md`]
Reframe Step 2-4 as AI-led over a deterministic substrate:
1. Build inventory (facts: points, weights, drop rules, published/counted, proctoring-flag
   presence, policy-text presence). The 20%.
2. **AI securing pass** (the 80%): read each significant assessment's prompt + rubric + the
   syllabus; classify each `secured`/`kind`/`evidence`. Identity control (proctoring/LockDown)
   OR authorship evidence (process trail, drafts, decision logs, reflection, graded AI-use
   appendix) both secure; a bare take-home upload does not. Cite the evidence phrase.
3. `score_course(inventory, ai_securing=...)` computes tier/coverage from the AI classification.
4. AI authors the narrative (substitutability, Bloom deep-pass, recommendations) as today.
- State the tradeoff explicitly in the SKILL: tier/coverage are now AI-classification-driven;
  the arithmetic is deterministic but the securing inputs are AI judgments, so they are
  within-cohort comparable only when the evidence is cited and the prompt version is fixed.
- Remove the old "AI judgments never alter the deterministic tier" rule; replace with "AI
  classifies securing (with cited evidence); tier is the deterministic arithmetic over that
  classification."

### 3. Definitions + standard
- `scripts/assurance_report_html.py` `_HOW_TO_READ`: "Secured" = a real basis to trust the
  student did the work, via an identity control (proctoring, LockDown) OR authorship evidence
  (drafts, decision logs, revision history, reflection); proctoring is the last-resort subtype.
  **Delete the open-question note about process evidence** -- it is resolved: process evidence
  secures.
- `standards/assessment-assurance-design.md`: encode the two-kinds-of-securing model and that
  securing is AI-judged from content, regex confirms only identity-control presence. Keep the
  motivation-as-third-axis item as the one remaining open question (unchanged).
- `standards/assurance-sources.md`: note the AI-led classification and that the 40%/0.75/bands
  remain house-calibrated arithmetic parameters.

### 4. Recommendation tags reflect securing
Process-evidence recommendation moves (milestone the paper, AI-use appendix, decision log) are
**securing** moves, not only design moves. In the report recommendations, such a move on an
unsecured item closes a security gap; tag/word it so.

## Out of scope (unchanged)
- Motivation as a third axis stays the single open model question.
- No new bespoke "process-evidence detector" regex -- AI does the reading (that is the point).

## Acceptance
- `score_course` deterministic-only path (no `ai_securing`): peer no longer auto-secures;
  proctoring still does; existing coverage math holds otherwise.
- `score_course` with `ai_securing`: a process-evidence item with `secured:true` counts toward
  coverage and can move the tier; an unsecured proctored-absent item does not.
- Tests updated for peer removal + new AI-securing path + evidence required. Full `pytest -q` green.
- Report definition updated, open-question note gone, COM 100 regenerated (the paper's
  milestone/appendix moves now read as securing).
- Opus reviews the diff + reruns suite; PR; release.
