# Plan: Assurance — per-assessment binary labels (Secured/Unsecured + AI-Forward/Resistant), hybrid detection

**Status:** spec for implementation
**Author (spec/review):** Opus
**Coder:** Sonnet general-purpose subagent
**Standard:** `standards/assessment-assurance-design.md`, CLAUDE.md §Operating Standard: Assurance by Design

---

## 1. Problem

The assurance security signal is effectively **proctoring-only** anywhere the interactive Step 2 AI pass has not run (the CLI scorer `score_course(cfg)` and the batch runner). Non-proctoring authorship evidence — drafts, decision logs, revision history, reflection, graded AI-use appendix, personal/local binding, milestone check-ins — is *defined* as secured (`assessment-assurance-design.md`) and described as secured in the report prose, but does not move the number. Because these signals are **text-based**, deterministic regex alone is unreliable (false positives on a title keyword; false negatives on real anchors phrased unusually). Proctoring/LockDown are last resorts, so the score must credit the preferred methods too.

## 2. Decision (confirmed with the user)

- **Drop the percentage/tier as the featured output.** No coverage %, no partial-weight math, no Assured/Partially/Unassured bands in what the reader sees.
- **Per-assessment, two plain labels:**
  - **Security:** `secured` | `unsecured`
  - **Design:** `ai_forward` | `ai_resistant` (or `n/a` where it does not apply, e.g. an auto-graded objective quiz)
- **Secured = identity control OR authorship evidence**, detected **hybrid**: deterministic nominator (regex/structural) → AI confirm with a cited quote.
- **Batch** ranks by **count of unsecured graded assessments** (descending), not by coverage.

## 3. Non-goals / guardrails (bound the blast radius)

- **Do NOT delete** `tier` / `assurance_coverage_pct` / the coverage arithmetic from the scorecard JSON or from `_compute_assurance_coverage`. Keep them as **internal, deprecated** fields for back-compat and as a batch tiebreak only. They must simply stop being the *featured* headline in the brief/report and stop being the primary batch sort key. (Rationale: many tests + downstream consumers read them; ripping them out is a separate, larger change. If the user later wants them fully removed, that is a follow-up PR.)
- **Do NOT** add the motivation third axis (explicitly deferred).
- **Do NOT** bump version, edit CHANGELOG, or tag. Review + release happen after.
- **Determinism preserved.** In a pure-deterministic run (no AI verdict available), a *nominated-but-unconfirmed* authorship item is labeled `unsecured` with `needs_ai_review: true` — never auto-secured on a keyword. Identity control (proctoring) is always `secured` deterministically.
- **Cite-or-fail.** An AI authorship verdict promotes to `secured` only when it carries a non-empty `evidence` citation. No citation → not secured.

## 4. Data model — new emitted field

Add to the `score_course` return a per-assessment ledger, the single source the renderers consume:

```python
"security_ledger": [
  {
    "id": int,
    "title": str,
    "grade_weight_pct": float,      # display only; NOT a score input
    "graded": bool,                  # counted, points > 0
    "security": "secured" | "unsecured",
    "security_basis": "identity" | "authorship" | "none",
    "security_evidence": str,        # citation when identity/authorship; "" otherwise
    "needs_ai_review": bool,         # nominee awaiting AI confirm (deterministic run)
    "design": "ai_forward" | "ai_resistant" | "n/a",
    "design_basis": str,             # short reason / cited signal; "" for n/a
  },
  ...
]
```

Rules for each field:

- **security / security_basis / security_evidence**
  - `proctoring_signal is True` → `secured`, basis `identity`, evidence `"proctoring_signal (API fact)"`. (House rule: a LockDown/proctored exam is a secured anchor; an AI "unsecured" judgment cannot downgrade it.)
  - else if an `ai_securing` entry for this id has `secured is True` AND non-empty `evidence` → `secured`, basis `authorship`, evidence = that citation.
  - else → `unsecured`, basis `none`, evidence `""`.
- **needs_ai_review** — `True` when the deterministic **nominator** flags the item as a plausible authorship anchor (see §5) but no confirming `ai_securing` entry is present. Signals "an AI read would resolve this." Never itself secures.
- **design / design_basis**
  - `n/a` for auto-graded objective instruments (quiz/exam with no free-response) — forward/resistant does not apply.
  - else derive from the existing per-item AI-forward anchor logic (`_aiforward_item_anchor_d4` / `_d5` and related): if the item requires *using and then critiquing/verifying* AI (AI-use appendix, "evaluate the AI output", "document what you accepted/rejected") → `ai_forward`; otherwise `ai_resistant`. Put the deciding signal in `design_basis`.
  - Where the deterministic read is low-confidence, default to `ai_resistant` and note it; the interactive Step 2 pass may refine (hybrid, same pattern as security).

## 5. Detection — hybrid nominator + AI confirm

### 5a. Deterministic nominator (always runs, cheap)
Extend the existing authorship signals. An item is **nominated** as a possible authorship anchor when ANY of:
- Process/oral keyword hit (existing `_PROCESS_TRAIL_KEYWORDS` / `_ORAL_KEYWORDS`) — **expand** the set to cover the named text signals: add `decision log`, `ai-use appendix`, `ai use appendix`, `ai appendix`, `milestone`, `check-in`, `version history`, `commit history`, `changelog`, `process memo`, `personal reflection`, `local`, `your own`, `your workplace`, `your community`.
- Structural facts already on the record: `peer_reviews is True`; submission types include `media_recording` (oral/authorship signal); rubric text (when available on the record) references drafts/process/revision/reflection.

Nomination alone does **not** secure. It sets the item up for confirmation and sets `needs_ai_review` when no AI verdict exists.

### 5b. AI confirm (interactive Step 2; already exists)
The existing Step 2 AI securing pass reads the assignment/rubric text and returns `{secured, kind, evidence}`. Extend its documented `kind` vocabulary in the SKILL to explicitly enumerate the authorship kinds the user listed (drafts, decision log, revision history, reflection, graded AI-use appendix, personal/local binding, milestone check-ins). This is a prompt/doc expansion, not new engine code. The verdict feeds `security_ledger` per §4.

**Why hybrid, not regex-only:** the nominator catches candidates cheaply and bounds which items get an AI read; the AI read is what actually confirms (rejecting keyword false-positives, catching unusually-phrased real anchors). In runs with no AI, the honest floor is `unsecured + needs_ai_review`, never a false secure.

## 6. Renderer changes

### 6a. Report (`assurance_report_html.py`)
- Replace the per-**type** security table (`security_types`, which carries `secured_share_pct`, a percentage) as the featured security view with a per-**assessment** ledger table sourced from `security_ledger`: columns **Assessment | Secured? | Basis | AI-Forward/Resistant**. Plain labels, no share %.
- Remove the coverage-% stat tile and the Assured/Partially/Unassured tier from the featured headline. (Keep the fields in the JSON per §3; just do not render them as the headline.)
- Keep the two-axis explainer prose (Security vs Design) — it is correct and stays.
- A `needs_ai_review` item renders `unsecured` with a small "needs review" marker, not a false secure.

### 6b. Brief (`assurance_brief_html.py`)
- The one-pager leads with the per-assessment labels: list the significant graded assessments, each with a **secured/unsecured** chip and an **AI-forward/resistant** chip.
- If the 2×2 conceptual diagram is retained, drive dot position from **counts** (how many secured vs unsecured; how many forward vs resistant), not from `coverage_pct`. Simplest acceptable version: drop the continuous axes and show the two label columns. Coder's choice, whichever is cleaner — no percentage may appear.

### 6c. Batch (`assurance_batch.py`)
- Replace the primary sort key (`tier` rank, then `coverage` desc) with **count of unsecured graded assessments, descending** (more unsecured = needs attention first). Keep `coverage` only as a final tiebreak if desired. Surface per course: `# secured`, `# unsecured` (graded), not a coverage %.
- `_ATTENTION_TIERS` / `certifying_on_nothing` logic that depends on tier may stay for now (internal), but the cohort table's featured column becomes the secured/unsecured counts.

## 7. Acceptance criteria (all must hold)

1. `score_course` emits `security_ledger` with one entry per assessment, fields per §4.
2. Proctored/LockDown item → `secured`, basis `identity`, deterministically, with no AI input.
3. Item with an `ai_securing` entry `{secured:True, evidence:"...quote..."}` → `secured`, basis `authorship`.
4. Item with an `ai_securing` entry `{secured:True, evidence:""}` (no citation) → **not** secured (cite-or-fail).
5. Nominated authorship item (keyword/structural) with **no** `ai_securing` entry → `unsecured`, `needs_ai_review:True`.
6. Plain take-home item with no signal → `unsecured`, basis `none`, `needs_ai_review:False`.
7. Auto-graded objective quiz → `design: "n/a"`.
8. A free-response item with an AI-use-appendix / critique-the-AI signal → `design: "ai_forward"`; an authorship/local-binding item → `ai_resistant`.
9. Report renders the per-assessment ledger table and shows **no percentage** and **no Assured/Partially/Unassured** in the featured security view.
10. Brief renders per-assessment secured/unsecured + forward/resistant labels and **no percentage**.
11. Batch orders courses by unsecured graded count desc; cohort table shows secured/unsecured counts, no coverage %.
12. `tier` / `assurance_coverage_pct` still present in the scorecard JSON (deprecated, unremoved) — existing JSON-shape tests still pass.
13. Full suite green: `.venv/bin/python3 -m pytest tests/ -q`.

## 8. Test plan (add to `tests/`)

New/extended unit tests (pure Python, no network) covering criteria 2–8 by constructing minimal `records` + `ai_securing` dicts and asserting on `security_ledger` entries. Add a batch-sort test (criteria 11) with three synthetic scorecards. Add a renderer smoke test asserting the report/brief HTML contains the assessment labels and does **not** contain a `%`-coverage string or the tier words in the featured block (criteria 9–10). Keep to the existing test style in `tests/` — no new frameworks.

## 9. Out of scope / follow-ups

- Full removal of `tier`/`coverage` (separate PR if wanted).
- Motivation third axis (deferred).
- Running the Step 2 AI securing pass inside the batch runner via subagents (cross-course comparability change) — deferred; batch stays deterministic (identity-secured + nominator flags), which is honest with `needs_ai_review`.

## 10. Workflow

Feature branch → implement → tests → push → open PR against `main` with `gh pr create`. Do **not** merge (coder ≠ reviewer; Opus reviews the diff and re-runs the suite). Report branch name, PR number, and the verbatim `pytest` summary line.
