---
name: assurance-audit
description: "Audit whether a Canvas course's assessment strategy can certify learning in an AI-capable world. Trigger phrases: 'is my course AI-proof', 'can students bypass with AI', 'assessment assurance', 'assessment integrity', 'AIAS', 'authentic assessment', 'does this course certify learning', 'AI-resistant assessments', 'substitutability check', 'AI-forward assessment', 'is this assessment AI-forward'."
---

# Assessment Assurance Audit Skill

> **Scope note**: This skill audits the course's *assessment strategy* -- lane structure, assurance coverage, authenticity, cognitive demand, and AI-use policy integration. It is SEPARATE from `course-audit`, which audits QA standards, accessibility, and structural completeness. Do not conflate them; do not route here for QA/a11y/standards work.

> **Plugin**: ASU Canvas Course Builder
> **Scripts**: `scripts/assurance_inventory.py` (build_inventory), `scripts/assurance_audit.py` (score_course)
> **References**: `skills/assurance-audit/references/assurance-model.md`, `skills/assurance-audit/references/authenticity-framework.md`, `skills/assurance-audit/references/learning-science-lens.md`, `skills/assurance-audit/references/ai-forward-framework.md`
> **Related**: `alignment` (CLO/MLO alignment, not covered here), `course-audit` (QA/accessibility, separate from this skill)

> **Adapted from**: the external "AI Assessment Evaluator" skill (Two-Lane assurance, AIAS, TVO, Authenticity, Counterfeit Assurance). No upstream text copied verbatim.


## Purpose

Evaluate whether a Canvas course's graded assessment architecture can certify that a human learner achieved the course's intended performance outcomes -- given the capability of current AI tools. The audit surfaces:

- Which assessments are structurally secure (Lane 1) vs. substitutable (Lane 2)
- What fraction of the final grade is "assured" vs. at risk
- Whether an AI-use policy is declared and at what integration level
- Where authenticity is weak and where cognitive demand is counterfeit
- What durable skills the assessment design is actually measuring

Recommendations use four verbs only: **secure**, **re-lane**, **declare**, **liberate**. No surveillance framing.


## When to Use

| User says... | Action |
|---|---|
| "Is my course AI-proof?" | Run this skill |
| "Can students bypass this course with AI?" | Run this skill |
| "Assessment assurance audit" | Run this skill |
| "AIAS level check" | Run this skill |
| "Authentic assessment audit" | Run this skill |
| "Does this course certify learning?" | Run this skill |
| "Substitutability check" | Run this skill |
| "AI-resistant assessments" | Run this skill |
| "Is this course accessible?" / "Check a11y" | Route to `course-audit` (SEPARATE skill) |
| "Is my course aligned?" / "CLO coverage" | Route to `alignment` (SEPARATE skill) |


## Setup Gate

Run the standard CLAUDE.md Setup Gate silently:

1. Check `.env` for `CANVAS_TOKEN` + `CANVAS_DOMAIN`.
2. Confirm course context (URL in message extracts directly; else registry dashboard).
3. Read or create `courses/{courseId}/course-memory.md`.
4. Load or create `courses/{courseId}/course-config.json`; detect `course.tier` from course code if not set.

The assurance-audit skill is **read-only** -- no Canvas writes, no staging gate required.


## Workflow

### Step 1: Inventory

Run `build_inventory(config)` from `scripts/assurance_inventory.py`.

This returns:
- `assessments`: list of normalized records with `id`, `title`, `canvas_type`, `engine`, `points`, `grade_weight_pct`, `submission_types`, `has_rubric`, `peer_reviews`, `proctoring_signal`, `dropped_by_rule`, `module`, `group`, plus `bloom_level` / `bloom_index` / `bloom_source`
- `course_weighting_mode`: `"weighted_groups"` or `"points_only"`
- `groups`: per-assignment-group meta -- `rules` (`drop_lowest`/`drop_highest`/`never_drop`), `counted_count`, `counted_after_drops`, `dropped_count`, `counted_points_before_drops`, `counted_points_after_drops`, `possible_choice_group` (flag only, never recomputed), `choice_signal` (the matched phrase when flagged)
- `totals`: `grade_weight_pct_sum` (should be ~100%)
- `course_clos`: CLOs read from the syllabus (endpoint), Bloom-classified
- `module_mlos`: per-module MLOs read from module overview pages, Bloom-classified
- `assumptions`: list of strings recording detection decisions (includes a `bloom_grounding=title:N,mlo:N,unknown:N` line)

**Objective grounding (CLO/MLO Bloom).** The inventory grounds each assessment's Bloom level in the course objectives, not just the assessment title. It uses the title's Bloom verb when present and falls back to the module's MLOs (from the overview page) when the title carries no verb (a "Module 3 Homework" inherits Module 3's MLO level). CLOs come from the syllabus. This is fill-Unknown: a clear title verb always wins; when neither resolves, the assessment stays Bloom Unknown. Without it, generically-titled courses score as 100% Unknown Bloom and the cognitive-demand + AI-Forward analysis goes blind. An objective quiz whose HIGH Bloom claim comes only from its module MLOs (not its own title) is sent to `needs_ai_review` as `counterfeit_objective_mlo` rather than deterministically branded counterfeit (a low-stakes concept check in a high-level module is legitimate).

**Drop rules and choice groups.** Before stating any assessment count or weight, check the assessment group's `rules` and `possible_choice_group` flag in the inventory output. When a group carries `drop_lowest` / `drop_highest`, report the group's post-drop counted points (`counted_points_after_drops`) and count (`counted_after_drops`), not the raw total -- the dropped item slots carry `grade_weight_pct=0` and `dropped_by_rule: true` on the assessment record (e.g., 12 quizzes at 10 points each with `drop_lowest: 2` reports as 10 counted quizzes worth 100 points, not 12 worth 120). State plainly that WHICH items a given student drops depends on their scores; the audit models the group's maximum achievable points (drops the lowest/highest `points_possible` items), not a per-student calculation. When `possible_choice_group` is true (e.g., 10 identically-weighted papers with a "Sign Up" sibling), do not state "10 deliverables" -- the syllabus likely describes a choose-1-of-N menu. Verify against the syllabus text and the `choice_signal` match before stating the real per-student deliverable count, which is lower than the item count the API returns. Never silently recompute counts or weights from this flag; it is a caveat to state and verify, not an automatic correction.

If `build_inventory` returns zero assessments, report that no graded assessments were found and stop. Do not fabricate a scorecard.

### Step 2: AI Securing Pass

Before scoring, classify each significant assessment's securing status. This is the 80% of the audit: AI reads and judges; regex only supplies deterministic facts (points, weights, drop rules, published/counted, proctoring-flag presence).

Secured = there is a real basis to trust the student did the work. Two kinds:
- **Identity/condition control** -- proctoring, LockDown Browser. Already a deterministic API fact (`proctoring_signal`); no AI read needed for this kind.
- **Authorship evidence** -- milestoned drafts, decision/process logs, revision history, reflection, an AI-use appendix graded for judgment, a defensible oral/process account. NOT an API flag. Read the assignment's prompt + rubric + the syllabus and judge whether the evidence is really there, citing the phrase that shows it.

For each weighted assessment (use the top-K set from `_select_top_k`, or every assessment on a small course), classify it into `{"secured": bool, "kind": "identity"|"process"|"unsecured", "evidence": str}`:
- `proctoring_signal is True` -> `{"secured": True, "kind": "identity", "evidence": "proctoring_signal (API fact)"}` (this one is deterministic; no read needed).
- Prompt/rubric/syllabus shows a real process trail -> `{"secured": True, "kind": "process", "evidence": "<cited phrase>"}`. The authorship-evidence kind covers, by name, any of: milestoned **drafts**, a **decision log**, **revision history** (or a version/commit history), **reflection** (including personal reflection), a **graded AI-use appendix**, **personal or local binding** (a task tied to the student's own workplace, community, or local context that AI cannot generically produce), and **milestone check-ins** (staged progress points across the term, not a single final submission). This list is the set of things to look for in the prompt/rubric/syllabus text, not a fixed enum; cite the actual phrase that shows the evidence.
- A bare take-home upload with no process trail and no identity control -> `{"secured": False, "kind": "unsecured", "evidence": "<why -- what is missing>"}`.

`peer_reviews` is a recorded fact on the assessment record; weigh it as circumstantial support for a process-evidence call if relevant, but it never secures on its own -- peer review does not verify authorship.

Build the `ai_securing` dict keyed by assessment id and carry it into Step 3. `score_course` also runs a deterministic nominator over every assessment whether or not `ai_securing` is supplied (see `security_ledger` in Step 3): it flags the same authorship-evidence signals as candidates for review, but nomination alone never secures anything -- only a confirmed, cited `ai_securing` entry (this step) or `proctoring_signal` promotes an item to secured.

**Tradeoff, stated explicitly.** Tier and coverage are now AI-classification-driven: the arithmetic (coverage % = secured grade-weight / total; tier from `ASSURED_COVERAGE_THRESHOLD` + anchor count) stays deterministic, but the securing inputs feeding it are AI judgments. They are within-cohort comparable only when the evidence is cited and the prompt version is fixed -- do not treat one course's tier as directly comparable to another's unless both ran the same prompt version.

### Step 3: Score

Run `score_course(config, inventory=inventory, ai_securing=ai_securing)` from `scripts/assurance_audit.py`, passing the dict built in Step 2. When no AI securing pass has run (e.g. a quick deterministic-only check), omit `ai_securing` -- `score_course` falls back to the conservative floor (`proctoring_signal` only; no keyword or peer-review promotion).

The scorecard contains:
- `security_ledger`: the FEATURED output (plan: assurance-binary-labels) -- one entry per assessment, `{id, title, grade_weight_pct, graded, security: "secured"|"unsecured", security_basis: "identity"|"authorship"|"none", security_evidence, needs_ai_review, design: "ai_forward"|"ai_resistant"|"n/a", design_basis}`. `security`/`security_basis` come from this step's `ai_securing` classification (or `proctoring_signal` for identity); `needs_ai_review` is true only for a deterministically-nominated item with no `ai_securing` entry yet. The report and brief lead with this ledger, not `tier`/`assurance_coverage_pct` below.
- `tier`: `"Assured"` | `"Partially Assured"` | `"Unassured"` (coverage-derived; DEPRECATED, NOT FEATURED -- kept for back-compat and as an internal batch tiebreak only). `"Counterfeit"` is a deprecated-not-assigned enum value kept for historical scorecard JSON compatibility (#219: it used to be forced by a mismatch item and dragged down courses with strong coverage; a mismatch is now the `counterfeit_claim` flag below instead).
- `ai_forward_rating`: `"Needs redesign"` | `"Use with safeguards"` | `"AI-resistant"` | `"AI-forward"` (deterministic; the AI-Forward axis, orthogonal to `tier`)
- `ai_forward_dimensions`: D1-D6 each `{anchor: 0-3, label, basis}` (see `ai-forward-framework.md`)
- `bloom_calibration_findings`: deterministic per high-weight item (Bloom Guide "Avoid" rules), each citing assessment + points + grade_weight_pct
- `dimensions.F`: AI-Forward axis detail (rating, rating_index, auto_flagged, the six dimensions)
- `assurance_coverage_pct`: float (% of grade weight on secured Lane 1 anchors). DEPRECATED, NOT FEATURED -- see `tier` note above.
- `ai_policy_in_syllabus`: bool -- was an AI-use policy declared in the syllabus? (#219: one of the two signals that replace `aias_transparency_pct`)
- `ai_declared_assessments`: `{count, total}` -- how many assignment/quiz descriptions carried a task-level AI declaration, of how many scanned (#219: the other replacement signal; a syllabus-wide policy statement and N per-assignment declarations are very different signals and no longer blended into one ratio)
- `counterfeit_claim`: bool (#219) -- true when a high-weight assessment claims security it does not have (the mismatch condition). A FLAG layered on the coverage-derived `tier`, never a tier override. Mirrored in `flags.counterfeit_assurance`.
- `has_graded_assessments`: bool (#219) -- False for a truly empty course (zero graded assessments). Such courses get a distinct "no graded assessments" state and are excluded from the batch runner's `certifying_on_nothing` count.
- `lane_balance`: `{lane1, lane2, mismatch, unclassified}` counts
- `dimensions.A`: Authenticity Index (Gulikers 5DF, AI-judged top-K)
- `dimensions.B`: Learning Science pointer (not scored -- defer to `alignment` + HPI/ID standard)
- `dimensions.C`: Cognitive Demand (deterministic Bloom distribution + arc + counterfeit check)
- `dimensions.D`: Assurance and AI-Integration (headline, deterministic + bounded AI annotation)
- `dimensions.E`: Equity and Feasibility (deterministic; surveillance/synchronous/concentration burdens + informational hand-graded count). Per-course only: never changes `tier`, never feeds cohort ranking, not cross-course comparable.
- `flags`: `{counterfeit_assurance, transparency_gap, no_anchor, over_surveillance}` (#219: `counterfeit_assurance` now tracks the mismatch condition directly -- same value as `counterfeit_claim` -- not `tier == "Counterfeit"`)
- `durable_skills`: list of inferred skill signals
- `needs_ai_review`: list of judgment items for Claude to evaluate inline
- `assumptions`: K, model_capability, modality, prompt_version, inventory_assumptions

### Course-Level Calibration

Before authoring the narrative, call `detect_level(course_code)` from `scripts/course_level.py`. Pass `course.code` from `course-config.json` (e.g. "PMG 525"); if it is empty, parse the code segment from the course name before the first colon instead (the same convention `_course_code()` in `assurance_brief_html.py` uses for the brief's matrix label). Merge the result into the scorecard before rendering: `scorecard["course"]["level"] = result["level"]`, `scorecard["course"]["level_label"] = result["label"]`. When `level_label` is present, `assurance_report_html.render()` shows it in the masthead sub line (e.g. "Assessment Assurance Report | 100-level").

Call `bloom_expectation(level)` and use its `floor` / `expected_ceiling` / `note` to calibrate the Bloom read in Dimension C and the Course Snapshot's Design measures. A course at or below the 200 level is EXPECTED to sit in Remember/Understand through Apply -- do not frame that distribution as a design gap or a defect; say so explicitly in the narrative (e.g., "Recall and understand are appropriate at the introductory level; this is not a design gap"). The expectation calibrates LANGUAGE only, never the deterministic Bloom distribution, tier, or coverage math -- those stay exactly as computed. A course with no parseable level (`level=None`) carries no calibration; describe the Bloom distribution plainly, with no level-relative claim either way.

### Step 4: Inline AI Judgment

For each item in `needs_ai_review`, Claude judges it INLINE using the rubrics in `references/`:

- `judgment_type: authenticity_5df` -- score each of the 5 Gulikers dimensions (1-5 per facet); see `references/authenticity-framework.md` for the full rubric
- `judgment_type: substitutability` -- apply the DS+CS+PI+OFA Substitutability test from `references/assurance-model.md`; return a 0-1 score (>=0.75 = substitutable)
- `judgment_type: fire_quality` -- score Focused/Integrative/Reflective/Expansive for Lane 2 assignments; see `references/assurance-model.md`
- `judgment_type: aias_level` -- classify AI-use policy integration as AIAS 1-5 using the level descriptors in `references/assurance-model.md`
- `judgment_type: evaluative_judgment` -- assess whether the task requires evaluative judgment (signal for AIAS 4-5 appropriateness)
- `judgment_type: counterfeit_freeresponse` -- for free-response items claiming Analyze+, judge whether the task actually requires that level or is counterfeit by design
- `judgment_type: judgment_of_ai` -- AI-Forward D4: does the task grade the *quality of the student's AI-use judgment* (critique, verify, defend, audit AI output), not just the output? Refines D4 narrative only; never alters `ai_forward_rating`
- `judgment_type: authorship_evidence_depth` -- AI-Forward D5: does the process evidence document prompts, revisions, verification moves, and reflection on AI limits/bias? Refines D5 narrative only; never alters `ai_forward_rating`
- `judgment_type: bloom_content` -- **deep-pass cognitive demand**: read the attached `content_excerpt` (prompt) and `rubric_excerpt` (rubric criteria) and classify the Bloom level the task **REQUIRES**, not the level inherited from the module objectives. One item is emitted per weighted assessment; structurally-identical items are deduped to one representative (the 10 short papers -> one). Items tagged `basis="format"` carry `bloom_content_available=False` (objective items whose bank-locked stems the API does not expose): classify those by instrument **format**, not content. Fold the per-cluster results into the narrative's `cognitive_demand` object for the renderer (see Step 4). The deep pass refines Dimension C only -- higher Bloom does not improve assurance, and the deep-read distribution never feeds the tier, coverage, or lane math.

Fold judgment results back into the report narrative for Dimension A and Dimension D. AI judgments are labeled "within-course diagnostic, not cross-course comparable" whenever referenced. AI classifies securing (with cited evidence, in Step 2); `tier` and `assurance_coverage_pct` are the deterministic arithmetic computed over that classification -- the arithmetic itself never changes, but its inputs are the Step 2 AI securing calls. The Step 4 judgment items (authenticity, substitutability, FIRE, AIAS level, bloom_content) are a separate layer and do not feed the tier or coverage at all.

Context for judgment: use `discipline` and `level` from `course-config.json`; default `modality` to async; read AI policy from syllabus text in the scorecard. Apply calibration from `references/authenticity-framework.md` Section "Discipline Calibration" when available.

### Step 5: Report

**Rendered report structure (two-axis, six-section).** The HTML report produced by `scripts/assurance_report_html.py` renders in the brief's editorial design system and runs two independent axes in parallel throughout: **SECURITY** (green: "can it prove a real student did the work") and **DESIGN** (maroon: "does the work measure real thinking and address AI use"). The two never collapse into one score, so a reader cannot mistake "secured" for "done". Section order and labels:

1. **How to Read This Report** (static): the two-questions frame plus the Secured/Unsecured and AI-Resistant/AI-Forward definitions. Not data-driven.
2. **The Bottom Line**: the Verdict matrix (X = proves a real student did it, Y = measures real thinking; the GOAL marker is gold, the course-position dot is neutral ink, placed from `security_ledger` counts -- secured/graded for X, ai_forward/(ai_forward+ai_resistant) for Y -- falling back to `assurance_coverage_pct` x `ai_forward_rating` only when no ledger is present), the two parallel reads (Security green, Design maroon), the "The Tension:" focus line, and the stat band (a "N/M Secured" count, Design rating, graded items, assessment types -- no coverage percentage). Always renders from the scorecard; the stat band falls back to scorecard-derived cells when the narrative omits one. When the scorecard carries `course.level_label`, the masthead sub line shows it (e.g. "Assessment Assurance Report | 100-level"; see Course-Level Calibration above).
3. **Course Snapshot**: a green **Security** subhead (per-assessment table sourced directly from `security_ledger`, not the narrative: Assessment | Secured? | Basis | AI-Forward/Resistant, plain labels, no share percentage; a nominated-but-unconfirmed item shows a small "needs review" marker) and a maroon **Design** subhead (the narrative-authored "What the Work Measures" Bloom bar plus "AI Use in Instructions").
4. **Where It Needs Work**: two side-by-side panels, a green **Security Gaps** column (AI-substitutable, unsecured weight) beside a maroon **Design Gaps** column (shallow or low AI-forward, even if secured). For most courses the two lists barely overlap.
5. **Recommendations by Assessment Type**: one block per type the course has, AI-forward moves visually emphasized (green, labeled "recommended"), AI-resistant next, and proctoring/oral/synchronous LAST as a last-resort line.
6. **Method and Caveats** (static, overridable): what was scanned, what was excluded (with the one optional-item edge case), and the within-course-only interpretation note.

**Resolved:** process evidence (decision logs, revision history, reflection) counts as a securing control -- it is authorship evidence, AI-classified per Step 2. **Deferred, not implemented:** whether motivation belongs as a third report axis remains the one open model question; do not implement it without an explicit spec.

The report is deterministic over an authored `narrative["report"]` object (see Step 5). Sections 01 and 06 are static boilerplate; 02 always renders; 03/04/05 render only when their data is present. The rendered report carries no framework jargon: "Dimension", "FIRE", "5DF", `needs_ai_review`, `lane_summary`, `ai_forward_rating`, and "counterfeit" (as a label) never appear in user-facing text.

**No-Unknown cognitive demand.** The inventory resolves two cases that previously read Unknown. First, graded discussions resolve their module via the discussion topic's `content_id` (which equals the module item's `content_id`), so they inherit their module's MLO Bloom level. Second, an auto-graded objective quiz/exam whose own module carries no MLOs (exam-holder or start-here modules) classifies as **Understand** deterministically (`bloom_source="objective_floor"`, since an auto-graded objective instrument cannot exceed Understand). Together these eliminate Unknown cognitive-demand weight for courses like 151110 (8 graded discussions plus Exam 1/2 plus the Syllabus Quiz). Neither change touches the deterministic tier or coverage math.

**Deep-pass cognitive demand (the standard).** The deterministic Bloom level above is a conservative inference from the module objectives plus the instrument type; it can misclassify. On course 151110 it read the short papers as Understand, but the actual prompt ("relate and explain the connection between the chapter and the lecture video", "support claims with examples from both") shows **Apply**; the read can also correct downward (an opinion-style discussion prompt is Apply, not the Evaluate a single keyword might suggest). So the standard cognitive-demand step is a deep pass: for each weighted assessment, fetch and read its **actual prompt and rubric** (assignment description, discussion message, rubric criteria; quiz/exam question stems where the API exposes them) and classify the Bloom level the task *requires* from the verbs in the prompt, not the level it inherits or a lone keyword implies. Where an item is objective and its questions come from a bank the API does not return (most exams), classify it by **format** (auto-graded objective cannot exceed Understand) and label that share explicitly. Write the result to the narrative as `cognitive_demand`:
```
"cognitive_demand": {
  "distribution": {"Understand": 88.6, "Evaluate": 10.9, "Remember": 0.6},
  "clusters": [{"label": "7 module discussions", "weight": 10.9, "level": "Evaluate", "basis": "prompts ask students to argue a position and judge ethics"}, ...],
  "format_inferred_weight_pct": 30.8
}
```
The renderer prefers `cognitive_demand` over the deterministic distribution, labels it "read from each assessment's prompt and rubric (deep pass)", shows the per-cluster basis, and states the format-inferred share. Cognitive demand is a within-course diagnostic only: it refines Dimension C and never feeds the tier or coverage. State plainly that higher Bloom does not improve assurance -- an Evaluate-level async post is as AI-substitutable as an Understand-level one.

**Recommendation authoring.** Author each recommendation in the house style: APA-capitalized verbs, stop-slop prose (no em-dashes, no throat-clearing, active voice), neutral tone, and fixes grounded in the Assessment Design Playbook (Step 5) for the specific assessment type. Ground every move in the course's actual items: name the real assessment, its actual prompt or rubric language, and its actual grade weight -- never a generic template move that could apply to any course. Where an ASU-supported tool fits a recommended move, name it from `standards/asu-assessment-tools.md` instead of a generic move.

Structure each recommendation in three parts in the narrative: `judgement` (the assessment and its grade weight, e.g. "Short papers (10 papers; 57.1% of the grade)"), `ai_forward` (a list of moves that build AI use into the task and grade the student's judgment), and `ai_resistant` (a list of process, authenticity, and identity moves). Lead with AI-forward. Within `ai_resistant`, place proctoring and LockDown Browser LAST and label them last-resort: they are weak against current AI (deepfakes and AI voiceovers skirt video), unscalable for faculty, a barrier for students, and they cut against the flexibility online students are promised. Proctoring/LockDown Browser is the last-resort subtype of securing, named identity/condition control (see the report's Section 01 definition); authorship evidence (drafts, decision logs, revision history, reflection, a graded AI-use appendix) is the other, AI-judged kind and also secures. "peer review" does not count on its own. Naming proctoring last is a remediation-presentation rule, not a measurement change.

**"Secure an anchor" defaults to authorship evidence, never proctoring.** When a recommendation's purpose is to secure grade weight (raise coverage), the move is to make an existing high-weight item produce authorship evidence: a required draft/outline, a process or decision log, a personal or course-specific artifact AI cannot generate for that student, or a graded AI-use appendix. Do NOT recommend adding a proctored or LockDown Browser assessment. Watch the trap: the tier is driven by secured-coverage %, and proctoring is the shortest path to more %, so authoring gravitates to surveillance; resist it. Proctoring/LockDown may appear in a recommendation ONLY as a bare trailing clause, and ONLY when authorship evidence genuinely cannot fit the assessment (e.g. a high-stakes closed-skill exam) and the faculty require a hard-secured score; it is never a standalone recommended move, and never on a course where an authorship-evidence path exists. Describing an assessment the course *already* proctors as current secured weight is fine (that is state, not a recommendation).

Do not recommend AI-resistant moves that current multimodal AI already defeats. A prompt that asks students to reference something "only a classmate or the cohort would know" is defeated the moment a student pastes or photographs the reference material into a multimodal model. A prompt built to be hard to type or search (deliberately awkward phrasing, image-only delivery) is defeated the moment a student screenshots it into a multimodal model that reads images directly. Do not present either move as AI-resistant.

Do not recommend removing recall wholesale, especially at an introductory-level course (see Course-Level Calibration above). A low-stakes recall or comprehension check is a legitimate formative step, not a design defect, and stripping it out removes the retrieval practice the higher-order work builds on. Recommend adding a securing or AI-forward layer on top of recall items, not deleting them.

Whenever an AI-forward move asks students to verify, critique, or improve AI output, the SAME sentence that describes the move must also name the rubric and the annotated model examples shared in advance -- do not split the move and its TILT requirement across separate bullets or notes, where a reader skimming the moves list alone would miss it. This is the Criteria component of the Transparency in Learning and Teaching (TILT) framework (Winkelmes); without models, a verify-the-AI task penalizes students who do not already know the target and widens the gap TILT was built to close. Treat specific tool or product recommendations as provisional, to be confirmed against TILT-vetted or program-provided materials rather than asserted.

**Source attribution.** When a recommendation move names a framework, append its short citation inline, drawn from `standards/assurance-sources.md`, e.g. "... shared up front (TILT; Winkelmes 2016)." Populate the report's `method.sources` field (a `[{label, cite}]` list, rendered in Section 06) with the frameworks the report actually used for this course, pulled from that file's citation table -- do not list frameworks the course did not trigger. Set `method.calibration` to a one-line disclosure that the assurance-tier cut (40% secured-weight line), the substitutability threshold (0.75), and the AI-forward rating bands are house-calibrated defaults, not research-derived findings; see `standards/assurance-sources.md` for the full breakdown of which parameters are program-cited, imported, or house-calibrated. Never present a numeric threshold as a finding -- state it as a chosen operating default.

Present the inline report in this section order:

**1. Tier and Assurance Coverage** (leads everything)
State the tier enum, assurance_coverage_pct, and what it means. (#219: tier is derived from secured coverage ONLY -- a mismatch never demotes it. State `counterfeit_claim` separately when true, as its own structural-integrity flag, not as a tier value.)
- `Assured` (>=40% on secured anchors): course has structural integrity; AI cannot substitute for the grade-determining assessments.
- `Partially Assured` (<40% but >0%): partial coverage; high-weight take-home items create substitutability risk.
- `Unassured` (0% secured weight): no authenticated component; the final grade is fully substitutable. If `has_graded_assessments` is False, state that this is a truly empty course (no graded assessments at all) rather than an unassured one -- there is nothing to certify yet.
- `counterfeit_claim` (mismatch items present, any tier): high-weight assessments claim security they do not have; a structural-integrity flag reported alongside the coverage-derived tier, not a tier of its own.

**2. Dimension D: Assurance and AI-Integration**
- Lane summary (Lane 1 count/weight, Lane 2 count/weight, Mismatch)
- Named assessment anchors with points and grade_weight_pct
- Mismatch items with points and grade_weight_pct, and the `counterfeit_claim` flag if true
- AI policy transparency: `ai_policy_in_syllabus` (syllabus-level) and `ai_declared_assessments` (per-assignment count/total) reported as two separate signals, not one blended ratio
- AI-judged findings from substitutability and aias_level items
- Flags: transparency_gap, no_anchor, over_surveillance
- Recommendations (secure / re-lane / declare / liberate vocabulary only)

**2.5. AI-Forward Rating** (deterministic, orthogonal to the tier)
- State `ai_forward_rating` and what it means: this is a SEPARATE axis from the assurance tier. Tier asks "can the course certify learning"; AI-forward asks "does the design teach and assess ethical, judgment-based AI use." Make the orthogonality explicit (e.g., "Assured but Needs-redesign on AI-forward" = secure but teaches no AI literacy).
- Report the six dimension anchors (D1-D6) with their labels: D1 transparency, D2 Bloom alignment, D3 assurance anchor, D4 human judgment, D5 process/authorship, D6 equity/feasibility.
- Name which dimension is the weakest link (it sets the overall rating). If `auto_flagged` is true, state that a high-stakes unanchored item floored the rating to Needs redesign.
- Fold in AI-judged refinements from `judgment_of_ai` and `authorship_evidence_depth` items (narrative only; they never change the rating).
- Recommend using the rubric's design moves: declare, secure, re-lane, liberate. To move toward AI-forward: declare task-level AI use with disclosure + verification (D1), grade the quality of AI-use judgment (D4), and require AI-use documentation / process trail (D5).
- See `ai-forward-framework.md` for the full anchor definitions.

**3. Dimension A: Authenticity Index**
- Rubric presence rate as a deterministic proxy
- AI-judged 5DF findings per top-K assessment (from authenticity_5df items)
- Flag any assessments with rubric_present=False AND high grade weight

**4. Dimension C: Cognitive Demand**
- Grade-weighted Bloom distribution (table or list)
- Arc: increasing / flat / decreasing
- Counterfeit objective items (auto-graded quiz claiming Analyze+)
- AI-judged counterfeit free-response findings (from counterfeit_freeresponse items)
- `bloom_calibration_findings`: deterministic Bloom Guide "Avoid" rules per high-weight item (recall_unprotected, apply_reusable, highorder_no_criteria, create_oneshot), each citing assessment + points + grade_weight_pct

**5. Dimension B: Learning Science (pointer only)**
- Report at most 1-2 assurance-specific deltas (e.g., high-weight summative missing rubric = no practice-feedback loop per terminal-performance test)
- Always include: "Full constructive-alignment analysis: use the `alignment` skill. Full HPI diagnosis: apply Gilbert BEM + Mager+Pipe per standards/instructional-design.md."

**6. Dimension E: Equity and Feasibility** (deterministic)
- `feasibility_band` (low / moderate / high burden) and the named burdens, each citing assessments + grade weight:
  - surveillance (proctored weight >= 50%; equity costs: bandwidth, private space, accommodations, anxiety)
  - synchronous (2+ live/oral Lane 1 anchors; recommend sampling, not all-students)
  - high-stakes concentration (>= 70% weight in items each >= 20%; fragile in a 7.5-week term)
- Report `handgraded_anchor_count` as informational only; do NOT assert a labor burden (no enrollment data).
- State explicitly: E never changes the `tier` and is not cross-course comparable. A course can be `Assured` and `high_burden` at once (it certifies learning but at an equity cost).

**7. Durable Skills**
List inferred durable-skill signals (oral communication, professional writing, reflective practice, etc.) found in the assessment titles.

**8. Assumptions**
State the K cap, model_capability, modality, and prompt_version used. Remind the user these are within-course diagnostics.

Every finding must cite a named assessment + points + grade_weight_pct. Never describe an issue in the abstract.

The rendered HTML report follows the two-axis six-section structure documented at the top of this step (How to Read, The Bottom Line, Course Snapshot, Where It Needs Work, Recommendations by Assessment Type, Method and Caveats). The inline narration above and the rendered report present the same findings; the rendered report drops the framework labels for a reader outside the model and keeps Security (green) and Design (maroon) on parallel axes throughout.

### Step 6: Opt-In Export

**Ask which deliverable(s) to generate.** After the audit runs, the same audit data can render three different documents. Ask the user which they want, and let them pick **any combination** (not one only):

1. **One-page brief**: ID pre-conversation prep: plain language, no jargon, a 60-second skim to feel ready for the first few minutes of a faculty conversation.
2. **Assurance report**: the full verdict-first audit (the detailed record).
3. **Assessment Design Playbook**: the generic five-category design reference (not course-specific).

Run the audit once, then render only the selected outputs. A common pick is the brief plus the report (walk-in brief backed by the full report). Never auto-generate; generate on an explicit choice.

The JSON scorecard and PDF flows below are shared plumbing for whichever documents the user picks.

**JSON scorecard.** Offer:

**JSON scorecard.** Offer:

"Want me to export the raw JSON scorecard for this course? It includes all deterministic fields and the AI-judged items with prompt versions and content hashes for auditability."

If yes: write to `courses/{courseId}/reports/{YYYYMMDD-HHmmss}-assurance-audit.json`.

**PDF report.** Separately offer:

"Want a PDF of this assurance report?"

Ask first, every time. Generate only on an explicit yes. The PDF reflects what the user saw inline: deterministic scorecard plus the AI-judged layer.

If yes, run the flow:

1. Write the scorecard JSON to `courses/{courseId}/reports/{YYYYMMDD-HHmmss}-assurance-audit.json` (the same artifact the JSON offer produces; reuse it if already written).
2. Build a narrative sidecar JSON capturing the inline layer from Steps 3 and 4. Write it to `courses/{courseId}/reports/{YYYYMMDD-HHmmss}-assurance-narrative.json`. The sidecar is optional; the renderer produces a complete deterministic report (sections 01, 02, 06) without it, but include the `report` object so 03/04/05 render. The report renderer reads one authored key, `report`:

   ```
   "report": {
     "period_lines": ["ASUO", "FY24-26"],
     "verdict": {"proof": {"answer","detail"}, "design": {"answer","detail"},
                 "tension": str, "stats": [{"value","label","kind": "sec|warn|plain"}]},
     "snapshot": {"measures": {"segments": [{"level","pct"}], "note": str},
                  "ai_use": str},
     "gaps": {"lede": str,
              "security": {"items": [{"lead","text"}], "note": str},
              "design":   {"items": [{"lead","text"}], "note": str}},
     "recommendations": [{"type","weight_pct","tags": [{"label","kind": "sec|vul"}],
                          "forward": [str], "resistant": [str], "last_resort": str}]
   }
   ```

   Author every field in the house style: APA-capitalized labels, stop-slop prose (no em-dashes), neutral tone. `verdict.proof`/`verdict.design` are the two parallel reads; AI-forward moves lead each recommendation and any proctoring/oral/synchronous move goes in `last_resort`. `snapshot.security_types`/`secured_share_pct` are gone (plan: assurance-binary-labels): the Course Snapshot's SECURITY half is no longer narrative-authored at all -- it renders automatically from the scorecard's `security_ledger`, so there is nothing to write for it here. Only author `snapshot.measures`/`ai_use` (the DESIGN half).

   **Bloom bar (`snapshot.measures.segments`).** Emit one segment per Bloom level present, `{"level": "Apply", "pct": 68.5}`, taken from the deep-pass `cognitive_demand` distribution. The renderer sorts them into canonical Bloom order (Remember, Understand, Apply, Analyze, Evaluate, Create), colors each on the canonical Bloom scale (the Vanderbilt CFT pyramid: blue at Remember rising through green and yellow to red-orange at Create), and draws a legend naming each level with its share. Hue carries the level, so the ordinal reading comes from the legend and the fixed left-to-right order, never from color alone. Zero-weight levels are dropped. Do not pre-bucket levels into "low" and "high": the whole point is that the reader sees which levels the course actually measures. The one-page brief still reads `narrative["brief"]` (unchanged); the two objects are independent. Any earlier flat keys (`substitutability_findings`, `bottom_line`, etc.) are no longer read by the report renderer.
3. Render and export in one command:

   ```
   python scripts/assurance_report_html.py \
     --scorecard courses/{courseId}/reports/{ts}-assurance-audit.json \
     --narrative courses/{courseId}/reports/{ts}-assurance-narrative.json \
     --output    courses/{courseId}/reports/{ts}-assurance-audit.html \
     --pdf       courses/{courseId}/reports/{ts}-assurance-audit.pdf
   ```

   Use the same `{ts}` (YYYYMMDD-HHmmss) for all four files.

**Playwright-missing behavior.** PDF export uses Playwright, an optional dependency. If it is not installed, the command writes the HTML, prints the install hint to stderr, and exits non-zero. It never installs Playwright on its own. When that happens, tell the user the HTML is ready and offer the install command, then stop. Install only on their yes:

```
pip install playwright && python -m playwright install chromium
```

The HTML output is always written even when the PDF step fails, so the user has a usable report either way.

**Assessment Design Playbook (optional companion).** Separately offer the playbook as a design-rationale companion to the audit:

"Want the Assessment Design Playbook? It covers five assessment categories with what makes each vulnerable to AI, the AI-resistant and AI-forward moves, the technologies that help, and the structural anchor for each."

Generate only on an explicit yes. It is self-contained and needs no scorecard:

```
python scripts/assessment_design_playbook.py \
  --output courses/{courseId}/reports/{ts}-assessment-design-playbook.html \
  [--pdf  courses/{courseId}/reports/{ts}-assessment-design-playbook.pdf]
```

The playbook reuses the audit report's CSS and the same Playwright PDF path (same opt-in install behavior as above). Offer it when the audit's recommendations target specific assessment types. For example, if the report recommends converting discussions to video posts, the playbook's Discussions section provides the design rationale and the technology options. It carries no per-course data; it is the same five-category reference every time.

**One-page faculty-conversation brief.** When the user picks the brief, render it with `scripts/assurance_brief_html.py` from the scorecard plus a `brief` object you author into the narrative sidecar:

```
python scripts/assurance_brief_html.py \
  --scorecard courses/{courseId}/reports/{ts}-assurance-audit.json \
  --narrative courses/{courseId}/reports/{ts}-assurance-narrative.json \
  --output    courses/{courseId}/reports/{ts}-assurance-brief.html \
  [--pdf      courses/{courseId}/reports/{ts}-assurance-brief.pdf]
```

The brief is **ID-facing pre-conversation prep**, not the audit record. It renders in a refined editorial layout (white, boxed data strips, ASU maroon eyebrow labels, Palatino numerals) on one US Letter page. Author `narrative["brief"]` in plain language, no framework jargon (no tier codes, Bloom, 5DF, lane, AIAS), a 60-second skim, following the report house style (APA capitalization, stop-slop, neutral tone). `brief` fields (all optional; a section is skipped when absent):

- `period_lines`: `["ASUO", "FY24-26"]`, the right-aligned masthead dateline (scope and data period).
- `course_url`: the live Canvas course link, rendered under the title so an ID can open the shell and confirm they are reviewing the right course. Use the **production** course URL (`https://canvas.asu.edu/courses/{id}`), not a dev or sandbox copy, even when the audit read a dev shell. Only `http://` and `https://` URLs render; any other scheme is dropped.
- `fetched`: the date the audit read the course (ISO `YYYY-MM-DD`), rendered as "... &middot; pulled 2026-07-09" beside the course link. This answers the "which shell, and how current" question a reviewer needs to check a finding against the live course; always set it when `course_url` is set.
- `glance`: `[{value, label, denom?, note?}]`, the Course at a Glance stat band. Use it for cohort context pulled from the program dataset (enrollment/term, sections/term, avg per section, success rate, DEW rate, course evaluation). `denom` renders a small "/5". A `note` renders a small line under the figure (green when it starts with "above"); do NOT put a benchmark note on the course-evaluation cell (show the score only, no "above avg" / "about avg" / "below avg"). Outcome data is population context, never a verdict on the course or faculty.
- `about`: the plain "COM 100 is ..." paragraph, from the syllabus modules and CLOs (read the syllabus; do not assume topics). One paragraph, no academese.
- `how_graded`: one line on how learning is assessed (how weight is spread and by what instrument), rendered under the "How Learning Is Assessed:" run-in heading. The renderer appends "Each type below shows its count and its percentage share of the final grade:" when `graded_items` is present.
- `graded_items`: `[{count, label, weight_pct}]` grouped by assessment type, from the deterministic inventory (the renderer builds the Graded Items band).
- `working` / `exposed`: plain-language bullets for What's Working / Where It's Exposed (green / maroon dots). In the exposed bullets, name the actor so the substitutability point is unambiguous: write "a student could complete the graded work with AI," not "AI can complete the grade" (the latter reads as AI doing the grading).
- `barrier`: one short note on what limits or shapes any change (distributed grade, scale across sections, faculty grading load), rendered as "The Constraint" (field key stays `barrier` for back-compat). This is a **feasibility constraint, not a quality judgment**: it names where a fix has leverage, never what is wrong with the course. Test before writing it: if the sentence names something wrong (no record of how work was done, unsecured weight, shallow items), it is an `exposed` bullet, not a constraint; a constraint names what any change must work around (no single lever across many small items, must stay auto-gradable at 1,000 students, real weight concentrated in two exams). If nothing meaningfully constrains the fix, omit the field.
- `directions`: `[{lead, text}]`, the ranked options rendered under "Improvement Opportunities" (numbered). Frame these as options to raise if the conversation invites them, never an agenda to deliver (diagnose before prescribe). Lead with AI-forward moves; when a direction's job is to secure grade weight, make it an **authorship-evidence anchor** (a required draft, a process/decision log, a course-specific or personal artifact, a graded AI-use appendix), NOT a new proctored or LockDown Browser assessment. Do not put "add a proctored/LockDown check" in the directions; it is a last resort, weak against current AI and an equity barrier, and belongs in the directions only as a bare trailing clause where authorship evidence genuinely cannot fit. Describing an assessment the course already proctors (in `working`/`verdict`) is fine.
- `verdict`: `{proof: {answer, detail}, design: {answer, detail}, x?, y?}`, the two-axis "The Verdict" close (rendered last, after Improvement Opportunities). `proof` answers the fixed question "Can it prove a real student did the work?" (green); `design` answers the fixed question "Does it measure real thinking?" (maroon). `answer` renders bold, `detail` follows in plain language -- author both, no framework jargon. The 2x2 matrix plot is deterministic: the horizontal axis (proof) auto-plots from `security_ledger` (secured / graded assessments) and the vertical axis (thinking) auto-plots from `security_ledger` (ai_forward / (ai_forward + ai_resistant), excluding `n/a` items), falling back to `assurance_coverage_pct` / `ai_forward_rating` only when no ledger is present; optional `x` / `y` (each 0..1) override the auto-plot when a closer read is available. Skipped entirely when `verdict` is absent.

**Assessment Security & Design ledger (leading section, automatic).** Before Course at a Glance, the brief renders a per-assessment "Assessment Security & Design" list -- each graded assessment with a Secured/Unsecured chip and an AI-Forward/AI-Resistant/N/A chip -- straight from the scorecard's `security_ledger`. This is NOT authored in `narrative["brief"]`; it renders automatically whenever `security_ledger` has at least one graded entry, even before you write anything else into the sidecar. Plain labels only, no percentage.

**Plain-language body (no ID jargon).** Author every prose field (`about`, `how_graded`, `working`, `exposed`, `barrier`, `directions`, and the `verdict` answers and details) as plainly as the Verdict block itself, so the brief needs no later plain-language pass. Replace ID vocabulary with plain words: "authorship" becomes "whose work it is" or "that a real student did the work"; "recall" and "recognition" become "memory"; a "process trail" or "process" (the work behind an answer) becomes "how the student worked" or "a draft"; "verify" / "verified" (of identity) becomes "confirm" / "confirmed"; "substitutable" becomes "could be done with AI" or "an AI version would look the same"; "cognitive" becomes "thinking"; "AI-resistant" becomes "harder to hand to AI"; "objectives" becomes "what the course is meant to teach." Keep the concrete terms that are already plain: proctored, webcam, locked browser, auto-graded, rubric, quiz, exam, discussion, paper, upload, take-home.

**Cohort stats (the `glance` band).** When the program enrollment/success/evaluation dataset is available, pull the course's figures filtered to the correct modality (ASU Online), label the scope and period, and treat DEW/success/evaluation as neutral context. Student course evaluations are a perception metric on a 1-5 scale and carry known demographic bias; show the score only, with no benchmark label ("above avg" / "about avg" / "below avg" were removed), and never present it as a quality claim or a verdict on the assessment design.

**ASU Online is the standing context.** Every audited course is asynchronous online, so every graded item is take-home by definition. Do NOT frame "take-home," "unsupervised," or "at home" as a finding in any deliverable; it is a constant across the population and reads as knocking the modality. The issue is always authorship and verification, not location.

### Context Defaults

- `discipline` and `course.level` / `course.tier`: read from `course-config.json`; fall back to course name/code inference.
- `modality`: default `async_online`; single-course interactive sessions MAY refine (ask once if it matters for the judgment).
- `AI policy`: read from syllabus text surfaced in the scorecard AIAS detection result.
- `model_capability`: default `conservative` (recorded in assumptions, never exposed to user as a setting).

Single-course sessions MAY ask one refinement question if context is genuinely ambiguous. Batch runs never stop to ask -- defaults are recorded in `assumptions`.

### Batch Mode

`assurance-audit` is batch-eligible. The generic CLAUDE.md §Batch Execution Agent mechanism applies: a registry fan-out runs `score_course` per course via Agent subagents, each subagent emits a per-course JSON scorecard (saved to `courses/{courseId}/reports/{ts}-assurance-audit.json`), then the parent session calls `scripts/assurance_batch.py` to aggregate and rank the scorecards into a cohort report at `courses/reports/{ts}-batch-assurance.html`.

The assurance batch runner is **fully independent** of `course-audit`'s batch code, `audit_report.py`, and `criterion_evaluator`. It uses only scorecard dicts and the `templates/report-html/report-base.html` shell.

Batch runs never stop to ask context questions. All defaults (modality, model_capability) are recorded in each scorecard's `assumptions` field. Only deterministic scorecard fields (`tier`, `assurance_coverage_pct`) feed the cohort ranking; `needs_ai_review` and AI-annotation fields are never ranking inputs. (#219) `ai_policy_in_syllabus` and `ai_declared_assessments` are reported in `cohort_stats` but never feed ranking. `certifying_on_nothing` excludes courses where `has_graded_assessments` is False; those are reported separately as `no_graded_assessments`.

Trigger phrases for batch: "Run assurance audit on all my courses", "Check which courses certify learning", "Assurance check on [list of courses]".

### Separation from Course-Audit

This skill is SEPARATE from `course-audit`. They audit different things:

| This skill (`assurance-audit`) | `course-audit` |
|---|---|
| Assessment strategy: can students bypass with AI? | QA standards: structural completeness, accessibility, WCAG |
| Produces assurance tier + coverage + authenticity findings | Produces standards compliance report (25 standards, 173 criteria) |
| Never calls `criterion_evaluator` | Drives `criterion_evaluator` |
| Read-only, no staging gate | Read-only (audit modes) or staged (guided review) |

Never route a QA/accessibility/standards request here. Never route an assurance/AI-integrity/authentic-assessment request to `course-audit`.
