# Assurance Model Reference

> Adapted from the external "AI Assessment Evaluator" skill. No upstream text copied verbatim.
> ASU-tuned. No IMSCC. 2026 AI-capability baseline.

## AI-Capability Assumption (2026 Baseline)

Claude uses a **Conservative** model assumption by default: current frontier AI tools (GPT-class, Claude-class) can:

- Generate coherent, well-structured essays and professional memos on any topic
- Answer objective knowledge questions across all disciplines at passing levels
- Synthesize multiple sources into a plausible argument
- Mimic a student's writing voice given a sample
- Complete unproctored online quizzes and take-home assignments without detection

This means take-home essays, online quizzes, and any unmonitored written submission are **fully substitutable** under the Conservative assumption unless the task requires something the AI cannot produce (unique process evidence, oral interaction, real-time response, artifact from lived experience, or evaluated by someone who knows the student's demonstrated work across the term).

The Conservative assumption is recorded in `scorecard.assumptions.model_capability`. To assess under a Permissive assumption (AI is a disclosed tool, not a threat), the AIAS integration level shifts the framing.


## Two-Lane Framework

Assessments fall into two lanes based on their structural security properties.

**Lane 1: Authenticated**
The assessment has at least one verifiable security property that makes AI substitution structurally difficult:
- Proctored (software lock, in-person, or LTI proctoring tool)
- Oral or live performance (defense, viva, moot, negotiation, presentation, deposition, hearing)
- Process-trail with dated evidence (portfolio with revision history, lab notebook, clinical log, practicum documentation, iterative drafts with instructor feedback at each stage)
- Peer-evaluated with relationship accountability (peer knows the student's argument across multiple sessions)

Lane 1 anchors are the scored basis of the assurance tier. They cannot be bypassed by submitting AI output because the security property requires the student to be present, traceable, or known.

**Lane 2: Uncontrolled**
The assessment has no verifiable security property:
- Take-home essays, online text submissions, uploaded documents
- Unproctored online quizzes
- Discussion posts submitted asynchronously
- Assignments where submission type does not imply the student did the work

Lane 2 assessments are not inherently bad; the FIRE quality test and AIAS integration determine how well they function. A high-FIRE Lane 2 assignment is harder to substitute than a generic essay prompt.

**Mismatch**
A high-weight item (>=20% of the final grade) that lacks any security property. Mismatch items create structural integrity failure at scale: the course's terminal grade is mostly determined by work the student may not have done. Presence of any Mismatch item triggers the `Counterfeit` assurance tier.


## FIRE Quality Test (Lane 2 Evaluation)

For Lane 2 items, score on four dimensions (1-3 each):

**Focused**: Does the task require engagement with specific course content, evidence from assigned readings, or unique course artifacts that a generic AI response would not produce convincingly?
- 1: Generic prompt (any essay topic, no course specificity)
- 2: References course content but not uniquely
- 3: Requires specific evidence from course materials, cases, or the student's own prior work

**Integrative**: Does the task require synthesizing multiple course concepts, or does it test isolated recall?
- 1: Single-concept recall or restatement
- 2: Connects two or more concepts
- 3: Requires integration across the course arc or with external context the student uniquely controls

**Reflective**: Does the task require the student to reason about their own experience, learning, or practice?
- 1: No reflection component
- 2: Asks for opinion but not tied to personal experience
- 3: Requires documented personal experience, practice log, or clinical encounter the student uniquely holds

**Expansive**: Does the task open onto genuine uncertainty, professional judgment, or contested terrain where AI defaults to the centrist safe answer?
- 1: Definitive correct answer; AI is as likely to be right as the student
- 2: Some judgment required; AI answer would be plausible but generic
- 3: Requires professional judgment, local context, or ethical reasoning where the student's position matters

FIRE score = (F + I + R + E) / 12. Interpretation:
- >=0.75: High FIRE -- substitution risk is structurally reduced even in Lane 2
- 0.5-0.74: Moderate FIRE -- AI can produce a passing answer but not a distinctive one
- <0.5: Low FIRE -- fully substitutable; consider re-laning or redesigning the prompt


## AIAS Level Descriptors (1-5)

The AI-use and Academic-Integrity Spectrum classifies how explicitly and constructively a course addresses AI in its assessments.

**Level 1: Silent**
No mention of AI in syllabus or assignment descriptions. Students have no guidance. Policy is absent or impossible to locate. Transparency flag = True.

**Level 2: Declared Prohibition**
Syllabus or assignment descriptions state that AI use is not permitted, or that all work must be the student's own. Policy exists but is enforcement-oriented. No guidance on productive AI use.

**Level 3: Differentiated Guidance**
Policy distinguishes between AI-permitted and AI-prohibited tasks. Some assignments explicitly allow AI for specific stages (brainstorming, editing) while prohibiting it for others. Students understand the difference.

**Level 4: AI-Integrated Task Design**
Assignments are designed with AI use as a variable. Tasks require evaluative judgment about AI output (critique, revision, comparison, accountability for AI-generated claims). The task itself cannot be completed by submitting AI output because the task requires the student to reason about the AI's work.

**Level 5: AI as Epistemic Partner**
Assessment architecture treats AI as a disclosed tool and evaluates the student's judgment, curation, and accountability for AI-assisted work. Assessment criteria include the quality of AI use decisions, not just the output. Students document their AI interaction and reflect on its limitations.

When scoring AIAS level for a single assessment, use the highest level the text evidence supports. When scoring for the course, use the predominant level across high-weight items. Undeclared = Level 1 by default.

**Construct-drift caution (Ateskan 2026).** AIAS describes the *degree of AI authority* a task grants, not the *frequency* of AI use. Faculty frequently confuse the two: a policy that says "use AI a lot" is not Level 4-5 unless the task also requires evaluative judgment about the AI output and holds the student accountable for it. When judging AIAS level, score the authority and accountability structure, not how often AI is mentioned. A high-frequency, low-accountability policy is Level 2-3, not 4-5. This caution is a within-course diagnostic; it never feeds the deterministic tier or the AI-Forward rating.

For the full AI-forward axis (six dimensions, the four categorical anchors, Bloom calibration, and the evidence base with per-source limits), see `ai-forward-framework.md`.


## Substitutability Test

For Lane 1 assessments, score on four dimensions (0-1 each):

**DS - Distal Substitutability**: Can an AI produce a plausible artifact that mimics this assessment's expected output without access to the student's process?
- 0: No -- output requires real-time presence, lived experience, or unique artifact
- 0.5: Partial -- AI can approximate but a skilled evaluator would notice
- 1: Yes -- AI can produce a passing artifact

**CS - Contextual Specificity**: Does the task require knowledge of context the AI does not have (the student's own prior submissions, local clinical case, specific cohort discussion, faculty relationship)?
- 0: Yes -- requires unique contextual knowledge
- 0.5: Partial -- context helps but is not required
- 1: No -- task can be completed with generic knowledge

**PI - Performance Immediacy**: Does the security property create a real-time accountability event?
- 0: Yes -- synchronous, observed, or witnessed
- 0.5: Partial -- asynchronous but with verifiable process evidence
- 1: No -- async, no process trail

**OFA - Observer/Feedback Accountability**: Does someone who knows the student evaluate the work and would notice inconsistency with prior performance?
- 0: Yes -- faculty or peer knows the student's baseline
- 0.5: Partial -- rubric-graded but single-submission
- 1: No -- autograded or anonymously evaluated

Substitutability score = (DS + CS + PI + OFA) / 4.

Thresholds:
- >=0.75: Substitutable -- this Lane 1 item is more porous than its classification suggests; flag for re-laning or redesign
- 0.50-0.74: Partially substitutable -- structural security is present but has gaps
- <0.50: Secure -- Lane 1 classification is well-supported

A Lane 1 item with Substitutability >=0.75 is a Mismatch candidate even if the heuristic classified it Lane 1. Surface as a finding but do not alter the deterministic `tier` (AI fields never feed tier).


## Assurance Coverage and Tier Bands

Assurance Coverage = sum(grade_weight_pct for secured Lane 1 anchors).

Tier enum (deterministic, the cross-course sort key):

| Tier | Condition |
|---|---|
| Counterfeit | Any Mismatch item present (high-weight item with no security property) |
| Assured | Coverage >=40% AND at least 1 secured anchor; no Mismatch |
| Partially Assured | Coverage >0% and <40%; no Mismatch |
| Unassured | Coverage = 0% (no secured assessment in the course) |

The tier is always set deterministically from the inventory + lane classification. No AI judgment feeds the tier.


## Counterfeit Assurance

Counterfeit Assurance refers to the structural condition where a course's assessment architecture signals security that the design does not deliver. Two forms:

1. **Mismatch**: high-weight item classified as Lane 1 by title or submission type but lacking a genuine security property (an "oral presentation" with no observed performance record; a "portfolio" that is a single document upload with no revision trail).

2. **Counterfeit Cognitive Demand**: an auto-graded objective quiz (multiple choice, true/false, matching) whose title or module context claims to measure Analyze, Evaluate, or Create. Multiple-choice cannot measure these levels deterministically; the claim is counterfeit by type.

Counterfeit Assurance is a structural design problem, not a student-behavior accusation. Report it factually and recommend re-laning or redesign.


## Durable Skills Taxonomy

The following skills are inferred from assessment title patterns. They are not scored; they are signals that the assessment design reaches beyond disciplinary recall.

- Oral communication (presentations, defenses, moots, negotiations)
- Professional writing (briefs, memos, reports, essays)
- Research and analysis (literature reviews, case analyses, evidence synthesis)
- Reflective practice and portfolio (journals, reflections, portfolios, e-portfolios)
- Collaborative work (team projects, peer review, group assignments)
- Clinical and fieldwork practice (clinicals, practica, fieldwork, placements)
- Dispute resolution and negotiation (mediations, negotiations, hearings)
- Timed performance under conditions (exams, quizzes, finals under exam conditions)
- Iterative revision and feedback (draft-revision cycles with instructor feedback)

These signals are additive. A course showing oral communication + professional writing + iterative revision has a richer durable-skill footprint than one showing only timed performance.
