# Plan: Remediation Opportunity Audit (file-remediation Phase 1.5)

**Status:** Rev 2 — post adversarial review (Opus, 2026-07-20; 10 findings, verdict REVISE; all
must-fixes addressed, referenced as [R#])
**Author:** Claude (Fable 5) with Brent, 2026-07-20
**Parent:** `docs/plan-file-remediation-skill.md` (Phase 1 shipped in v7.63.0)

## 1. Goal

Per-course, read-only-toward-Canvas audit that turns Phase 1's flat Assess output into a
**prioritized remediation worklist**: per file an impact ÷ effort score, a quadrant, and one
**disposition** — including "retire, don't remediate" (user-approved). Answers "which fixes buy
the most student access for the least effort before session start."

## 2. Non-goals & write posture [R4]

- **No Canvas writes, no destructive or file-mutating actions from this audit itself.**
  "Retire" is always a recommendation, never auto-executed. The recommended action is
  delete-with-local-backup via `scripts/retire_file.py` (not archive + unpublish — field
  lesson 2026-07-22: Ally scans unpublished files too, so unpublishing a stray does not
  remove it from the accessibility score), gated behind `retire_file.py`'s own explicit-verb
  confirmation and executed by a human, never by this audit.
- Local artifacts it DOES write (and nothing else): content-cache refresh, the JSON + HTML
  report under `courses/{id}/reports/`, and (opt-in `--emit-backlog`) backlog entries.
- No cross-course rollup (per-course only). No Ally integration (no official API; JWT-intercept
  is brittle/ToS-gray — Ally rescan lag is a report footnote). No usage analytics (file download
  counts not API-accessible). No new skill — a mode of `file-remediation` Stage 1.

## 3. Data sources — with the live-join made explicit [R1][R2]

The content cache holds **pages HTML and extracted file text only** (verified); assignment/
discussion/quiz/syllabus bodies are NOT cached, and Assess records carry no file_id/visibility.
So linkage and visibility come from a **live read-only join** at audit time:

| Signal | Source | Live or cache |
|---|---|---|
| File inventory + class/route/flags | Phase 1 Assess records | cache of prior run or fresh run |
| `canvas_file_id`, `hidden`, `locked` | `canvas_api.list_files` | **live** (Phase-1 record extension below) |
| Links in pages | `content_cache/pages/*.html` after refresh | cache (refreshed first; age stamped) |
| Links in assignments/discussions/quizzes/syllabus | existing `canvas_api` list/get methods' body fields | **live sweep** [R1] |
| Links via module items (type File) + module/item published state | modules API | **live** |
| Page published state | `get_all_pages` index | **live** (cache doesn't persist `published`) [R2] |
| Urgency | `session_calendar` via the **term normalizer** below | local |

**Phase-1 record extension** [R2][R7]: `remediate_assess.py` course mode persists
`canvas_file_id`, `hidden`, `locked` into each record (fields exist in the Files-API response it
already receives and currently discards). No `--records-out`; the existing `--json` output is
the reuse surface.

**Link matching** [R9][R10]: regex `/files/(\d+)` (matches `/files/123`, `/files/123/download`,
`/files/123/preview`, `/courses/X/files/123`, with query params). Reach counts **distinct
(source_type, source_id) pairs**, not raw hits — a page linking the same file thrice counts
once. Known-invisible link carriers (LTI, Canvas Studio, New Quizzes, `canvas-user-content`
CDN URLs, `/media_objects/`, other courses, direct shares) are enumerated in the report's
uncertainty note.

## 4. Term normalizer + urgency [R3]

Registry terms look like `"2026 Spring C"` / `"2026 DEV"`; the calendar keys look like
`"Fall 2026"` with session as a separate argument, and currently contains **only Fall 2026**.
Deterministic normalizer: `"2026 Spring C"` → (`"Spring 2026"`, session `"C"`); unparseable or
not-in-calendar (incl. `"2026 DEV"` and all non-Fall-2026 terms today) → urgency ×1 + `no_term`
flag surfaced in the report ("urgency neutral — term not in calendar"). Inside the 4-week build
window (`build_deadlines`) → ×1.5. AC7 tests the normalizer + window math from injected
term/today fixtures, never the sandbox (whose term is `"2026 DEV"` — permanently neutral).

## 5. Scoring model (deterministic; no LLM anywhere in scoring)

- **Severity**: scanned 4 · digital-untagged/hybrid 3 · digital-tagged **2 flat when veraPDF is
  not vendored** (`severity_basis: "class-only"` flag) or 1–2 by failedRules when it is [R10].
  Office files: 1 + 1 per flag category, cap 4.
- **Reach**: distinct visible linking sources (published pages/modules/assignments/etc.).
  Hidden/locked file, or all linkers unpublished → reach 0 (+`unpublished_only` where apt),
  reported but excluded from the headline.
- **Impact = severity × min(reach, 5) × urgency.**
- **Effort** [R10]: E3=1 · E1=2 · E2-insitu=3 · OCR+E2=4 · **E2+OCR-pages=4** ·
  forms/math-heavy flag=5 · encrypted/corrupt/missing/unsupported = **n/a — cannot process**
  (disposition Flag; excluded from effort math entirely).
- **Quadrant — fixed thresholds, not a median split** [R5] (the discrete value space is
  tie-dense; a median is degenerate): *Quick win* = impact ≥ 6 and effort ≤ 2 · *Plan* =
  impact ≥ 6 and effort ≥ 3 · *Batch later* = 0 < impact < 6 · *Question the file* = impact 0
  or dispositions 3–5. Ties need no rule — thresholds are total. <6 scoreable files → plain
  ranked list, no quadrant.

Headline: **student-facing files needing work** (reach ≥ 1); total files as the secondary line.

## 6. Dispositions

1. **Remediate now / Plan** — per quadrant; Plan items scheduled against `runway_status`.
2. **Request source** — untagged/scanned PDF with **no source match in the cache**, phrased
   exactly so [R8]: matching uses normalized stems (strip `{id}-` prefix, drop ALL extensions
   including `.docx.txt` doubles, `_`↔space, casefold). The cache omits sources that failed
   text extraction or exceeded caps, so absence-of-match ≠ absence-of-source — the phrasing
   carries that uncertainty, mirroring retire. Emitted as **blocked-on-instructor** via the
   backlog's blocked routing (`append_finding`-style status), not the actionable queue [R4].
3. **Convert to Canvas page** — *advisory only; the executor ships in a later phase* (parent
   plan defers the conversion engine) [R10]. Suggested only when digital-untagged AND
   page_count ≤ 5 AND math_hint false AND image_total == 0 — and labeled with the math_hint
   blind-spot caveat (Form-XObject fonts invisible).
4. **Retire candidate** [R1] — zero inbound links across **all searched surfaces** (pages,
   assignments, discussions, quizzes, syllabus, module items — the live sweep makes this claim
   honest). MANDATORY phrasing: "no links found in searched course content (pages, assignments,
   discussions, quizzes, syllabus, modules); links from LTI tools, Studio, New Quizzes, or
   other courses are not visible to this audit." Never "unused." Human-confirmed; act via
   `scripts/retire_file.py` (delete-with-local-backup, not archive + unpublish — unpublished
   files are still scanned by Ally). The confirmation gate protects against false orphans,
   not the softness of the action. Emitted as blocked-on-human.
5. **Flag** — cannot-process + forms/math-heavy: named handoff.

**Delete-verb gate** [R6]: the report generator lints the FULL rendered report — including the
Claude-written narrative — against a denylist (`delete`, `remove`, `purge`, `trash`, `erase`
as recommended actions) plus the inherited forbidden phrases ("fully compliant/conformant").
Lint failure blocks report emission. Test-checkable on both surfaces.

Claude's role stays narrative-only: the summary applies the 4-part recommendation contract
(gap/data/principle/mechanism) over the deterministic JSON; scores are ordinals, shown next to
their inputs.

## 7. Implementation

- `scripts/remediation_opportunities.py`: orchestrates refresh → Assess reuse-or-run → live
  join (files/pages/modules + assignment/discussion/quiz/syllabus body sweep) → linkage index →
  score → dispositions → JSON + HTML report (`report-base.html`) → optional backlog emit
  (actionable via `add`, blocked via the blocked-status path) [R4]. Budget honestly revised to
  ~300 lines given the live sweep [R1].
- `remediate_assess.py`: persist `canvas_file_id`/`hidden`/`locked` in course-mode records.
- SKILL.md: "Opportunity Audit" section — triggers ("what should we fix first", "remediation
  priorities", "which files are worth fixing", "opportunity audit"), command, report rules.
- No new dependencies.

## 8. Acceptance criteria

- AC1 Linkage + dedup [R9]: fixture with file A linked from 2 pages (one page links it twice
  via `/preview` and `/download`) + 1 module item → reach 3; file B unlinked anywhere → 0.
- AC2 Visibility [R2]: linked only from unpublished page → reach 0 + flag; hidden file →
  excluded from headline; both via stubbed live join.
- AC3 Scorer: deterministic; fixed-threshold quadrant boundaries unit-tested incl. reach cap,
  effort n/a exclusion, <6-file ranked-list path.
- AC4 Retire safety: orphan → `retire-candidate`, output contains the full searched-surfaces
  phrasing, passes the delete-verb lint; canvas_api stub raises on any write method — audit
  completes without touching one.
- AC5 Request-source [R8]: normalized-stem match suppresses the disposition
  (`12345-Lecture 1.pdf` vs cached `Lecture_1.docx.txt`); versioned stem
  (`syllabus_v2_FINAL.pdf` vs `syllabus.docx`) → no match → disposition with
  uncertainty phrasing; both directions tested.
- AC6 Report lint [R6]: renders from report-base.html; forbidden phrases AND delete-verb
  denylist enforced on the rendered output including an injected narrative; failing narrative
  blocks emission.
- AC7 Urgency [R3]: normalizer unit tests (`"2026 Spring C"` → Spring 2026/C; `"2026 DEV"` →
  no_term); window ×1.5 via injected term+today; sandbox never used for urgency assertions.
- AC8 Backlog routing [R4]: actionable dispositions → standard `file_a11y` entries with
  disposition in issue text; request-source and retire-candidate → blocked-status entries;
  counts assert the split.

## 9. Risks

| Risk | Mitigation |
|---|---|
| Linkage blindness → false orphans | Live sweep of all six searchable surfaces [R1]; mandatory uncertainty phrasing naming the invisible carriers; human-confirmed, archive-not-delete. |
| Stale page cache → wrong reach | Refresh first; cache age stamped in report; assignment/quiz/discussion/syllabus are live, not cached. |
| Narrative safety drift | Delete-verb + forbidden-phrase lint on the rendered report blocks emission [R6]. |
| Score theater | Inputs shown beside every score; ordinals, not claims. |
| Calendar coverage (one term today) | Neutral urgency + visible `no_term` note; resync calendar annually per standards doc. |
| API weight of live sweep | Single pass per object type via existing paginated list calls; per-course scale (tens of objects), acceptable. |

## 10. Out of scope, noted for later

Cross-course rollup; student-exposure metric; official Ally API (ask Michael Arsenal);
TagFlow-style manual zone pass; conversion-engine execution for disposition 3; usage analytics
if New Analytics ever exposes file downloads.
