# Plan: `file-remediation` skill — PDF/DOCX/PPTX accessibility remediation

**Status:** Rev 2 — post adversarial review (Opus, 2026-07-20; 15 findings, verdict REVISE → all
must-fix findings addressed below; finding numbers referenced as [R#])
**Author:** Claude (Fable 5) with Brent, 2026-07-20
**Evidence base:** every *shipped-path* engine has a passing live test behind it (§10); items
still marked *to build* carry explicit fallbacks that are themselves the default pass condition.

## 1. Goal

One skill that remediates course files — PDFs (tagged, untagged, scanned), DOCX, PPTX — to the
**PDF/UA-1 machine-verifiable subset** (veraPDF-clean) plus a judgment layer for alt text,
headings, and tables, with a hard product constraint: **remediated output must be visually
identical to the input**, enforced by a render-diff gate (§3, [R3] for gate soundness limits).
[R6]: this plan deliberately does NOT claim "WCAG conformance" — color contrast, alt-text
*quality*, reading-order *correctness*, and link purpose are Matterhorn human checkpoints; the
report always carries the human-subset count as a required field (§7).

## 2. Non-goals

- **No cloud remediation APIs in the default path.** ODL runs **local mode only**; hybrid flags
  are never passed; the E2 regression test runs offline to prove it [R: hybrid leak].
- **No math remediation (MathML/OMML) in v1.** Math-heavy docs get flagged, not mangled.
- **No claim of full PDF/UA or WCAG conformance, ever** [R6]. Report wording is fixed (§7).
- **No PDF forms remediation** in v1 — Assess flags and routes out.
- Rebuild-to-DOCX and PDF→Canvas-page conversion **deferred** (they change appearance; opt-in
  later phase).

## 3. Architecture

Stages: **0 Source-first guardrail** (ask for the DOCX/PPTX source before remediating any PDF;
blocked → backlog item) → **1 Assess** (read-only, batch-eligible) → **2 Route + Remediate**
(engines below) → **3 Verify** (gates) → **4 Deliver + Report**.

### Stage 1 — Assess classifier [R8]

Per-**page**, not per-document: text-layer quality score (real text vs. garbage-OCR vs. none),
image census, table/math density, tag state, encryption, file size. Document classes emitted:
`digital-untagged`, `digital-tagged`, `scanned`, `hybrid` (mixed pages), plus hard routes for
`encrypted` (flag, never crack), `corrupt` (flag), `oversized` (> configurable page/MB caps →
flag or page-range processing), `non-English/RTL` (flag; v1 remediates LTR English/Spanish-class
scripts only, `/Lang` set from detection). AC1 tests all of these, including the negatives.

### Engine E2 — PDF in-situ (the centerpiece)

1. `opendataloader-pdf` CLI (local, bundled JAR) → `tagged-pdf` + `json`. **Sandboxed run**
   [R7]: bounded heap (`-Xmx`), wall-clock timeout, isolated temp dir, input-size cap before
   parse; malformed/hostile-PDF negative test asserts graceful bounded failure.
2. `pikepdf` metadata pass: MarkInfo, DisplayDocTitle, `/Lang`, dc:title + docinfo title,
   `pdfuaid:part=1`, `/Tabs /S`.
3. **Judgment repairs** as post-hoc tag-tree edits, matched via ODL JSON (id/pdfua_tag/page/bbox)
   ↔ PyMuPDF manifest bbox alignment:
   - **Alt text** on every `/Figure` (Claude vision over crops). *Proven live.*
   - **Heading detection + retag** [R1]: detection is NOT left to ODL — the PyMuPDF manifest
     classifier (font size/weight/pattern heuristics + Claude confirmation of ambiguous
     candidates) is the heading authority; it found 28/28 on the fixture where ODL found 10.
     Retag `/P`→`/Hn` by bbox match, then level-normalize (never skip). Low-confidence
     candidates go to Claude with page renders; unresolved ones stay `/P` and are reported.
   - **Table reconstruction** [R2]: topology comes from the judgment layer (Claude reads the
     page render, emits cell grid + header spec — proven on both fixture tables), then
     `pdf_table_weave.py` re-parents the region's MCIDs into Table/TR/TH/TD by cell bbox.
     **Default pass condition is the fallback**: if weave confidence is low, cells stay `/P`,
     the region is flagged, and the report says "table semantics incomplete" — the doc still
     machine-passes (verified). Correct TH/TD is a stretch goal (AC6-S), never a gate.
   - Residual artifacting (page numbers, decorations) via the proven wrapper.
4. **Gates**: veraPDF `--flavour ua1` COMPLIANT (hard), plus the fidelity gate below.

**Fidelity gate** [R3]: pixel-identity was proven at n=1 (clean LaTeX doc). Phase 2 includes a
**diff-evidence task across all Assess classes** (image-heavy, JPEG2000, transparency, scans)
BEFORE the gate policy is frozen. Policy as planned: exact-zero required for tree-edit-only
passes (pikepdf edits never touch content streams); for the ODL re-save, exact-zero is
*expected* (observed) but if per-class evidence shows benign re-encode diffs, the gate becomes a
bounded perceptual threshold **per class**, with the bound and renderer (PyMuPDF, version
pinned; dpi fixed and documented) recorded in the report. Nonzero-beyond-bound = hard abort,
staging preserved, failing page reported. Wording corrected [R3]: ODL *re-serializes* the file
(observed 2.1→3.3 MB); the invariant is *rendered-pixel* identity, not byte identity.

Scanned PDFs: Vision-framework invisible text layer (render mode 3) via PyMuPDF, then E2.

### Engine E1 — Office in-place

DOCX (python-docx + oxml): alt text (`wp:docPr descr`), heading styles with
appearance-preservation (capture run formatting → re-apply as direct formatting, or restyle
Heading N to the doc's visual conventions), `w:tblHeader`, title/lang, numbered-list restart via
`startOverride`, missing styles defined before use.
PPTX (python-pptx + oxml): real title placeholders, **overwrite `descr` on every picture**
(python-pptx defaults it to the filename), `first_row` on tables, core title/lang, z-order
review. Output: same format back. Optional tagged-PDF export via LibreOffice
(`PDFUACompliance=true`, Impress artifact post-fix) with mandatory side-by-side visual QA —
this export never gets the "identical" claim.
E1 follows §Edit Discipline / Office-artifact preservation: copy original, modify minimally,
archive under `courses/{id}/backups/` when in course context.

### Engine E3 — Metadata patch (tagged-but-sloppy PDFs)

pikepdf one-liners only (title/lang/DisplayDocTitle/UA-id/Tabs, bookmarks for long docs);
no retagging. Assess routes here when the structure tree is fundamentally sound.

### Additional judgment items [R15]

v1 flags (report-only): Lang spans for foreign phrases (Latin legal terms — JD relevance),
ActualText for ligature/dropcap runs (LaTeX fixtures emit ﬁ/ﬀ glyphs), outline/bookmarks for
long documents (E3 can add outlines from the heading tree), link-tag audit beyond the 2-link
fixture. Promotion to automated fixes is a later phase.

## 4. Skill surface & routing

- **Skill name:** `file-remediation` (**28th** skill). Intent family: Edit & Remediate.
- Triggers: "make this file/PDF/deck accessible", "remediate this PDF", "tag this PDF",
  "alt text for this document", "Title II / 508 this file".
- **Hard-scope rule to add to CLAUDE.md + AGENTS.md** [R13]: accessibility fixes on **files**
  (PDF/DOCX/PPTX) → `file-remediation`; on **Canvas HTML content** → `bulk-edit`; accessibility
  **assessment of a course** → `course-audit`. 
- **Backlog integration** [R5]: the route is `backlog_manager.py` (`FIX_PATH_TEMPLATES`) — NOT
  the nonexistent `fix_path_router.py` (CLAUDE.md §backlog reference is stale; fix in AC15).
  Add a `file_a11y` template key + the course-audit detector that emits it. Explicit AC (AC13).
- **Setup gate:** courseless operation on local files (join `plugin-feedback` in the exception
  lists of BOTH CLAUDE.md and AGENTS.md [R9]); course context only for Canvas discovery/delivery.
- **Batch:** Assess only (read-only), added to the Batch Execution lists in both contract docs
  [R9]. Stages 2–4 never batch-auto-approve.
- **Canvas delivery** [R12]: replacing an existing course file is a destructive overwrite
  (`canvas_api.upload_file` uses `on_duplicate: overwrite`). Required flow: download + backup
  the live file to `courses/{id}/backups/` → announce + explicit push verb → upload → verify —
  mirroring `push_to_canvas.py` discipline. Never the bare exempt-path upload for replacements.

## 5. Dependencies & bootstrap [R4]

`scripts/setup_remediation.py`, consent-first (one summary prompt: what/where/size — consistent
with Action First: resolve everything else silently, ask only for the download consent), into
**`~/Library/Application Support/asucb/vendor/<tool>-<version>/`** — OUTSIDE the plugin root, so
plugin zip updates never wipe it and no repo bloat is possible. Version-stamped dirs: a pin bump
in the plugin triggers re-fetch of just that tool (re-runs are otherwise no-ops) [R14].
Components: opendataloader-pdf (pip, pinned), Temurin JRE 21 (tarball), veraPDF (unattended
izpack), LibreOffice (deferred until first E1 export request), pymupdf/pikepdf/python-pptx/
pyobjc-Vision (pinned wheels). No brew/sudo/Docker. `ASUCB_REMEDIATION_OFFLINE=1`: bootstrap
skipped; surface = Assess *lite* (format/tag-state probes via already-vendored pip deps if
present, else metadata-only listing) — the degraded capability is stated in the skill output
[R14]. Add the support-dir cache path to `.gitignore` defensively.

## 6. New scripts

`setup_remediation.py`, `remediate_assess.py`, `remediate_pdf.py` (E2 orchestrator),
`pdf_tree_edit.py` (bbox-matched tag-tree editor), `pdf_table_weave.py` (judgment-spec →
Table/TR/TH/TD; fallback-first), `ocr_text_layer.py`, `remediate_docx.py`, `remediate_pptx.py`,
`lo_export.py`, `fidelity_gate.py`, `verapdf_gate.py`. (11 scripts; bump script counts in docs —
see AC15.) Claude-side work lives in SKILL.md: manifest review, alt writing, table specs,
heading confirmation, visual QA, report.

## 7. Judgment-layer contract (SKILL.md)

- Engine → Claude: dense JSON manifest (flagged items, crop paths, candidates); never whole docs.
- Claude → engine: `judgment.json` (alt texts, table specs, heading overrides, title/lang) —
  same shape as the validated prototype.
- **Report enforcement** [R6]: `verapdf_gate.py` emits, as REQUIRED non-omittable fields:
  machine-checks passed/failed, the Matterhorn human-checkpoint count for the profile, the list
  of judgment items Claude resolved, and the list flagged for human spot-check. SKILL.md's
  report template renders all four; "fully compliant/conformant" is a lint-forbidden phrase in
  the report generator (audience-lint style token check).
- Alt-text rules: content + function, no "image of", data figures carry the pedagogically
  relevant trend/values; decorative → artifact.

## 8. Phases & acceptance criteria

**Phase 1 — Assess + bootstrap** (ship alone)
- AC1: classifier fixtures for `digital-untagged`, `digital-tagged`, `scanned`, `hybrid`,
  `encrypted`, `corrupt`, `oversized` — correct class + route for each, including the negative
  routes (flag-not-process) [R8].
- AC2: bootstrap on a clean macOS account → healthcheck green; re-run no-op; **version-bump
  re-fetch** verified; vendor dir survives a simulated plugin dir wipe [R4].
- AC3: batch assess across 2 sandbox courses → per-course + aggregate reports; `file_a11y`
  findings in `backlog.jsonl` via `backlog_manager.py` templates [R5].

**Phase 2 — E2 + E3**
- AC4: **synthetic fixture** reproducing the STP 530 defect classes (untagged LaTeX-style,
  booktabs tables, figures, math, links — built by us, no faculty content committed [R11])
  runs E2 end-to-end offline: veraPDF COMPLIANT + fidelity gate pass + real alts.
- AC5 [R1]: heading pipeline detects the fixture's full heading set via manifest classifier +
  judgment, retags with no level skips, compliant; ambiguous-candidate path exercised (one
  deliberately ambiguous heading → flagged, not guessed).
- AC6 [R2]: table weave applies a judgment table spec; **pass = weave-or-fallback with honest
  report**; AC6-S (stretch): correct TH/TD semantics on both fixture tables.
- AC7: tagged-but-sloppy fixture routes to E3, passes without retag.
- AC8: fidelity gate negative test (corrupted stream → abort, staging preserved) AND the
  **per-class diff-evidence report** (image-heavy/JPX/transparency/scan classes) that freezes
  the gate policy [R3].
- AC9: scanned fixture → OCR layer + tags, compliant, gate pass.
- AC-sec [R7]: hostile-input suite (decompression bomb, malformed xref, oversized) → bounded
  graceful failure, no hang beyond timeout, no temp-dir escape.

**Phase 3 — E1 Office in-place**
- AC10: DOCX fixture — properties set, rendered before/after diff ≤ noise, opens clean in Word.
- AC11: PPTX bad-deck fixture — checklist applied, Impress export + post-fix compliant, deck
  visually unchanged.
- AC12: preservation compliance — original copied; unrelated XML parts byte-identical.

**Phase 4 — Canvas integration + release**
- AC13: course-file discovery via content cache; **backup → announce → push-verb → upload →
  verify** replacement flow [R12]; backlog auto-close on verified delivery.
- AC14: live integration test on sandbox 260866 that **creates then deletes a throwaway file**
  (never overwrites a standing fixture) [R10].
- AC15 [R9] — release-sync checklist, all items explicit:
  README.md:7, CLAUDE.md:12, CLAUDE.md §Skills Reference ("27 skills"→28), plugin.json
  description (verify ≤500 chars after edit — hard marketplace cap), coda-page-template.md
  (both count lines; fix already-drifted script counts 87/83 → actual), Batch Execution lists
  (CLAUDE.md + AGENTS.md) += file-remediation Assess, courseless-exception lists (both docs)
  += file-remediation, stale `fix_path_router.py` reference in CLAUDE.md corrected to
  `backlog_manager.py`, routing table + hard-scope rule added (both docs), both manifests +
  VERSION + tag + CHANGELOG move together per Write Rule 4.
- **CI story** [R10]: heavy-engine tests carry a `remediation_tools` pytest marker; CI runs the
  suite with the marker skipped (deps not provisioned); the FULL suite including marked tests
  is a documented **pre-tag local requirement** alongside the existing live-suite rule. The
  merge gate = CI green + reviewer's local full run; stated in CONTRIBUTING/CLAUDE.md.

Every phase: Codex co-implements, coder ≠ reviewer, PR-only, suite green per the CI story above.

## 9. Risks (updated post-review)

| Risk | Mitigation |
|---|---|
| ODL structure quality varies by class | Assess routes out-of-class docs to flag/manual; judgment layer repairs known gap types; never ship a bad auto-result. |
| Heading detection wrong on non-fixture docs [R1] | Manifest classifier confidence bands; low-confidence → Claude with renders; unresolved stays `/P` + reported. |
| Table weave complexity [R2] | Fallback IS the default pass; stretch AC only. |
| Fidelity gate fails-closed on re-encodes [R3] | Per-class diff evidence task BEFORE policy freeze; bounded per-class thresholds if needed; renderer + dpi pinned. |
| Vendor wipe on plugin update [R4] | Vendor dir outside plugin root, version-stamped. |
| Untrusted-PDF parsing [R7] | Heap cap, timeout, temp isolation, size cap, hostile-input suite. |
| ODL version drift | Pin + regression fixture; deliberate upgrades re-fetch via version stamp. |
| Validator disagreement | veraPDF is the contract gate (profile+version named in report); PDF4WCAG optional second opinion. |
| Non-engineer install friction | One consent prompt, deferred LibreOffice, healthcheck, offline-lite mode. |
| Faculty-content fixtures [R11] | Synthetic fixtures only; no faculty PDFs in git history. |

## 10. Evidence appendix (verified live, 2026-07-20)

1. ODL in-situ auto-tag: pixel-identical 20/20 pages; 4 metadata-only veraPDF failures →
   pikepdf one-liners → PDF/UA-1 COMPLIANT 106/106; alt injection via bbox matching verified;
   4 s runtime. ODL gaps on fixture: 10/28 headings, 0/2 tables, placeholder alts.
2. Manifest classifier + judgment found 28/28 headings and both tables on the same document
   (rebuild-path test) — the detection authority for E2's heading/table repairs.
3. Rebuild chain (opt-in E4, deferred): full PDF/UA-1 pass via DOCX + LibreOffice.
4. Word for Mac AppleScript export = untagged Quartz (excluded). LO 26.2 tags by default;
   `PDFUACompliance` adds the UA identifier.
5. Impress PPTX export viable: placeholders→/H, descr→/Alt, first_row→/TH; double-background
   bug fixed by deterministic artifact wrapper; both decks 106/106; bad deck also machine-passes
   → judgment layer is the real accessibility.
6. Vision OCR: 1.00-confidence lines with bboxes, pip-only.
7. Environment: no brew/Docker/Java/soffice on target machines; unattended JRE/veraPDF/LO
   installs work without admin.
