# Plan: Live Canvas Integration Test Suite

## Goal

Close the mock/live gap: a small, env-gated integration suite that runs against the
user's prod sandbox course (canvas.asu.edu, course 260866) and pins the Canvas
behaviors mocks cannot catch — HTML re-encoding, the full staged-push cycle, the
3-step file upload, the delete gate, and real Link-header pagination.

## Hard constraints

1. **Tests-only PR.** No changes to `scripts/` or existing tests. If a script change
   seems necessary, stop and report — do not make it.
2. **The mock suite is untouched.** `python -m pytest tests/` with no env vars must
   still pass everything, skip all integration tests, add zero new warnings, and stay
   fast. CI (`.github/workflows/ci.yml`) runs exactly that command and has no Canvas
   token — integration tests must skip cleanly there.
3. **Gating.** Integration tests run only when `ASUCB_LIVE_TEST=1`. Course ID comes
   from `ASUCB_LIVE_COURSE_ID` (default `"260866"`). Domain/token come from `.env`
   via the existing `canvas_api` import (it calls `load_dotenv` on import).
   Additionally hard-fail (not skip) if `ASUCB_LIVE_TEST=1` but the resolved domain
   is not `canvas.asu.edu` or the token is missing — never silently run against the
   wrong instance.
4. **No orphans.** Every Canvas object a test creates is titled/slugged with the
   prefix `[ITEST-{run_id}]` (run_id = short uuid per session, generated in
   conftest). Teardown deletes what a test created. A session-start autouse sweeper
   additionally lists pages, assignments, quizzes, and discussions and deletes any
   whose title starts with `[ITEST-` (leftovers from crashed runs), using
   `confirm=True`.
5. **Local side effects cleaned too.** Tests that stage content or produce backups
   under `courses/{course_id}/` must remove their own staged files and `[ITEST-`
   backup dirs in teardown. Do not touch pre-existing local course data.
6. **Coder does NOT run the live tests.** Implement, run the mock suite
   (`python -m pytest tests/ -q`) to prove constraint 2, and verify collection
   (`python -m pytest tests/integration/ --collect-only -q`). The reviewer runs the
   live suite against the sandbox.

## Files to create

```
tests/integration/__init__.py
tests/integration/README.md          # how to run, what it covers, sweeper behavior
tests/integration/conftest.py
tests/integration/test_live_html_roundtrip.py
tests/integration/test_live_push_cycle.py
tests/integration/test_live_file_upload.py
tests/integration/test_live_delete_gate.py
tests/integration/test_live_pagination.py
```

## conftest.py

- Module-level gate: if `ASUCB_LIVE_TEST != "1"`, skip the whole directory via
  `collect_ignore` is NOT enough — use a `pytest_collection_modifyitems` hook or a
  session-scoped autouse fixture that calls `pytest.skip(..., allow_module_level=True)`
  pattern. Simplest correct form: in `conftest.py`,
  `pytestmark`-style skipping does not apply to conftest, so use:

  ```python
  def pytest_collection_modifyitems(config, items):
      if os.environ.get("ASUCB_LIVE_TEST") != "1":
          skip = pytest.mark.skip(reason="live integration tests: set ASUCB_LIVE_TEST=1")
          for item in items:
              item.add_marker(skip)
  ```

  (Applies only to items collected under this conftest's directory.)
- `sys.path` handling: mirror whatever the existing unit tests do to import
  `scripts/` modules (check an existing test file and copy the idiom).
- Fixtures:
  - `live_config` (session): asserts token present + domain == `canvas.asu.edu`
    (hard `pytest.fail` otherwise, per constraint 3), returns
    `canvas_api.get_config(course_id=os.environ.get("ASUCB_LIVE_COURSE_ID", "260866"))`.
  - `run_id` (session): `uuid4().hex[:8]`.
  - `itest_prefix` (session): `f"[ITEST-{run_id}]"`.
  - `sweep_orphans` (session, autouse when gate is on): before yield, list pages /
    assignments / quizzes / discussions via existing `canvas_api` list functions and
    delete anything titled starting `[ITEST-` (confirm=True). Silent on empty.
  - `page_factory` / `assignment_factory` / etc. as needed: create object via
    `canvas_api.create_*` (pass `from_staging=True` where the signature supports it
    to avoid the ungated-create warning — check each signature), register for
    teardown deletion with `confirm=True`.
- Keep API call volume low; no sleeps unless Canvas eventual-consistency forces one
  (if a just-created object 404s on immediate read, retry up to ~5x with 1s sleep —
  put that in one helper, don't scatter sleeps).

## Test files

### test_live_html_roundtrip.py — pin what Canvas does to page bodies

Create one page, then for each tricky body: update (via the gated path:
`stage` + `push_page`, or `update_page(..., from_staging=True)` — prefer the
simpler `update_page` here since push-cycle mechanics are covered separately),
fetch back with `get_page`, and assert the *documented* behavior:

1. HTML entities: `&amp;`, `&lt;`, curly quotes, em-dash — assert what survives
   (assert equality where it holds; where Canvas rewrites, assert the rewritten
   form so the quirk is pinned, with a comment naming it).
2. HTML comments including `<!-- canvas-wrapped -->` — pin whether Canvas strips
   them (MUST-2 says it does; the test makes that executable).
3. `<script>alert(1)</script>` — assert Canvas strips/neutralizes it (this is the
   iframe-only rule's foundation).
4. Inline `style=""` attributes — assert preserved.
5. `<iframe src="https://canvas.asu.edu/...">` — assert preserved.

Write each expectation as its own test function (parametrize where natural) so a
future Canvas behavior change fails with a named quirk, not a blob diff.

### test_live_push_cycle.py — the money path, end to end

1. Create a page with body A (factory).
2. `staging_manager.stage(course_id, "pages", slug, body_B)`.
3. `push_to_canvas.push_page(config, slug, course_id)` → assert `ok`.
4. `get_page` → assert body B live (field-targeted: assert a unique sentinel
   substring from body B is present, and body A's sentinel absent — NOT whole-body
   equality, per MUST-2).
5. Assert staging cleared (`get_staged` returns None) and a backup of body A exists
   (`backup_manager.list_backups`).
6. `backup_manager.restore_backup(...)` → `get_page` → assert body A's sentinel back.

### test_live_file_upload.py

1. `upload_file` a small text/html file into a folder from `get_or_create_folder`
   → assert file id returned; fetch the file via API to confirm existence; delete
   the file via API afterward (use the Files DELETE endpoint through
   `_request_with_retry` or a direct authorized request if no helper exists —
   tests may call the API directly for *cleanup only*).
2. Staged-image token rewrite: `stage_file` a tiny PNG (generate 1x1 PNG bytes in
   the test, no binary fixture files), stage a page body containing
   `<img src="asucb-staged-file://{key}">`, `push_page`, then assert the live body's
   img src matches `/files/{id}/preview` and the file exists in course files.
   Clean up page + uploaded file.

### test_live_delete_gate.py

1. Create page. `delete_page(config, slug)` with no confirm → assert
   `PermissionError` raised AND page still live.
2. `delete_page(config, slug, confirm=True)` → assert gone (get_page returns
   None/404) AND a pre-delete backup exists locally.

### test_live_pagination.py

1. Create 5 pages via factory.
2. `paginated_get(f"{config['course_url']}/pages", config["headers"], params={"per_page": 2, "search_term": itest_prefix_fragment})`
   — note: if `search_term` can't match the bracketed prefix, fetch unfiltered and
   filter client-side; the assertion is that all 5 created slugs appear and no
   duplicates exist across pages (proves Link-header traversal on a real response).

## Acceptance criteria

- [ ] `python -m pytest tests/ -q` (no env vars): all existing tests pass, all
      integration tests skip, zero new warnings.
- [ ] `python -m pytest tests/integration/ --collect-only -q` shows all new tests.
- [ ] Live run (reviewer only): `ASUCB_LIVE_TEST=1 python -m pytest tests/integration/ -q`
      passes against course 260866.
- [ ] After a live run: no `[ITEST-` objects remain in the sandbox; no `[ITEST-`
      staging or backup files remain under `courses/260866/`.
- [ ] No diffs outside `tests/integration/` and this plan doc.
- [ ] README.md in the directory documents the run command, env vars, sweeper, and
      the release-checklist role ("run before tagging").

## Process

Coder: Sonnet subagent — feature branch `test/live-integration-suite`, conventional
commit `test(integration): live Canvas integration suite against sandbox`, push,
open PR. Reviewer: parent session (reads full diff, runs mock suite, runs live
suite against 260866). Merger: user, or gate script on all-green.
