test-e2e

In plain terms

Runs the end-to-end (Playwright) tests in the background and reports what passed or failed.

What this skill does

Run Playwright E2E tests headless and report results

fastapiexpress

Runs all Playwright E2E tests headless and reports a structured pass/fail summary with diagnosis of any failures.

Trigger Conditions

• Keywords: "run e2e", "run end-to-end tests", "playwright tests", "test-e2e", "e2e tests"
• Called automatically by /pr-ready (Step 11)

---

Step 0 — Load config & discover layout

This skill is project-portable. Resolve a few values once here, then substitute them literally into every step below (shell state does not persist between commands, so don't rely on exported vars — capture the values and reuse them).

1. Read the optional per-project config at .claude/test-e2e.config.json (repo root):

cat .claude/test-e2e.config.json 2>/dev/null || echo "NO_CONFIG"

Schema — every key optional (a documented copy is at templates/test-e2e.config.sample.json):

{
  "frontendDir": "frontend",
  "seedCommand": "npm run test:e2e:seed",
  "backend": {
    "healthUrl": "http://localhost:8000/api/system/health",
    "startCommand": "cd backend && uvicorn main:app --reload --port 8000"
  }
}

1. Auto-detect whatever the config omits:

FE=$(dirname "$(find . -maxdepth 3 -name 'playwright.config.*' -not -path '*/node_modules/*' | head -1)")
echo "frontendDir: ${FE:-NOT_FOUND}"
grep -q '"test:e2e:seed"' "$FE/package.json" 2>/dev/null && echo "seedCommand: npm run test:e2e:seed" || echo "seedCommand: (none)"

Resolved values (config wins; else detection; use these literally below): - <frontendDir> — the directory holding playwright.config.*. If not found, STOP and ask the user where the E2E tests live. - <seedCommand> — the MFA seed command, or none → skip Step 1.5 (the app logs in without MFA). - <healthUrl> / <startCommand> — backend health URL + start command. If neither config backend nor a backend/ dir exists, the project is frontend-only → skip Step 1.

---

Step 0.5 — Scaffold MFA helpers if missing

This skill bundles reusable, parameterized helpers for apps where login requires mandatory TOTP MFA backed by Supabase Auth (see templates/ and templates/SETUP.md).

Only relevant when <seedCommand> is set (or the app uses Supabase MFA but isn't wired up yet):

ls <frontendDir>/e2e/seed-mfa.mjs <frontendDir>/e2e/helpers/auth.ts 2>/dev/null

• If both exist: already wired up — continue.
• If missing AND the app uses Supabase Auth with mandatory MFA: offer to scaffold, then follow templates/SETUP.md (edit the CONFIG block in helpers/auth.ts, add the test:e2e:seed npm script, merge the Playwright projects, copy test-e2e.config.sample.json → .claude/test-e2e.config.json, gitignore .env.e2e + e2e/.auth/):

cp -R "$CLAUDE_SKILL_DIR/templates/e2e/." <frontendDir>/e2e/

Do NOT blindly overwrite existing files — if only some are present, diff before copying. - If the app does not use MFA: skip this step and Step 1.5.

---

Step 1 — Backend Reachability Check

Skip if the project is frontend-only (no <healthUrl>). Otherwise verify the backend is reachable:

curl -s -o /dev/null -w "%{http_code}" <healthUrl> || echo "unreachable"

• If reachable (200): continue to Step 1.5.
• If not reachable: attempt to start the backend with <startCommand> in the background:

<startCommand> &

Then poll until it responds (max 15 seconds):

for i in {1..15}; do
  sleep 1
  STATUS=$(curl -s -o /dev/null -w "%{http_code}" <healthUrl>)
  [ "$STATUS" = "200" ] && echo "Backend ready" && break
  echo "Waiting... ($i/15)"
done

• If the backend comes up: inform the user ("Backend was not running — started it automatically") and continue.
• If still not reachable after 15 seconds: STOP and tell the user:

  "Could not start the backend automatically. Please start it manually with <startCommand> and re-run /test-e2e."

---

Step 1.4 — Preflight the config (recommended)

Misconfiguration (wrong Supabase env, bad creds, dead service-role key) otherwise fails late and cryptically. If the project has a doctor script, run it to validate every parameter against reality first — it's read-only and exits non-zero on the first problem with a pinpointed message:

cd <frontendDir> && npm run test:e2e:doctor 2>&1

• If the test:e2e:doctor script doesn't exist: skip — the per-step checks below still apply.
• All ✅: continue.
• Any ❌: STOP and report the failing check verbatim (it names the exact parameter to fix — e.g. E2E credentials valid — sign-in failed, service_role key works — admin factors → 403).

The doctor does NOT validate the login-form data-testids in helpers/auth.ts CONFIG. Those are exercised by the setup project, which logs in first — so a wrong selector fails the setup (and Step 2) immediately, before any feature test, and Step 4's diagnosis table maps it to a missing/mismatched data-testid.

---

Step 1.5 — Seed the mandatory MFA factor

Skip if <seedCommand> is none. Otherwise: login goes through mandatory TOTP MFA, and the suite derives codes from E2E_TOTP_SECRET, provisioned by seeding a verified factor on the E2E user (a single factor only allows one verification per 30s window, so this must run before the tests):

cd <frontendDir> && <seedCommand> 2>&1

• On success (✓ Wrote E2E_TOTP_SECRET to …): continue to Step 2.
• If it fails (e.g. sign-in failed): the E2E user may not exist in Supabase, or the local Supabase stack isn't running. Tell the user to start local Supabase and ensure the user in <frontendDir>/.env.e2e exists, then re-run /test-e2e.

---

Step 2 — Run Tests

cd <frontendDir> && npx playwright test --reporter=line 2>&1

Tests run headless (Chromium only). Capture stdout for analysis.

---

Step 3 — Parse Results

From the output, extract: - Total tests run - Number passed - Number failed - Number skipped (if any) - Names of failed tests (from ✘ <test name> lines) - Error messages for each failure

---

Step 4 — Diagnose Failures

For each failed test, classify the failure using the table below and provide a diagnosis:

Error pattern	Diagnosis
locator.click: Element not found with data-testid	Missing data-testid attribute in source component — needs to be added
locator.click: Element not found with getByRole	Component not rendered or dialog not open
strict mode violation: resolved to N elements	Selector too broad — needs more specific locator
Timeout 30000ms exceeded on login or navigation	Backend not responding or login credentials wrong
Expected URL to match /pattern/	Route changed or redirect not working
Expected: visible / Received: hidden after action	Async operation not awaited, missing waitFor
net::ERR_CONNECTION_REFUSED	Backend or frontend dev server not running
Test fails in suite but passes alone	Race condition — test file needs test.describe.configure({ mode: "serial" })

---

Step 5 — Report

Present a structured summary:

E2E Test Results
================
✅ Passed:  N
❌ Failed:  N
⏭ Skipped: N
Total:      N

Spec files:
  ✅ e2e/example.spec.ts    (N tests)
  ✅ e2e/auth.spec.ts       (N tests)
  ✅ e2e/users.spec.ts      (N tests)
  ❌ e2e/roles.spec.ts      (N/N failed)

For each failure, show:

❌ <test name>
   File:      e2e/<spec>.spec.ts:<line>
   Error:     <error message>
   Step:      <last step before failure>
   Diagnosis: <one-line diagnosis>
   Fix:       <suggested fix>

If all tests pass:

✅ All N E2E tests passed.

---

Step 6 — Auto-Fix (Optional)

If failures are simple and contained (missing data-testid, wrong selector, changed button text), apply the fix directly: 1. Read the affected source file and/or E2E spec 2. Apply the targeted fix using Edit tool 3. Re-run the failing spec: cd <frontendDir> && npx playwright test e2e/<spec>.spec.ts --reporter=line 4. Confirm the fix resolved the failure before reporting

Do NOT auto-fix if the failure indicates broken application logic, a missing backend endpoint, or a UI regression — report these to the user instead.

Max 1 fix iteration. If the test still fails after fixing, report the remaining issue to the user.

Runs all Playwright E2E tests headless and reports a structured pass/fail summary with diagnosis of any failures.

Trigger Conditions

• Keywords: "run e2e", "run end-to-end tests", "playwright tests", "test-e2e", "e2e tests"
• Called automatically by /pr-ready (Step 4b)

---

Step 1 — Backend Reachability Check

Before running tests, verify the backend is reachable:

curl -s -o /dev/null -w "%{http_code}" http://localhost:5001/health || echo "unreachable"

• If reachable (200): continue to Step 2.
• If not reachable: attempt to start the backend automatically:

cd /Users/ronaldtermaat/Applications/SalesFocus/backend && npm run dev &

Then poll until it responds (max 15 seconds):

for i in {1..15}; do
  sleep 1
  STATUS=$(curl -s -o /dev/null -w "%{http_code}" http://localhost:5001/health)
  [ "$STATUS" = "200" ] && echo "Backend ready" && break
  echo "Waiting... ($i/15)"
done

• If the backend comes up: inform the user ("Backend was not running — started it automatically") and continue to Step 2.
• If still not reachable after 15 seconds: STOP and tell the user:

  "Could not start the backend automatically. Please start it manually with cd backend && npm run dev and re-run /test-e2e."

---

Step 2 — Run Tests

npx playwright test --reporter=line 2>&1

Tests run headless (Chromium only). Capture stdout for analysis.

---

Step 3 — Parse Results

From the output, extract: - Total tests run - Number passed - Number failed - Number skipped (if any) - Names of failed tests (from ✘ <test name> lines) - Error messages for each failure

---

Step 4 — Diagnose Failures

For each failed test, classify the failure using the table below and provide a diagnosis:

Error pattern	Diagnosis
locator.click: Element not found with data-testid	Missing data-testid attribute in source component — needs to be added
locator.click: Element not found with getByRole	Component not rendered or dialog not open
strict mode violation: resolved to N elements	Selector too broad — needs more specific locator
Timeout 30000ms exceeded on login or navigation	Backend not responding or login credentials wrong
Expected URL to match /pattern/	Route changed or redirect not working
Expected: visible / Received: hidden after action	Async operation not awaited, missing waitFor
net::ERR_CONNECTION_REFUSED	Backend or frontend dev server not running
Test fails in suite but passes alone	Race condition — test file needs test.describe.configure({ mode: "serial" })

---

Step 5 — Report

Present a structured summary:

E2E Test Results
================
✅ Passed:  N
❌ Failed:  N
⏭ Skipped: N
Total:      N

Spec files:
  ✅ tests/e2e/smoke.spec.ts     (N tests)
  ✅ tests/e2e/auth.spec.ts      (N tests)

For each failure, show:

❌ <test name>
   File:      tests/e2e/<spec>.spec.ts:<line>
   Error:     <error message>
   Step:      <last step before failure>
   Diagnosis: <one-line diagnosis>
   Fix:       <suggested fix>

If all tests pass:

✅ All N E2E tests passed.

Write Results File (REQUIRED)

After parsing results, always write a JSON results file that CI uses as a deploy gate:

cat > tests/e2e/.results.json << EOF
{
  "commit": "$(git rev-parse HEAD)",
  "timestamp": "$(date -u +%Y-%m-%dT%H:%M:%SZ)",
  "status": "passed|failed",
  "total": N,
  "passed": N,
  "failed": N,
  "skipped": N
}
EOF

• Set status to "passed" only if all tests pass (failed = 0)
• This file MUST be committed alongside the code — the deploy workflow verifies it matches the deployed commit

---

Step 6 — Auto-Fix (Optional)

If failures are simple and contained (missing data-testid, wrong selector, changed button text), apply the fix directly: 1. Read the affected source file and/or E2E spec 2. Apply the targeted fix using Edit tool 3. Re-run the failing spec: npx playwright test tests/e2e/<spec>.spec.ts --reporter=line 4. Confirm the fix resolved the failure before reporting

Do NOT auto-fix if the failure indicates broken application logic, a missing backend endpoint, or a UI regression — report these to the user instead.

Max 1 fix iteration. If the test still fails after fixing, report the remaining issue to the user.