Skip to content

TESTING.md

Latest commit

 

History

History
214 lines (148 loc) · 8.08 KB

TESTING.md

File metadata and controls

214 lines (148 loc) · 8.08 KB

Testing

WinBorg uses Vitest for unit tests and Playwright for Electron E2E tests.

For the overall approach and risk mapping, see docs/TEST_STRATEGY.md.

Automated tests

Local quick start

npm install

Windows PowerShell note (npm.ps1 blocked)

On some Windows setups, PowerShell script execution is disabled and running npm may fail with an error about npm.ps1 and Execution_Policies.

Workarounds (pick one):

  1. Use the Windows CMD shim explicitly:
npm.cmd run test:pr
  1. Run npm via Node directly (bypasses PowerShell scripts entirely):
node "$env:ProgramFiles\nodejs\node_modules\npm\bin\npm-cli.js" run test:pr
  1. (Optional) Adjust PowerShell execution policy for your user. Only do this if you understand the security implications:
Set-ExecutionPolicy -Scope CurrentUser -ExecutionPolicy RemoteSigned
# Typecheck
npm run typecheck

# Unit tests
npm test

# Unit tests with coverage (threshold gated)
npm run test:coverage

# E2E (Playwright launches Electron)
npm run test:e2e

# Fast smoke subset
npm run test:e2e:smoke

# Linux / headless Electron (requires xvfb-run)
npm run test:e2e:xvfb

# Linux / headless Electron smoke subset
npm run test:e2e:xvfb:smoke

# Full E2E suite
npm run test:e2e:full

# Manual E2E (may trigger admin prompts / app close)
npm run test:e2e:manual

PR gate

npm run test:pr

This runs typecheck + unit tests (coverage) + E2E smoke.

CI note:

  • PRs run the smoke suite.
  • Nightly and main run the full suite.
  • Linux/Electron E2E in CI runs behind xvfb-run because Electron needs a display server even when the UI is not visibly rendered.

Linux note for Electron E2E

If you run Electron Playwright tests on Linux without an active desktop session, Electron will usually fail very early with Process failed to launch!.

That is the difference between a plain local shell and CI today:

  • your current shell has no X server and no xvfb-run
  • GitHub Actions provides a virtual display via xvfb-run

Use the dedicated scripts when running locally on headless Linux:

npm run test:e2e:xvfb
npm run test:e2e:xvfb:smoke
npm run test:e2e:xvfb:full

If xvfb-run is missing, install Xvfb first, for example on Debian/Ubuntu:

sudo apt update
sudo apt install -y xvfb

Notes about E2E determinism

  • E2E tests run with NODE_ENV=test to enable isolated Electron userData per run.
  • IPC is mocked in E2E so CI does not require real WSL/SSH/Borg.
  • Recovery-drill coverage now includes deterministic Playwright UI flows in e2e/recovery.spec.ts.

Notes about manual E2E

  • test:e2e:manual is intentionally not part of the PR gate.
  • test:e2e and test:e2e:full intentionally exclude @manual tests by default.
  • These tests can click onboarding actions that may lead to UAC prompts and/or app termination on some setups.

CI/CD pipeline

Tests are executed via GitHub Actions on pushes and pull requests. See .github/workflows/cicd.yml.

Manual verification (release-focused)

This test case is designed to verify the experience of a new user on a fresh Windows 11 machine.

Pre-condition:

  • Windows 11 (Fresh Install or VM).
  • No WSL installed (or default state).
  • No borgbackup installed.

Steps:

  1. Download & Install:

    • Download WinBorg Manager Setup x.x.x.exe.
    • Run installer.
    • Pass: App installs and opens Dashboard.

  2. Verify Setup Validation:

    • Observed Behavior: If WSL is missing, the app shows "System Setup" modal with instruction to run wsl --install.
    • If WSL is present but Borg is missing, app prompts to install Borg automatically.
    • Action: Click "Install Borg (Auto)" if prompted.
    • Pass: App completes setup and shows checkmark "System Ready!".

  3. Manual Prerequisites (Only if Auto-Setup fails):

    • Open PowerShell as Admin: wsl --install.
    • Reboot.

  4. Functional Test:

    • Open WinBorg.
    • Go to Connections → add a connection (e.g. ssh://user@host:22).
    • Generate/deploy SSH key if needed → Test SSH succeeds.
    • Go to Repositories → Add Repo → select the connection → set repo path (e.g. /./backup) → Test SSH & Remote Connection → Initialize.
    • Pass: Initialization succeeds.

  5. Backup Test:

    • Create job for Documents.
    • Run Job.
    • Pass: Activity log shows "Success".

Functional test scenarios

Execute these tests manually to certify a release.

4.1 Repository Management

ID Test Scenario Steps Expected Result
R1 Connect to Repo (SSH via Connection) 1. Go to Connections → add a connection, e.g. ssh://user@host:22.
2. Generate key (if missing) and Deploy Key (password prompt may appear).
3. Click Test SSH and verify it succeeds.
4. Go to Repositories → click + Add Repo → select the connection → set repo path, e.g. /./backup → click Test SSH & Remote Connection → "Connect" / "Initialize".		 Connection test succeeds and repo becomes Connected. If host key is unknown, it follows the configured host-key policy (settings) and the UI provides remediation on failure.
R3 Edit Repository 1. Click "Edit" (Pencil) on a Repo Card.
2. Change Name to My Renamed Repo.
3. Save.		 Repo Card title updates immediately to My Renamed Repo. Data persists after restart.
R4 Delete Repository 1. Click "Trash" icon on Repo Card.
2. Confirm "Remove from App".		 Repo is removed from UI. Important: Actual files on disk must remain untouched.
R5 Destroy Repository 1. Click "Trash" icon.
2. Select "Delete files on disk".
3. Type "DELETE".		 Repo is removed from UI AND the remote repository path is gone (or empty).

4.2 Archive Operations

ID Test Scenario Steps Expected Result
A1 Create Backup 1. Open Repo.
2. Click "Create Backup".
3. Select My Documents.
4. Run.		 Job starts. Spinner active. Activity Log shows Success. New archive appears in "Archives" list.
A2 Compare Archives (Diff) 1. Go to "Archives" view.
2. Select Archive A and Archive B.
3. Click "Diff Selected".		 Modal opens showing file tree. Added files in Green, Deleted in Red, Modified in Yellow.
A3 Mount Archive 1. On an Archive, click "Mount" (Drive icon).
2. Select Letter Z:.		 Windows Explorer opens Z:\. You can browse files. Dashboard shows "Active Mounts: 1".
A4 Unmount Archive 1. Click "Unmount" on Dashboard or Tray Icon. Drive Z:\ disappears. Dashboard "Active Mounts" count returns to 0.

4.3 App Configuration

ID Test Scenario Steps Expected Result
S1 Change Theme 1. Open Settings.
2. Toggle "Dark Mode".		 UI switches instantly between Light/Dark mode. Setting persists after restart.
S2 Notifications 1. Settings > Notifications.
2. Enter Discord Webhook.
3. Click "Test".		 Discord receives a "Test Notification" message.
S3 General Settings 1. Toggle "Start with Windows".
2. Restart App.		 Registry keys are set (check Task Manager > Startup).

Release checklist

See docs/RELEASE_CHECKLIST.md.

Supporting docs

Additional checklists: