Skip to content

feat(CC-ERATE-000056B): Postgres import validation — dude-mcp-01 runbook#51

Open
steven-dracker wants to merge 5 commits intomainfrom
feature/postgres-import-validation
Open

feat(CC-ERATE-000056B): Postgres import validation — dude-mcp-01 runbook#51
steven-dracker wants to merge 5 commits intomainfrom
feature/postgres-import-validation

Conversation

@steven-dracker
Copy link
Copy Markdown
Owner

Summary

  • Adds scripts/validate-postgres-import.sh — self-contained end-to-end Postgres validation script for dude-mcp-01
  • Cherry-picks 000056 Phase 1 + 000056A (dual-provider config + PRAGMA guard) as this branch's foundation
  • No new application code changes beyond what is in feat(CC-ERATE-000056A): Postgres-first ingestion foundation #50

What the script validates

  1. Credential retrieval — Keeper Commander get with KEEPER_RECORD env var, or ERATE_PG_PASSWORD override
  2. Postgres connectivitypg_isready / nc check before attempting startup
  3. App startupDatabaseProvider=Postgres, migrations via Migrate(), /health poll
  4. Migration logging — confirms Database provider: Postgres — applying migrations in stdout
  5. EPC entity import — live USAC dataset 7i5i-83qf (supplemental entity info, ~50–100k rows)
  6. Idempotency rerun — second POST to /import/usac, asserts recordsInserted == 0
  7. Read-path smoke — Dashboard, Search, Search+filter, Analytics all return HTTP 200
  8. Row count spot-checkpsql COUNT(*) on Entities table

Execution (on dude-mcp-01)

# One-time: clone or pull this branch
git clone https://github.com/steven-dracker/erate-workbench-poc.git ~/erate-workbench
cd ~/erate-workbench
git checkout feature/postgres-import-validation

# Option A — Keeper automatic
KEEPER_RECORD=<record-uid-or-title> bash scripts/validate-postgres-import.sh

# Option B — password env var (if Keeper record name is unknown)
ERATE_PG_PASSWORD=<pw> bash scripts/validate-postgres-import.sh

# Option C — interactive prompt (no args)
bash scripts/validate-postgres-import.sh

Blocker note

SSH from dev WSL to dude-mcp-01 is not authorized (WSL id_ed25519 not in dude-mcp-01's authorized_keys). This is a security boundary — the script must be run on-node. VS Code Remote is configured for the user's primary machine and is the intended execution path.

Build

  • 0 errors, 0 warnings (beyond pre-existing TD-013)
  • Self-contained linux-x64 artifact verified locally

🤖 Generated with Claude Code

steven-dracker and others added 5 commits March 29, 2026 16:34
Introduces DatabaseProvider config key ("Sqlite" | "Postgres") alongside
Npgsql.EntityFrameworkCore.PostgreSQL 8.0.11. Startup branches on provider:
SQLite continues to use Migrate(); Postgres uses EnsureCreated() to bypass
SQLite-specific PRAGMA migrations. Provider name is logged at startup.
Default remains SQLite — no behaviour change unless DatabaseProvider=Postgres.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Guard SQLite-specific PRAGMA journal_mode=WAL in migration
  20260315000001 with ActiveProvider.Contains("Sqlite") check
- Add Npgsql.EntityFrameworkCore.PostgreSQL 8.0.11 to Infrastructure
  project for migration design-time support
- Remove EnsureCreated() from startup — both providers now use Migrate()
- Update startup log: "Database provider: {Provider} — applying migrations"

SQLite path: unchanged, Migrate() runs as before.
Postgres path: Migrate() runs cleanly without hitting PRAGMA.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…p-01

Adds scripts/validate-postgres-import.sh — a self-contained end-to-end
validation script to run on dude-mcp-01:
  - Keeper Commander credential retrieval (with ERATE_PG_PASSWORD override)
  - App startup against local Postgres with DatabaseProvider=Postgres
  - Migration application verification
  - EPC entity import from live USAC dataset (7i5i-83qf)
  - Idempotency rerun with duplicate-row assertion
  - Dashboard / Search / Analytics read-path smoke
  - psql row-count spot-check
  - Structured pass/fail summary

SSH access to dude-mcp-01 not available from dev WSL (key not in
authorized_keys). Script is designed to run from dude-mcp-01 directly
after git clone/pull of this branch.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ASPNETCORE_URLS was being overridden by launchSettings.json (port 5075)
when the app started via `dotnet run`. Adding --no-launch-profile causes
ASPNETCORE_URLS=http://localhost:5080 to be respected, matching the
health-check poll target in the script.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…validation script

Three bugs fixed:
- ImportJob.Status is an int (2=Succeeded), not the string "Completed"
- ImportJob has no recordsInserted field — idempotency now proven via
  psql COUNT(*) before/after rerun (count must be stable)
- Row count spot-check was querying "Entities" table; correct table
  for EPC entity import is "EpcEntities"

Also adds explicit "Row write confirmed: N records" assertion to
fail fast if the import completed but wrote zero rows.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant