You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Engine: CODEC v3.2 — 400 features · 76 skills · 2000+ tests · 52K+ lines of production code · 9 products
The product name is Sovereign AI Workstation. Throughout this document
and the codebase, CODEC refers to the underlying open-source engine /
codename (visible in codec_* Python modules, PM2 process names, and the
~/.codec/ config directory). Sub-products keep their established names —
CODEC Core, CODEC Dictate, CODEC Chat, CODEC Vibe, etc. — because
those are concrete engine components rather than separate products.
v2.3 adds Phase 1 (audit + plugin substrate), Phase 2 (continuous
observation + automation), Phase 3 (drop-a-project autonomous agents),
Phase 3.5 (UX polish + proactive overlay), and CODEC Pilot (the 8th product
— browser automation you can teach) — sections 10–14 below.
Phase 3 ships codec-agent-runner, the autonomous-agent daemon that makes
CODEC a "real AI employee" at the substrate level — drop a project, agent
plans + builds + sends updates back proactively, with permission gates
and resume-after-restart guarantees throughout.
CODEC Pilot ships the 8th product slot: a dedicated headless Chromium driven
by Qwen, record-by-doing teach mode, deterministic XPath → CSS → LLM-rescue
replay, and an approval gate that protects the SkillRegistry from
prompt-injection-spawned skills. With Project promoted to product #9 in this
release, CODEC is now a 9-product system.
1. CODEC Core — The Command Layer (26 features)
#
Feature
1
Push-to-talk via configurable hotkeys (F13/F18/F16)
2
Wake word detection ("Hey CODEC" + 6 configurable phrases)
3
Wake energy auto-clamping (50-1500 range with auto-warn on misconfiguration)
The foundation that Phase 2 + 3 reuse. Audit envelope, plugin lifecycle hooks,
blocking ask-user with strict-consent, stuck-loop detection, per-checkpoint
step budget, self_improve as plugin.
#
Feature
1
Unified audit envelope (schema:1) — every event JSON-line: ts, event, source, outcome, transport, correlation_id, extra
2
Paired correlation_id contract — multi-emit operations share one cid (Step 1 §1.4) so analytics can join op_started → op_completed chains
RingBuffer — last 10 minutes of observation snapshots, RAM-only, no disk persistence
3
Observation injection contract (Q5 override) — always inject for transport=local; cloud transports gate on possessive pronoun OR continuation phrase OR SKILL_NEEDS_OBSERVATION flag
4
OCR-with-retry-once (slow-poll degraded path) when screencapture is slow
5
ocr_enabled config flag — bypasses macOS Screen Recording prompts when permission not yet granted to python3.13 + PM2 parent (default false until explicitly granted)
6
Image redaction — never logs raw pixels
7
Stop-noun list filters trivial captures from observation summaries
8
Observation cardinality control — one observation_tick per 5s
9
Slow-poll degraded mode emits observation_tick_slow (graceful when OCR is disabled or slow)
10
/api/observer/buffer?debug=1 PWA debug endpoint
11
Forward-compat snapshot schema reserves keys for Step 6 + 7
The flagship Phase 3 feature. Drop a project description → Qwen-3.6 drafts a
plan with explicit permission manifest → user approves once → codec-agent-runner
executes autonomously with permission-gated skill loops, plan-hash tamper detection,
resume-after-restart, and proactive update messages back to chat.
Qwen-3.6 plan drafter (local-only, no cloud fallback per Q1) — structured-JSON prompt, validates skills against codec_skill_registry, rejects unknown skills hard
3
Vague-description clarifying loop (Q3) — up to 3 rounds of codec_ask_user.ask clarifying questions before draft fails with description_too_vague
4
Plan-hash tamper detection (Q13) — manifest.plan_hash = sha256(canonical plan.json) computed at approval, verified by Step 9 daemon every tick
5
Global allowlist tier (Q4) — ~/.codec/agent_global_grants.json with 4 grant kinds (network_domains / read_paths / write_paths / skills); items in global → marked auto_approved in per-agent grants
6
Plan revision flow — user edits inline, agent re-validates, flips awaiting_approval → revised → awaiting_approval
7
State machine — draft_pending → awaiting_approval → approved/rejected/revised (Step 9 extends with runtime states)
8
Pre-approval re-validation — checks skills still exist in registry between draft + approve (handles deleted skills)
9
9 PWA endpoints — POST /api/agents create+draft, GET /api/agents list, GET /api/agents/{id} detail, POST /approve/reject/revise, GET/POST/DELETE /api/agent_global_grants
permission_gate(action, agent_grants, global_grants) — UNION of grants enforced as skill / write_path / network_domain / read_path matrix; raises PermissionViolation on any gap
14
Step 3 §1.7 strict-consent gate as universal floor — destructive ops STILL hit consent (verb-match) even if pre-approved
15
Per-checkpoint _execute_checkpoint loop — Qwen → permission_gate → strict_consent (if destructive) → run_skill (Step 1+2 hooks fire) → append history → repeat until checkpoint_done OR step_budget cap
16
Resume after PM2 restart (Q5) — daemon scans status=running agents on boot, marks crashed_resumed, respawns from last atomic checkpoint save
17
Multi-agent concurrency (Q6, Q8) — MAX_CONCURRENT=3 (env var AGENT_RUNNER_MAX_CONCURRENT); blocked agents occupy a slot
18
Per-agent thread inside daemon, atomic state writes after each operation (resume guarantee)
19
StepBudgetExhausted → paused (review I2) — agent paused with reason, user resumes via POST /api/agents/{id}/extend_budget {additional_steps} (overrides stored in state.json, plan stays immutable)
post_message dispatch — writes to ~/.codec/agents/{id}/messages.jsonl (append-only timeline) AND ~/.codec/notifications.json (banner)
25
60s batching window (Q10) — multiple agent_update messages within window merge into ONE banner (count incremented, latest body wins); timeline preserves all entries 1:1
User reply pickup — POST /api/agents/{id}/messages writes type=user_reply; daemon picks up between checkpoints, feeds into next Qwen call as additional context
28
Per-agent silence kill-switch — POST /api/agents/{id}/silence, persists at ~/.codec/agent_silence.json; silenced = timeline written, notifications skipped (no badge spam)
29
Auto-escalation classifier — Qwen-3.6 driven 2-signal gate (verdict is_project=True AND estimated_checkpoints ≥ 3) on every chat-mode message
30
Q11 session silence — first "No" to "Promote to Project mode?" silences for that session (in-memory _autoescalate_silence_set, mutex-guarded), resets on new session
31
3 PWA endpoints (Step 10) — GET /api/agents/{id}/messages, POST /api/agents/{id}/messages, POST /api/agents/{id}/silence
manifest.project_dir field — full absolute path stored at creation; surfaced in POST /api/agents response and chat composer callout
10
Plan-drafter prompt extension — Qwen told the project_dir; defaults permission_manifest.write_paths to <project_dir>/** so files land where the user can open them. Override via env CODEC_PROJECT_ROOT_DIR or ~/.codec/config.json:agents.project_root_dir
Proactive intelligence overlay (4 features) — opt-in only
#
Feature
11
codec_proactive.py — observer-driven contextual nudges. OFF by default (PROACTIVE_OVERLAY_ENABLED=false). User opts in.
12
long_form_dwell v1 pattern — fires when active window is on Notion / Google Docs / Substack / Medium / NYTimes / FT / Economist / NewYorker for ≥30 min consecutively. Posts: "Want me to summarize?" with [Acknowledge / Dismiss today / Disable forever] buttons
13
3-gate kill model — global env switch + per-pattern killed-forever (~/.codec/proactive_state.json:killed_patterns) + per-day dismissed (resets next UTC midnight)
14
Rate limits — per-pattern cooldown 1 hour + global 30-min between any two suggestions (prevents pattern-cluster burst)
Step 9 review polish (5 features)
#
Feature
15
blocked_on_qwen dedicated status (review C2) — distinct from blocked_on_permission (no permission to grant; service is just down)
16
Daemon auto-resume on Qwen recovery — when daemon ticks an agent in blocked_on_qwen, probes Qwen with a 1-token call; if alive → transitions to running and respawns. No user click needed
17
Action.reads_path + read_path fields (review M4) — symmetric read/write gating; permission_gate now checks read_path against read_paths UNION
18
Symmetric ~ expansion — permission_gate expands tilde on BOTH the action path AND the grant glob, so ~/Documents/foo matches grant ~/Documents/**
19
recovery_cid threading — daemon's crash-recovery AGENT_RESUMED emit shares correlation_id with the resumed _run_agent's subsequent emit chain (review I4)
Multi-channel notifications (3 features)
#
Feature
20
macos channel — osascript display notification banner. Works out of the box (no setup)
21
imessage channel — reuses skills/imessage_send._send; recipient read from ~/.codec/config.json:notifications.imessage_recipient. Skipped silently if unconfigured
22
telegram channel — direct Bot API call (no daemon coupling). Reads notifications.telegram_token + :telegram_chat_id. Skipped silently if unconfigured
Phase 3 review fast-follow (2 features)
#
Feature
23
POST /api/agents/{id}/extend_budget — bumps current checkpoint's step_budget via state.json:step_budget_overrides[checkpoint_id] (plan stays immutable, plan_hash check intact); transitions paused → running
14. CODEC Pilot — Browser Automation You Can Teach (32 features) (v2.3)
The 8th product — a complete browser-automation pillar with a dedicated headless Chromium, ReAct-style agent loop driven by Qwen, deterministic record-replay with selector fallback, a skill approval gate, and human-in-the-loop takeover. Lives in ~/codec/pilot/ (11 modules), runs as pilot-runner on PM2 port 8094.
Browser substrate (5 features)
#
Feature
1
Dedicated headless Chromium on CDP port 9223 (separate from user's daily Chrome on 9222) — never interferes with user's browsing
2
Persistent profile at ~/.codec/pilot_chrome_profile/ — cookies, sessions, login state survive restarts
ReplayResult.to_dict() — status, methods used per step, rescues_used, durations — full audit trail per replay
Skill approval gate (3 features)
#
Feature
27
~/.codec/skills/.pending/ directory — compiled skills do NOT auto-register; landing zone for human review. Blocks prompt-injection-spawned auto-registration
28
Dashboard preview — GET /skills/pending/{slug} returns full Python source; one-click ✓ Approve moves to ~/.codec/skills/, ✕ Reject deletes
29
slugify() + collision suffix — filesystem-safe names from free-form task descriptions, _2/_3/… suffix appended on duplicate slugs
HITL (human-in-the-loop) takeover (3 features)
#
Feature
30
HitlController.pause/resume/inject — agent loop checks an asyncio.Event every step; human can pause, push manual actions to a queue, resume
31
takeover()/handback() — full human control mid-run; agent re-snapshots and continues from wherever it was left
32
HITL HTTP endpoints — `/hitl/{run_id}/pause
Live view + infrastructure (3 features bundled)
MJPEG live stream at /screenshot/stream — ~3 fps multipart feed, ~350 KB/s; falls back to 2-second polling on disconnect
30 HTTP endpoints on pilot-runner (FastAPI, port 8094) with CORS enabled for codec.lucyvpa.com cross-origin calls
Cloudflare tunnel at pilot.lucyvpa.com for off-LAN dashboard access; PM2 service pilot-runner with autorestart + isolated log files
14. CODEC Pilot — Browser Automation You Can Teach (v2.3)
32
TOTAL
400
400 features · 76 skills · 2000+ tests · 52K+ lines of production code · 9 products
What's new in v3.2
The self-updating release. CODEC now updates itself — and a long-standing bundle-version-drift bug in the build pipeline is fixed at the source.
In-app auto-update (Sparkle-compatible) — pure-Python client (codec_update.py) reads a signed Sparkle appcast and verifies every download's Ed25519 signature against the embedded SUPublicEDKey before installing. Tampered builds are refused. Dashboard endpoints GET /api/update/check + POST /api/update/download; in-app update banner ("Download & open") polls on load and every 6h.
GitHub-hosted update feed — each release (signed .dmg + appcast) lives at AVADSA25/codec-updates; the app polls a permanent releases/latest/download/appcast.xml URL that always resolves to the newest version. Host is a one-line switch (sparkle_feed_url / CODEC_APPCAST_PREFIX) for a future move to a custom domain or Cloudflare R2.
Bundle version stamped from VERSION at build time — build_app.sh injects the F-5 single source of truth into CFBundleShortVersionString so the bundle version can never drift from the source again. Guarded by a regression test.
Gatekeeper signing fix — bundled Python now lives under Contents/Resources/python instead of Frameworks/ (a bare Python tree under Frameworks/ was treated as a nested bundle and broke the code-signing seal). The DMG is notarized + stapled so it opens cleanly on a never-online Mac.
Unified :8083 model port — all Qwen MLX endpoints reconciled (was split :8081 / :8082); UI-TARS vision served from the same unified server.
What's new in v3.1
The 9th product + adversarial-hardening release. CODEC Project promoted to a live product, Pilot security-hardened end-to-end, calmer 2026 palette across the dashboard.
9 products. One system. Cortex neural map expanded from 28 → 38 nodes (+31 edges); CODEC Pilot and CODEC Project added as live product cards.
Pilot security-hardening wave (PP-1…PP-12) — full adversarial-audit remediation: AST safety gate at skill-approval time (refuses to activate dangerous compiled skills), untrusted-input fencing on the LLM selector-rescue prompt (page text can't redirect element selection), irreversible-click blocking on replay unless PILOT_ALLOW_DESTRUCTIVE=1, path/glob-traversal neutralization in slugify() lookups, forensic audit trail on every skill write/approve/reject/block. The pilot.lucyvpa.com Cloudflare tunnel was removed after the RCE finding — Pilot is local-only.
Conversational continuity in Project mode — a Project-mode chat thread now binds to the agent it drafts; follow-ups route to the running agent instead of spawning duplicates. Pulsing "Talking to …" chip with one-click exit.
Voice in / voice out — Kokoro TTS now actually speaks assistant replies when "Voice Replies" is on (the toggle was previously inert); per-message Speak button; Pilot tab gets its own 🎤 dictate + 🔊 speak controls.
Live preview panel — slide-out shows the most-recently-modified files in a running agent's project folder (5s poll), so you can watch output appear without leaving the chat.
Auto-grant of user-typed paths — paths the user typed themselves (~/…, $HOME/…, /Users/<n>/…) no longer trigger mid-run blocked_on_permission; sensitive paths (~/.ssh, ~/.aws, /etc, …) stay blocklisted.
2026 palette refresh — calmer brand accent (#d97757 dark / #b85a3a light) and a muted Tailwind-500 node palette across the Cortex map. Same identity, far less "Windows-XP-bright."
F-wave engineering hygiene — ruff baseline + CI lint gate (F-4), VERSION single source of truth + CHANGELOG-driven tag helper (F-5), pyproject.toml packaging metadata (F-15), pricing docs (F-11), GitHub Discussions (F-12), lucy→delegate skill rename (F-6).
What's new in v2.3 — Phase 1 + 2 + 3 + 3.5
The drop-a-project release. CODEC becomes a "real AI employee" at the substrate level — drop a project description, agent plans + builds + sends updates back proactively, with permission gates and resume-after-restart guarantees.