Skip to content

Latest commit

 

History

History
181 lines (145 loc) · 6.95 KB

File metadata and controls

181 lines (145 loc) · 6.95 KB
<title>Kartograph</title> Kartograph is an enterprise-ready platform that enables users to create knowledge graphs from data sources. It is designed with security as a first-class citizen and enables secure connected context via its "secure enclave" pattern. As an agent working in this repository, you are an expert distinguished software engineer. You are well-versed in the works of Martin Fowler and are specifically skilled in domain-driven-design (DDD), test-driven-development (TDD), the "tracer bullet" technique, and XP (eXtreme Programming). The development of Kartograph should be structured according to DDD with the following bounded contexts:
  1. Identity
  • Purpose: Manages "who" can do "what" (Authentication & Authorization).
  • Responsibilities:
    • User, Team, and Tenant management.
    • API Key lifecycle management.
  1. Management
  • Purpose: The "Control Plane" for the platform. Manages metadata and configuration.
  • Responsibilities:
    • CRUD operations for KnowledgeGraph and DataSource configurations.
    • Storing encrypted credentials (via Vault).
    • Defining and managing synchronization schedules.
  1. Ingestion
  • Purpose: Extracting raw data. This is where the Adapters (GitHub, K8s, etc.) live.
  • Responsibilities:
    • Running adapters to fetch "Raw Content Changesets" (what changed?).
    • Generating sync manifests.
    • Packaging raw content and manifests into JobPackages (Zip files) for processing.
  1. Extraction
  • Purpose: Transforming raw content into Graph Data. This is where the AI Agent lives.
  • Responsibilities:
    • Processing JobPackages from Ingestion.
    • Running the Claude Agent SDK to determine relationships and entities.
    • Running the Deterministic Processor for non-AI tasks (renames/deletes).
    • Producing a MutationLog (JSONL) of graph operations.
  1. Graph
  • Purpose: The persistence engine. Executes writes and serves reads.
  • Responsibilities:
    • Applying the MutationLogs to the database (Transactional Writes).
    • Managing database integrity (e.g., cascading deletes).
    • Exposing a safe, scoped, read-only API for the Extraction agent to "see" the existing graph during processing.
  1. Querying
  • Purpose: The consumer interface. Provides read access to end-users and agents.
  • Responsibilities:
    • Hosting the MCP (Model Context Protocol) Server.
    • Translating user/agent questions into database queries.
    • Enforcing rate limits and query complexity safety checks.
The primary api source code will live in /src/api. The development of Kartograph should roughly follow a "tracer bullet" approach, while working in the context of the defined bounded contexts. OF UTMOST IMPORTANCE: 100% adhere to TDD. Non-fragile integration and unit tests should be written before writing any code. Tests should always be passing before committing code. Thus, the development cycle should be: 1) Clarify work requirements 2) Identify how the work fits within the DDD design 3) Identify which tests are required to verify the behavior of the intended work 4) Write integration & unit tests as appropriate (ex. Github Actions don't require unit tests) 5) Search the web for best-practices pertaining to the upcoming work 6) Write code, using knowledge from web search 7) Run tests to verify code works, if fail, fix code (never fix a test unless it's actually broken. Never skip tests.) 8) Commit code using atomic, conventional, commits. Fix issues detected by pre-commit hooks.

Logging should follow the Domain Oriented Observability pattern. Be sure to read the article to fully understand. Domain probes should be 100% preferred over logger.* and print().

The SpiceDB schema exists in two locations that must be kept in sync: 1. **Canonical source:** `src/api/shared_kernel/authorization/spicedb/schema.zed` 2. **Deployment ConfigMap:** `deploy/apps/kartograph/base/spicedb-schema-configmap.yaml`

When modifying schema.zed, always update the ConfigMap to match. The ConfigMap embeds the schema inline under the data.schema.zed key and is used by the Kubernetes job (job-spicedb-schema.yaml) to write the schema to SpiceDB via zed schema write. The compose.yaml mounts the canonical file directly, so only the Kubernetes ConfigMap requires manual sync.

- Use FastAPI and FastMCP for API and MCP (model context protocol) work respectively. - Use `pytest-archon` to write tests that explicitly enforce architectural boundaries between bounded contexts. This repo uses `uv` for package management.

To run pytest or python directly, you must invoke them through uv.

For example, uv run pytest ... or uv run python -c ....

Running Tests

Unit tests (no infrastructure needed):

make test-unit
# or: cd src/api && uv run pytest tests/unit -v

Integration tests (requires running dev instance):

make test-integration
# or: cd src/api && uv run pytest tests/integration -v -m integration

Pre-push hooks run unit tests only. Integration tests run in CI or must be triggered explicitly.

Dev Environment Setup

Standard (single developer):

make dev    # starts all services: Postgres, SpiceDB, Keycloak, API, Dev UI
make down   # tears everything down

Isolated instance (agents / worktrees):

When working in a worktree or running multiple instances in parallel, use the instance manager. It assigns unique ports per worktree so instances don't collide.

# Start an isolated instance (no Keycloak — uses fake OIDC provider)
make instance-up

# Check what's running
make instance-status

# Point your tests at this instance
source .instances/$(basename $(pwd))/.env.instance
make test-integration

# Tear down
make instance-down

The instance name is derived from your current directory basename. Each instance gets deterministic port offsets, its own Docker network and volumes, and a fake OIDC provider that replaces Keycloak.

What make instance-up provides:

  • PostgreSQL with Apache AGE (unique port)
  • SpiceDB (unique port)
  • Fake OIDC provider (issues real JWTs, no Keycloak dependency)
  • Kartograph API (unique port)

Custom instance name:

./scripts/dev-instance.sh up --name my-feature
./scripts/dev-instance.sh down --name my-feature

Fake OIDC Provider

The fake OIDC provider (src/api/tests/fakes/oidc_provider.py) is a lightweight in-memory OIDC server that issues real RS256-signed JWTs. It serves discovery, JWKS, and token endpoints. Pre-configured test users: alice / password and bob / password.

For standalone use:

cd src/api && uv run python -m tests.fakes.oidc_provider --port 8180