feat(cli): add push/pull/list/create commands for publishing and managing CALM Hub content

[jimthompson5802]

Feature Proposal

Target Project:

cli (@finos/calm-cli) — the CALM command-line tool that currently supports generate, validate, template, docify, and init-ai commands.

Description of Feature:

Extend the calm CLI with push, pull, list, and create commands that allow users to publish and retrieve CALM documents (architectures, patterns, standards, interfaces) and manage organisational resources (namespaces, domains, controls, decorators) against a CALM Hub instance.

This closes the read/write gap: the CLI can already fetch schemas and patterns from a Hub via the --calm-hub-url flag, but has no way to publish new documents, retrieve a specific versioned document, or manage the Hub's organisational resources (namespaces, domains, controls, decorators) from the command line.

User Stories:

• As a CALM architect, I want to run calm push architecture my-arch.json --namespace my-org so that I can publish a new architecture to our shared CALM Hub without writing raw HTTP calls.
• As a developer, I want to run calm pull architecture my-arch --version 1.2.0 --out my-arch.json so that I can fetch a specific versioned architecture from the Hub and work on it locally.
• As a CI/CD pipeline author, I want to automate CALM Hub publishing as part of a build step using the CLI, so that approved architecture documents are published without manual intervention.
• As a compliance officer, I want to run calm push standard my-standard.json --namespace my-org so that I can publish governance standards to the Hub for teams to reference.
• As a developer, I want to run calm pull standard my-standard --version 1.0.0 --out my-standard.json so that I can fetch the exact version of a standard my architecture must comply with.
• As a platform team member, I want to list available architectures in a namespace (calm list architectures --namespace my-org) so that I can discover what is already published.
• As a Hub administrator, I want to run calm create namespace --name my-org --description "My org namespace" so that I can provision a new namespace without using the Swagger UI or raw HTTP.
• As a Hub administrator, I want to run calm list namespaces so that I can see all provisioned namespaces in a Hub instance.
• As a Hub administrator, I want to run calm create domain --name payments and calm list domains so that I can manage top-level domain classifications from the CLI.
• As a CALM architect, I want to run calm push interface my-interface.json --namespace my-org and calm pull interface my-interface --version 1.0.0 so that I can publish and retrieve reusable interface definitions from the Hub.
• As a compliance officer, I want to run calm create control --domain payments --name PCI-DSS and calm list controls --domain payments so that I can manage control requirements directly from the CLI.
• As a platform team member, I want to run calm create decorator my-decorator.json --namespace my-org and calm list decorators --namespace my-org so that I can attach and query metadata decorators on Hub resources without using the Swagger UI.

Current Limitations:

• The CLI only supports read access to CALM Hub. The generate and validate commands accept --calm-hub-url to resolve calm:// URIs via CalmHubDocumentLoader (in shared/), but this only performs HTTP GET requests to fetch schemas/patterns.
• There are no publish, push, pull, list, or create commands in the CLI.
• The CLI has no authentication/token handling. CALM Hub enforces OAuth scopes (via @PermittedScopes) on mutating endpoints (POST/PUT), so publishing requires an authenticated request.
• Users must interact with the Hub REST API directly (e.g., via curl or Swagger UI) to publish documents (including standards and interfaces), and to manage namespaces, domains, controls, and decorators.
• Namespaces (POST /calm/namespaces, GET /calm/namespaces) and domains (POST /calm/domains, GET /calm/domains) are top-level Hub resources with no CLI representation today.
• Interfaces share the same versioned CRUD shape as architectures/patterns/standards but have no CLI push/pull/list support.
• Controls (nested under domains: POST/GET /calm/domains/{domain}/controls) and their versioned requirements and configurations have no CLI representation.
• Decorators (POST/GET/PUT /calm/namespaces/{ns}/decorators) — filterable metadata attachments — have no CLI representation.

Proposed Implementation:

New commands:

Command	HTTP operation	Notes
calm push architecture <file> --namespace <ns>	POST /calm/namespaces/{ns}/architectures	Create new; POST …/{id}/versions/{v} for versioned update
calm pull architecture <id> --namespace <ns> --version <version>	GET /calm/namespaces/{ns}/architectures/{id}/versions/{version}	Write to stdout (or --out <file>)
calm list architectures --namespace <ns>	GET /calm/namespaces/{ns}/architectures	Print IDs/versions
calm create control --domain <domain> --name <name>	POST /calm/domains/{domain}/controls	409 shown as friendly error
calm list controls --domain <domain>	GET /calm/domains/{domain}/controls	Print control IDs/names
calm push control-requirement <file> --domain <domain> --control-id <id> --version <version>	POST /calm/domains/{domain}/controls/{id}/requirement/versions/{version}	Publish a requirement version for a control
calm pull control-requirement --domain <domain> --control-id <id> --version <version>	GET /calm/domains/{domain}/controls/{id}/requirement/versions/{version}	Write to stdout (or --out <file>)
calm push control-config <file> --domain <domain> --control-id <control-id> --config-id <config-id> --version <version>	POST /calm/domains/{domain}/controls/{controlId}/configurations/{configId}/versions/{version}	Create or version a control configuration
calm pull control-config --domain <domain> --control-id <control-id> --config-id <config-id> --version <version>	GET /calm/domains/{domain}/controls/{controlId}/configurations/{configId}/versions/{version}	Write to stdout (or --out <file>)
calm create decorator <file> --namespace <ns>	POST /calm/namespaces/{ns}/decorators	JSON body read from <file>
calm list decorators --namespace <ns> [--target <target>] [--type <type>]	GET /calm/namespaces/{ns}/decorators	Optional --target, --type query filters
calm pull decorator <id> --namespace <ns>	GET /calm/namespaces/{ns}/decorators/{id}	No --version flag (decorators are not versioned)
calm create domain --name <name>	POST /calm/domains	409 shown as friendly error
calm list domains	GET /calm/domains	Print domain names
calm push interface <file> --namespace <ns>	POST /calm/namespaces/{ns}/interfaces	Same versioned shape as architectures
calm pull interface <id> --namespace <ns> --version <version>	GET /calm/namespaces/{ns}/interfaces/{id}/versions/{version}	Write to stdout (or --out <file>)
calm list interfaces --namespace <ns>	GET /calm/namespaces/{ns}/interfaces	Print IDs/versions
calm create namespace --name <name> --description <description>	POST /calm/namespaces	409 shown as friendly error
calm list namespaces	GET /calm/namespaces	Print name + description
calm push pattern <file> --namespace <ns>	POST /calm/namespaces/{ns}/patterns	Same versioned shape as architectures
calm pull pattern <id> --namespace <ns> --version <version>	GET /calm/namespaces/{ns}/patterns/{id}/versions/{version}	Write to stdout (or --out <file>)
calm list patterns --namespace <ns>	GET /calm/namespaces/{ns}/patterns	Print IDs/versions
calm push standard <file> --namespace <ns>	POST /calm/namespaces/{ns}/standards	{ name, description, standardJson }; POST …/{id}/versions/{v} for versioned update
calm pull standard <id> --namespace <ns> --version <version>	GET /calm/namespaces/{ns}/standards/{id}/versions/{version}	Write to stdout (or --out <file>)
calm list standards --namespace <ns>	GET /calm/namespaces/{ns}/standards	Print name + description + ID

Output format:

• Default output for all commands is JSON to stdout. Intended for scripting and CI/CD pipelines.

• --output table flag (shorthand -o table) renders results as a human-readable table. Intended for interactive terminal inspection.

• list commands default to a JSON array of the raw Hub API response values.

• pull commands write raw, pretty-printed JSON to stdout by default. When --out <file> is provided the raw JSON is written to disk with no additional formatting. The --output flag has no effect on pull (output is always JSON).

• push and create success output defaults to a JSON object. For versioned resources: { "id": <id>, "version": "<semver>", "location": "<Location URL>" }. For non-versioned resources (namespace, domain, control, decorator): { "id": <id>, "location": "<Location URL>" }. With --output table, a single-row confirmation is rendered:

  STATUS   ID   VERSION   LOCATION
  Created  42   1.0.0     /calm/namespaces/my-org/architectures/42/versions/1.0.0
  

• Error output always goes to stderr as a JSON object: { "status": 409, "error": "namespace 'my-org' already exists", "request": "POST /calm/namespaces" }. With --output table, errors print as plain text to stderr instead.

• Table columns for list commands (when --output table is used):

  Command	Columns
  calm list architectures / patterns / interfaces	ID, NAME, VERSIONS
  calm list standards	ID, NAME, DESCRIPTION, VERSIONS
  calm list controls	ID, NAME
  calm list decorators	ID, TARGET, TYPE
  calm list namespaces	NAME, DESCRIPTION
  calm list domains	NAME

Exit codes:

Code	Meaning
0	Command completed successfully
1	Command failed — covers all error conditions including: invalid flag combinations, authentication/authorisation errors (401/403), resource not found (404), conflict (409), Hub validation rejection (400), network errors, and unhandled exceptions

Errors are distinguished by the JSON error object written to stderr ({ "status": <code>, "error": "<message>", "request": "<method> <path>" }), not by the exit code. Scripts should parse stderr for the HTTP status to branch on specific failure conditions.

Technical design considerations:

• Reuse the existing --calm-hub-url / ~/.calm.json configuration mechanism for the Hub base URL.
• Add --namespace (default: default) flag for all namespace-scoped commands. Control commands use --domain instead; the --namespace default does not apply to them.
• Introduce an --auth-token flag (or read from CALM_HUB_TOKEN env var / ~/.calm.json) passed as Authorization: Bearer <token> header for mutating operations.
• Extend CalmHubDocumentLoader in shared/ (or introduce a CalmHubClient class) to support POST/GET for all versioned namespace resource types (architectures, patterns, standards, interfaces) and domain-scoped resources (controls with requirements and configurations) plus decorators.
• calm push should auto-detect the resource type from the document's $schema field when not specified explicitly.
• calm pull should default to writing to stdout; an --out <file> flag writes to disk.
• calm create namespace, calm create domain, calm create control, and calm create decorator follow the same auth pattern as push (Bearer token required for mutating operations).
• calm list namespaces, calm list domains, calm list controls, and calm list decorators are read-only and require no auth token.
• Decorator list supports optional --target and --type query filters mapping to ?target= and ?type= on GET /calm/namespaces/{ns}/decorators.
• Decorator pull retrieves a single decorator by ID; decorators are not versioned so no --version flag is needed.
• Controls are domain-scoped (--domain flag), not namespace-scoped; requirements and configurations are further nested under a control ID.

API changes:

• New top-level Commander.js command groups push, pull, list, and create in cli/src/cli.ts.
• New CalmHubClient (or extension of CalmHubDocumentLoader) in shared/src/ covering: versioned namespace resources (architectures, patterns, standards, interfaces); domain-scoped controls with createControl, listControls, publishRequirementVersion, getRequirementVersion, createControlConfig, getControlConfig; decorators with createDecorator, listDecorators, pullDecorator; and organisational resources createNamespace, listNamespaces, createDomain, listDomains.

Data model changes:

• No changes to CALM schemas required.

Dependencies:

• CALM Hub must be running and accessible (existing dependency).
• OAuth token must be obtained out-of-band (or a future calm login command can be tracked separately).

Alternatives Considered:

• Separate calm hub sub-command group — cleaner namespacing but adds one extra word to every invocation. Deferred to a follow-up refactor if the command surface grows.
• calm publish / calm retrieve — longer verb names, inconsistent with common CLI conventions (push/pull align with git/container-registry mental models).
• Not implementing list — could have been omitted in a minimal scope, but included because discoverability is necessary for pull to be useful without knowing exact IDs.
• Omitting namespace/domain management — considered out of scope, but including them avoids requiring Hub administrators to use the Swagger UI for basic setup tasks; the HTTP surface is simple (one POST, one GET each).
• Omitting interfaces — same CRUD shape as architectures, so implementation cost is minimal; omitting them would create a confusing inconsistency in the CLI surface.
• Omitting decorators — the --target/--type filter flags add slight complexity, but decorators are a core metadata mechanism and their absence would force ongoing manual API calls.
• Omitting controls — controls are more complex (nested requirements + configurations), but they are the primary way governance is expressed in CALM Hub and should be accessible from CI/CD pipelines.

Testing Strategy:

• Unit tests (vitest): mock CalmHubClient HTTP calls; test command flag parsing, file I/O, and error handling (4xx/5xx responses, missing auth token, invalid namespace, 409 conflict on duplicate namespace/domain/control).
• Integration tests: extend existing CLI integration test fixtures in cli/test_fixtures/; spin up a local CALM Hub (NitriteDB mode, no Docker required) and exercise full round-trips: create namespace → push → list → pull for each namespace-scoped resource type (architecture, pattern, standard, interface); create domain → create control → push requirement → pull requirement; create decorator → list decorators with filters.
• Auth error case: verify CLI writes a JSON error object to stderr with status: 401 or status: 403 when an invalid or missing token is provided.
• Conflict case: verify calm create namespace, calm create domain, and calm create control write a JSON error object to stderr with status: 409 on duplicate resource creation.
• Filter test: verify calm list decorators --target my-node --type tag correctly passes query parameters and filters results.

Documentation Requirements:

• Update cli/README.md with push, pull, list, and create command reference sections.
• Update docs/ (Docusaurus site) CLI reference page.
• Add a tutorial page: "Publishing and retrieving CALM documents with the CLI."
• Add a tutorial section: "Managing namespaces and domains from the CLI."
• Add a tutorial section: "Managing controls and governance requirements from the CLI."
• Add a tutorial section: "Working with decorators from the CLI."
• Update ~/.calm.json schema documentation to include authToken field.

Implementation Checklist:

• Design reviewed and approved
• Implementation completed
• Tests written and passing
• Documentation updated
• Relevant workflows updated (if needed)
• Performance impact assessed

Additional Context:

• Versioned namespace resources (architectures, patterns, standards, interfaces) all share the same CRUD shape under /calm/namespaces/{namespace}/{resource-type}. Standards are the exception: they use { name, description, standardJson } where standardJson is the raw content as a JSON string. Architectures, patterns, and interfaces use the document JSON directly as the request body. All versions are semver-validated.
• Controls are domain-scoped (/calm/domains/{domain}/controls). Each control has independently versioned requirements (/controls/{id}/requirement/versions/{v}) and configurations (/controls/{id}/configurations/{configId}/versions/{v}).
• Decorators live at /calm/namespaces/{ns}/decorators. GET supports ?target= and ?type= filters. A separate GET /decorators/values endpoint returns full decorator payloads (vs. IDs only) — this endpoint is not exposed as a CLI command in this proposal (see Out of Scope).
• Namespace API: POST /calm/namespaces accepts { name, description }, returns 201 + Location or 409.
• Domain API: GET/POST /calm/domains — top-level, not namespace-scoped.
• The calm:// URI scheme used in generate/validate for reading can be extended or complemented by the new pull command.
• Authentication is currently OAuth-based in CALM Hub (@PermittedScopes). A dedicated calm login command (storing tokens in ~/.calm.json) is a natural follow-on issue.
• Related existing code: shared/src/document-loader/calm-hub-document-loader.ts, cli/src/cli.ts.

Out of Scope:

The following CALM Hub endpoints are intentionally excluded from this proposal. Each is a candidate for a separate follow-on issue.

Endpoint(s)	Reason deferred
ADRs — POST/GET /calm/namespaces/{ns}/adrs, revision management, status transitions	ADRs use a revision model (integer, not semver) and have a dedicated status lifecycle (DRAFT → ACCEPTED → etc.); the UX requires separate design consideration
User Access — POST/GET /calm/namespaces/{ns}/user-access	Requires NAMESPACE_ADMIN scope; user/permission management is a distinct operational concern better addressed in a dedicated calm access command proposal
Core Schemas — GET/POST /calm/schemas, GET /calm/schemas/{v}/meta, GET /calm/schemas/{v}/meta/{name}	Schema publication is a maintainer/release-time operation, not a day-to-day developer workflow; warrants its own proposal
FrontController (slug API) — POST/GET /calm/namespaces/{ns}/{customId} and versioned variants	The type-agnostic slug-based API overlaps with the typed push/pull commands; should be evaluated after the typed commands are stable to avoid conflicting UX
List versions sub-command — GET …/{id}/versions for all versioned resources	Low priority; version discovery requires a separate calm list <resource> --versions <id> sub-option; can be added as a minor enhancement
GET /decorators/values — full decorator payload list for a namespace	calm list decorators already returns IDs; the values endpoint is a convenience optimisation that can be added as a minor enhancement
Flows — POST/GET /calm/namespaces/{ns}/flows, GET …/flows/{id} (latest), versioned CRUD	Flows are deprecated as a standalone Hub resource; flow data is part of an architecture document. Existing flow endpoints in CALM Hub remain for backward compatibility but new CLI commands should not encourage their continued use.
Update (PUT) operations — PUT …/{id}/versions/{v} for architectures, patterns; PUT /decorators/{id}	Guarded by the allow.put.operations server flag; mutating published versions is a destructive operation that needs explicit opt-in design in the CLI

[jimthompson5802]

@markscott-ms @rocketstack-matt @willosborne @jpgough-ms @opsflowanoop per #2334 any feedback will be appreciated. I'm hoping we can get consensus on this at next Office Hours.

Since I used Github Copilot to help me draft this issue it made a recommendation, which I had not thought of, in the Technical Design and API Changes sections. The recommendation for the CalmHubClient module or extension of CalmHubDocumentLoader. As I understand the CalmHubDocumentLoader its function is to retrieve documents from Calm Hub, basically read-only.

I'm thinking the new CalmHubClient is probably better for this issue where it support both read and write functions. If this is the case, then the question becomes, should CalmHubDocumentLoader function be rolled into CalmHubClient and CalmHubDocumentLoader be retired.

Another part to pay attention is the Output Format section. The proposal is the primary output format is JSON to facilitate automation. There is an option for some of the operations to make the output human readable. Any thoughts here?

From a security perspective any thoughts on the --auth-token flag? I'm using this as an indicator for the cli to retrieve the auth token from either from ~/.calm.json or from an environment variable. The alterntive is not have the --auth-token flag and the cli always checks for whether an auth token is present in ~/.calm.json or as an environment variable. If it is present use it to populate the Bearer token. Otherwise assume no authentication is needed, such as in local testing.

[willosborne]

This looks generally really good, but a couple qs from me:

• why the distinction push for some resources vs pull for others? This might be confusing for users.
• do you think it would be worth grouping all these commands together in a fashion similar to calm workspace so the CLI doesn't get too bloated at the top level?

In response to your other points:

• JSON as default output makes sense. Look at calm validate for reference - you can pick an output format for that. We should replicate that pattern where it makes sense, IMO.
• Regarding auth token / authentication on the CLI, @markscott-ms and I have been doing some thinking on the best way to do this. We have an idea, which Mark has already proven in a slightly simplified form, to have a sort of 'plugin' that gets a token for you and handles any caching. That means the CLI itself doesn't need to worry about the authentication mechanism or storing any tokens - this will probably vary between every organisation. I suggest we proceed with this with no authentication for now.

[jimthompson5802]

@willosborne Thanks for the comments.

re: push vs pull: push should map to the POST api call and pull to the GET. I'll take a look at the calm workspace to see how I could consolldate or come back with a reason on why that won't work.

I'll talk to you @markscott-ms on the token topic.