feat(cmd/dbc): versioned JSON schema, --json on all commands, NDJSON progress

[zeroshade]

Summary

Improves the --json output to be more structured and versioned. All changes are additive — existing human-readable output and exit codes are unchanged.

Changes

internal/jsonschema — versioned schema package

Extracts a new internal/jsonschema package with typed Go structs for every --json payload the CLI emits. All structs carry schema_version: 1 and a kind discriminator.

--json on all commands

• install — refactored to use jsonschema.InstallStatus envelope
• uninstall / search / info — refactored to use jsonschema package
• init / add / remove — new --json flag emitting init.response, add.response, remove.response
• sync — new --json flag emitting NDJSON progress events + final sync.status

NDJSON progress streaming from install --json

When --json is set, dbc install emits one NDJSON line per phase (download.start, download.progress, extract.start, verify.start, etc.) to stdout as they occur, followed by the terminal install.status envelope.

sha256 checksum verification on install

After downloading, dbc install now computes and verifies the sha256 of the tarball. The checksum is included in the install.status JSON payload. --no-verify bypasses both signature and checksum checks; --insecure-no-checksum bypasses only the checksum.

File-locking on mutating commands

Adds internal/fslock (advisory flock/LockFileEx) wrapping install, uninstall, sync, add, remove to prevent concurrent mutations. Returns a clear error with the owner PID if the lock cannot be acquired within 10s.

auth login --json — verification_uri_complete

When --json is set, the device-code flow immediately emits an auth.device_code envelope containing verification_uri_complete (the pre-filled URL) so a consumer can open a browser directly if desired.

Dead code removal

Removes the unused TuiCmd struct and cmd/dbc/view_config.go.

Testing

All existing tests pass. New tests added for every changed command.

go test ./...

[amoeba]

I took a quick look and made some comments in line. Other thoughts:

1. I'm not so sure about install --json streaming. it means the return shape varies based on the state of the system. i.e., run the same command twice and your return shape is different. Could we remove that bit and add it if someone ever asks for it?2. Did you consider making the error responses structured like ${subcommand}.error instead of every error being "kind": "error"?

2. The binary structure of the kind responses seems like it could be two fields. For example, the current output,

$ dbc init --json
{
  "schema_version": 1,
  "kind": "init.response",
  "payload": {
    "driver_list_path": "/Users/bryce/src/columnar-tech/dbc/dbc.toml",
    "created": true
  }
}

could be

{
  "schema_version": 1,
  "command": "init",
  "status": "success",
  "payload": {
    "driver_list_path": "/Users/bryce/src/columnar-tech/dbc/dbc.toml",
    "created": true
  }
}

3. There's an encoding bug here,

$ go run ./cmd/dbc add --json "mysql<1.0.0"
{"schema_version":1,"kind":"add.response","payload":{"driver_list_path":"/Users/bryce/src/columnar-tech/dbc/dbc.toml","drivers":[{"name":"mysql","version_constraint":"\u003c1.0.0"}]}}

4. What made you go with locking rather than atomic writes? Were we vulnerable to concurrent mutations before?

[zeroshade]

I'm not so sure about install --json streaming. it means the return shape varies based on the state of the system. i.e., run the same command twice and your return shape is different. Could we remove that bit and add it if someone ever asks for it?2.

I've been playing around with a GUI being built using Tauri or electron. Either way the install --json streaming is so that the eventual GUI will be able to have a visual progress bar indicator (matching the current progress bar we do in the terminal). Most of the changes and updates here came from that research and development of trying to construct a GUI.

Did you consider making the error responses structured like ${subcommand}.error instead of every error being "kind": "error"?

That wasn't something I thought of since the assumption was that we'd have specific shapes for the kinds and ALL errors would have the same shape, hence kind: "error" as opposed to ${subcommand}.error which would require parsing out the .error to know that it's an error. The other assumption being that the caller knows what arguments they passed and don't need the error to output that. Also, most of the time this is covered in the "code" of the error response which usually has something like "info_failed" or "init_failed" etc.

The binary structure of the kind responses seems like it could be two fields....

I don't think the "status" field is necessary since the kind will either be "error" to indicate an error, or something else indicating success. The format of the kind being something like install.status or add.response is solely to uniquely identify the shape of a particular payload. It's not intended to be a binary structure, beyond helping to associate the payload shapes.

What made you go with locking rather than atomic writes? Were we vulnerable to concurrent mutations before?

Yup, we were vulnerable because all of these paths would open the file first, read it, do something and then write it back out. At any point in there the file could be mutated causing the final write to clobber anything that was modified in between when we opened and read it and when we wrote out the result.

Consider the scenario of running dbc add mysql and dbc add duckdb in the same directory at the same time. Depending on the order of operations, OS scheduling, filesystem I/O scheduling, you could end up with any potential scenario of one or both getting added or even file corruption (since NewEncoder(f).Encode isn't an atomic write). Using atomic writes would eliminate the potential for corrupting the files, but it wouldn't remove the possibility of losing information:

| Process 1 | -> Open File -> Decode TOML into memory
| Process 2 | -> Open File -> Decode TOML into memory
| Process 1 | -> Adds mysql driver -> atomically writes the file out
| Process 2 | -> Adds duckdb to the ORIGINAL set of drivers -> atomically writes the file out, missing the mysql addition

The better solution here is file lock of some kind to prevent the concurrent mutation in the first place

[amoeba]

Thanks for the responses @zeroshade. I'm generally on board with this. I'm still a bit uneasy about having install --json stream the progress by default. Could that be put behind a flag since it's not intended for CLI users?

And item (3) above seems like a real bug worth addressing here. Other than that, I'm +1.

[zeroshade]

Fixed the last issue