diff --git a/CHANGELOG.md b/CHANGELOG.md index 573ca3880..870d887c3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -14,6 +14,37 @@ This driver uses semantic versioning: - A change in the major version (e.g. 1.Y.Z -> 2.0.0) indicates _breaking_ changes that require changes in your code to upgrade. +## [Unreleased] + +### Fixed + +- **Stream transactions (DE-10):** `Transaction#step` now keeps the transaction + context active until the callback's returned Promise settles. Async/await and + delayed Promise chains inside a step are correctly included in the transaction + (abort rolls them back). On Node.js, transaction IDs are tracked per async + context via `AsyncLocalStorage`, so concurrent stream transactions on the + same `Database` instance are supported. See [docs/stream-transactions.md](docs/stream-transactions.md). + +- **`Database#listTransactions`:** Requests now use `Database#request` (path + `/_db/:database-name/_api/transaction`) instead of calling the connection + without a database prefix. The method now lists stream transactions for the + `Database` instance's database, consistent with `beginTransaction`, `commit`, + and `abort`. **`Database#transactions`** is unaffected in API but benefits + transitively because it delegates to `listTransactions`. + +### Changed + +- **Internal (not a semver-major change):** Removed the connection-level + `_transactionId` field and `Connection#setTransactionId` / + `Connection#clearTransactionId` (`@internal` APIs). Transaction scope is + managed through async context (`src/lib/transaction-context.ts`). Public + transaction APIs are unchanged. + +### Added + +- [docs/stream-transactions.md](docs/stream-transactions.md) — user guide for + stream transactions (DE-10): behaviour, examples, migration, troubleshooting. + ## [10.3.1] - 2026-06-02 ### Fixed diff --git a/README.md b/README.md index 34a723067..cde689361 100644 --- a/README.md +++ b/README.md @@ -347,27 +347,45 @@ of Node.js at the time of this writing. When using arangojs in the browser, self-signed HTTPS certificates need to be trusted by the browser or use a trusted root certificate. -### Streaming transactions leak +### Stream transactions -When using the `transaction.step` method it is important to be aware of the -limitations of what a callback passed to this method is allowed to do. +Stream transactions associate database operations with a transaction using the +`x-arango-trx-id` HTTP header. When you call `trx.step(callback)`, the driver +keeps that transaction ID active for the entire lifetime of the callback's +returned Promise — including across `await` and `.then()` chains. + +On **Node.js** (including LTS versions 22 and 24), the driver uses +[`AsyncLocalStorage`](https://nodejs.org/api/async_context.html#class-asynclocalstorage) +to track the transaction ID per async context. That means: + +- Async work inside a step stays in the transaction (abort rolls it back). +- Multiple stream transactions can run concurrently on the same `Database` + instance (e.g. parallel HTTP handlers sharing one connection pool). + +In **browsers**, use one transaction at a time per `Database`, or use separate +`Database` instances for concurrent transactions. ```js const collection = db.collection(collectionName); -const trx = db.transaction(transactionId); +const trx = await db.beginTransaction(collection); -// WARNING: This code will not work as intended! +// Async work and multiple DB calls in one step are supported: await trx.step(async () => { + await loadFromExternalApi(); await collection.save(doc1); - await collection.save(doc2); // Not part of the transaction! + await collection.save(doc2); }); -// INSTEAD: Always perform a single operation per step: -await trx.step(() => collection.save(doc1)); -await trx.step(() => collection.save(doc2)); +// Or use the helper for automatic commit/abort: +await db.withTransaction(collection, async (step) => { + await step(() => collection.save(doc1)); + await step(() => collection.save(doc2)); +}); ``` -Please refer to the [documentation of the `transaction.step` method](https://arangodb.github.io/arangojs/latest/classes/transaction.Transaction.html#step) +Please refer to the +[stream transactions guide](docs/stream-transactions.md) and the +[documentation of the `transaction.step` method](https://arangodb.github.io/arangojs/latest/classes/transaction.Transaction.html#step) for additional examples. ### Streaming transactions timeout in cluster diff --git a/docs/stream-transactions.md b/docs/stream-transactions.md new file mode 100644 index 000000000..2462f08f4 --- /dev/null +++ b/docs/stream-transactions.md @@ -0,0 +1,540 @@ +# Stream transactions in arangojs + +This guide explains how **stream transactions** work in the arangojs JavaScript driver after the DE-10 fix. It is written for developers integrating arangojs and for anyone who needs a clear picture of what changed and how to use transactions safely. + +**Related Jira:** DE-10 — Review Stream Transactions in JavaScript driver. + +--- + +## Table of contents + +1. [What is a stream transaction?](#1-what-is-a-stream-transaction-plain-language) +2. [Quick start](#2-quick-start) +3. [What changed — old vs new](#3-what-changed--old-vs-new) +4. [How it works under the hood](#4-how-it-works-under-the-hood) +5. [Usage patterns and examples](#5-usage-patterns-and-examples) +6. [Concurrent requests and mixed workloads](#6-concurrent-requests-and-mixed-workloads) +7. [Node.js vs browser](#7-nodejs-vs-browser) +8. [Cluster and server limits](#8-cluster-and-server-limits) +9. [API reference (summary)](#9-api-reference-summary) +10. [Migration from the old behaviour](#10-migration-from-the-old-behaviour) +11. [Troubleshooting](#11-troubleshooting) +12. [Further reading](#12-further-reading) + +--- + +## 1. What is a stream transaction? + +A **transaction** groups several database changes into one unit: either **all** changes are applied together, or **none** of them are (on abort / error). + +**Stream transactions** are ArangoDB’s way of doing this across **multiple HTTP requests**: + +1. You **begin** a transaction on the server → you get a transaction ID. +2. Each database operation you want inside the transaction is sent with that ID (HTTP header `x-arango-trx-id`). +3. You **commit** (apply everything) or **abort** (roll back everything). + +In arangojs you do not set that header yourself. You wrap operations in `trx.step(...)` (or use `db.withTransaction(...)`), and the driver attaches the header for you. + +**Analogy:** Think of a transaction as a shopping cart. Items you add while the cart is open belong to the same purchase. Commit = checkout. Abort = empty the cart. The driver’s job is to make sure every “item” (DB call) you intend for the cart actually goes into the right cart — even when your code uses `async/await` or runs in parallel with other requests. + +--- + +## 2. Quick start + +### Manual transaction (begin → steps → commit) + +```js +import { Database } from "arangojs"; + +const db = new Database({ url: "http://localhost:8529" }); +const collection = db.collection("orders"); + +const trx = await db.beginTransaction(collection); + +try { + await trx.step(() => collection.save({ _key: "order-1", total: 100 })); + await trx.step(() => collection.save({ _key: "order-2", total: 200 })); + await trx.commit(); +} catch (err) { + await trx.abort().catch(() => {}); + throw err; +} +``` + +### Recommended helper (auto commit / abort) + +```js +await db.withTransaction(collection, async (step) => { + await step(() => collection.save({ _key: "order-1", total: 100 })); + await step(() => collection.save({ _key: "order-2", total: 200 })); +}); +// Commits on success; aborts if the callback throws. +``` + +### Async work inside one step + +```js +await trx.step(async () => { + await validateWithExternalApi(data); // non-DB async work is fine + await collection.save(data); // this save is part of the transaction +}); +``` + +--- + +## 3. What changed — old vs new + +### Summary for everyone + +| Topic | Before (old driver) | After (current driver) | +|-------|---------------------|-------------------------| +| **`await` before a DB call inside `trx.step`** | DB call often ran **outside** the transaction | DB call runs **inside** the transaction | +| **`trx.abort()` after async step** | Might **not** roll back async work | Rolls back async work correctly | +| **Multiple saves in one async step** | Only the first sync part was in the transaction | All saves in the step are in the transaction | +| **Two transactions at once on one `Database` (Node.js)** | Unreliable (shared connection ID) | **Supported** via AsyncLocalStorage | +| **Non-transactional request while a transaction runs (Node.js)** | Could accidentally share ID in edge cases | **No header** — isolated async context | +| **Where transaction ID lived** | One global field on the connection | Per async context (Node) or step-scoped slot (browser) | +| **Public API** | `beginTransaction`, `step`, `commit`, `abort`, `withTransaction` | **Unchanged** — same methods | + +### Technical comparison + +| Aspect | Old behaviour | New behaviour | +|--------|---------------|---------------| +| **When transaction ID was cleared** | Immediately when `step()` callback **returned** (sync `finally`) | When the callback’s **Promise settles** (Node: AsyncLocalStorage; browser: `.finally()`) | +| **Async callback** | Context lost at first `await` | Context kept for full Promise lifetime | +| **`.then()` delay** | e.g. `sleep(500).then(() => save())` ran outside trx | Delayed save still inside trx | +| **Concurrent `trx1.step` + `trx2.step` on same `db`** | Second could overwrite first’s ID | Each async context has its own ID (Node.js) | +| **Internal API** | `Connection#setTransactionId` / `#clearTransactionId` | Removed — use async context only | + +### Example: the original bug + +**Old (broken):** + +```js +await trx.step(async () => { + await someAsyncWork(); // driver cleared transaction ID here + return collection.save({ _key: "x" }); // sent WITHOUT x-arango-trx-id +}); +await trx.abort(); // save was NOT rolled back — data could remain +``` + +**New (fixed):** + +```js +await trx.step(async () => { + await someAsyncWork(); // transaction ID still active + return collection.save({ _key: "x" }); // sent WITH x-arango-trx-id +}); +await trx.abort(); // save IS rolled back +``` + +--- + +## 4. How it works under the hood + +### 4.1 High-level flow + +```mermaid +sequenceDiagram + participant App as Your application + participant Driver as arangojs driver + participant Ctx as Transaction context + participant Server as ArangoDB + + App->>Driver: db.beginTransaction(collections) + Driver->>Server: POST /_api/transaction/begin + Server-->>Driver: { id: "74780", status: "running" } + Driver-->>App: Transaction instance + + App->>Driver: trx.step(callback) + Driver->>Ctx: Activate transaction ID for this async context + App->>Driver: collection.save(...) (inside callback) + Driver->>Ctx: getActiveTransactionId() → "74780" + Driver->>Server: POST /_api/document/... + x-arango-trx-id: 74780 + Server-->>Driver: 202 Accepted + Note over Ctx: Promise settles → context released + + App->>Driver: trx.commit() + Driver->>Server: PUT /_api/transaction/74780 + Server-->>Driver: { status: "committed" } +``` + +### 4.2 What happens inside `trx.step()` + +```mermaid +sequenceDiagram + participant App as Application + participant Step as runTransactionStep + participant ALS as AsyncLocalStorage (Node.js) + participant Conn as Connection#request + participant DB as ArangoDB + + App->>Step: step(async callback) + Step->>ALS: run("74780", invoke callback) + Note over App,ALS: Async context now carries trx ID 74780 + + App->>App: await externalWork() + App->>Conn: collection.save() + Conn->>ALS: getStore() → "74780" + Conn->>DB: HTTP request + x-arango-trx-id: 74780 + + Note over App,ALS: Step Promise settles + ALS-->>Step: Context released (no trx ID in this chain) +``` + +**Key rule:** The HTTP header is added only when `Connection#request()` runs **and** the current async context has an active transaction ID. That ID is set only while you are inside `trx.step()` (or `withTransaction`’s `step` function). + +### 4.3 Node.js: AsyncLocalStorage + +On Node.js (20, 22, 24 LTS and later), the driver uses [AsyncLocalStorage](https://nodejs.org/api/async_context.html#class-asynclocalstorage) from `node:async_hooks`: + +- Each **async execution context** (e.g. one HTTP request handler) can have its **own** transaction ID. +- The ID propagates through `await`, `.then()`, and related Promise chains. +- When the Promise returned from `step()` settles, that context no longer carries the ID. +- Code that **never** entered `trx.step()` sees **no** transaction ID → requests go out without `x-arango-trx-id`. + +Implementation lives in `src/lib/transaction-context.ts`; `src/connection.ts` reads the ID at request time. + +### 4.4 Browser fallback + +Browsers do not provide AsyncLocalStorage. The driver uses a **single module-level slot** that is held until the step Promise settles. That fixes async-inside-step but does **not** safely support multiple concurrent transactions on one `Database`. Use separate `Database` instances for concurrent work in the browser. + +--- + +## 5. Usage patterns and examples + +### 5.1 Sequential steps (most common) + +One operation per step, awaited in order: + +```js +const trx = await db.beginTransaction({ write: ["orders", "inventory"] }); +const orders = db.collection("orders"); +const inventory = db.collection("inventory"); + +await trx.step(() => orders.save({ _key: "o1", sku: "A" })); +await trx.step(() => inventory.update("A", { stock: 9 })); +await trx.commit(); +``` + +```mermaid +sequenceDiagram + participant App + participant Trx as Transaction + + App->>Trx: step(save order) → ID active → save → ID released + App->>Trx: step(update stock) → ID active → update → ID released + App->>Trx: commit() +``` + +### 5.2 Multiple DB calls in one step + +```js +await trx.step(async () => { + const left = await vertices.save({ label: "left" }); + const right = await vertices.save({ label: "right" }); + return edges.save({ _from: left._id, _to: right._id }); +}); +await trx.commit(); +``` + +All three writes share the same transaction ID until the step Promise completes. + +### 5.3 Delayed work with `.then()` + +```js +const delay = (ms) => new Promise((r) => setTimeout(r, ms)); + +await trx.step(() => + delay(500).then(() => collection.save({ _key: "delayed" })) +); +// Save runs 500 ms later but still inside the transaction. +``` + +### 5.4 `withTransaction` (recommended) + +```js +const result = await db.withTransaction( + { write: ["vertices", "edges"] }, + async (step) => { + const start = await step(() => vertices.document("a")); + const end = await step(() => vertices.document("b")); + return step(() => edges.save({ _from: start._id, _to: end._id })); + } +); +// Returns the edge metadata; commits automatically. +``` + +On error, the driver attempts `trx.abort()` before rethrowing. + +### 5.5 Passing work to helper functions + +Helpers should receive `trx` or `step` and wrap each DB call: + +```js +async function saveOrder(step, order) { + await step(() => orders.save(order)); + await step(() => inventory.update(order.sku, { reserved: true })); +} + +const trx = await db.beginTransaction({ write: ["orders", "inventory"] }); +try { + await saveOrder(trx.step.bind(trx), { _key: "o1", sku: "A" }); + await trx.commit(); +} catch (e) { + await trx.abort().catch(() => {}); + throw e; +} +``` + +Or with `withTransaction`: + +```js +await db.withTransaction({ write: ["orders", "inventory"] }, async (step) => { + await saveOrder(step, { _key: "o1", sku: "A" }); +}); +``` + +### 5.6 What **not** to do + +```js +// WRONG: pass a promise instead of a callback +await trx.step(collection.save(data)); + +// WRONG: forget to return the promise from an async callback +await trx.step(async () => { + collection.save(data); // missing return/await — may not wait correctly +}); + +// WRONG: commit before step finishes +trx.step(() => collection.save(data)); // not awaited +await trx.commit(); // step may still be running +``` + +**Always:** `await trx.step(() => ...)` and await commit/abort after all steps complete. + +--- + +## 6. Concurrent requests and mixed workloads + +### 6.1 Two transactional handlers on one `Database` (Node.js) + +```js +const db = new Database(config); // shared across the app + +app.post("/orders", async (req, res) => { + await db.withTransaction("orders", async (step) => { + await step(() => db.collection("orders").save(req.body)); + }); + res.json({ ok: true }); +}); + +app.post("/inventory", async (req, res) => { + await db.withTransaction("inventory", async (step) => { + await step(() => db.collection("inventory").update(req.body._key, req.body)); + }); + res.json({ ok: true }); +}); +``` + +If both endpoints are hit at the same time, each request runs in its **own** async context with its **own** transaction ID. They do not interfere on Node.js. + +```mermaid +sequenceDiagram + participant ReqA as Request A (orders) + participant ReqB as Request B (inventory) + participant ALS as AsyncLocalStorage + participant DB as ArangoDB + + par Concurrent + ReqA->>ALS: context A → trx-id 111 + ReqA->>DB: save + x-arango-trx-id: 111 + and + ReqB->>ALS: context B → trx-id 222 + ReqB->>DB: update + x-arango-trx-id: 222 + end +``` + +### 6.2 Transactional write + non-transactional update at the same time + +```js +async function writeWithTransaction(doc) { + const trx = await db.beginTransaction(collection); + await trx.step(() => collection.save(doc)); + await trx.commit(); +} + +async function updateWithoutTransaction(doc) { + await collection.update(doc._key, doc); // NOT inside trx.step +} + +await Promise.all([ + writeWithTransaction(doc1), + updateWithoutTransaction(doc2), +]); +``` + +| Call | Inside `trx.step`? | `x-arango-trx-id` on request? | +|------|-------------------|-------------------------------| +| `writeWithTransaction` → `save` | Yes | **Yes** | +| `updateWithoutTransaction` → `update` | No | **No** | + +The update is a normal, immediate write — not part of any transaction. + +**Caution:** If you call `updateWithoutTransaction` **from inside** a `trx.step` callback, it **will** inherit the transaction ID because it runs in the same async context. Keep non-transactional calls outside `step` if you want them committed independently. + +### 6.3 Two stream transactions in parallel (explicit) + +```js +const trx1 = await db.beginTransaction(collection); +const trx2 = await db.beginTransaction(collection); + +await Promise.all([ + trx1.step(() => collection.save({ _key: "a" })), + trx2.step(() => collection.save({ _key: "b" })), +]); + +await trx1.commit(); +await trx2.commit(); +``` + +Supported on **Node.js**. In the **browser**, use separate `Database` instances instead of parallel transactions on one instance. + +--- + +## 7. Node.js vs browser + +| Capability | Node.js 20+ (incl. 22 & 24 LTS) | Browser | +|------------|----------------------------------|---------| +| Async work inside `trx.step` | Yes | Yes | +| Multiple DB calls in one step | Yes | Yes | +| `trx.abort()` rolls back async steps | Yes | Yes | +| Concurrent transactions on one `Database` | **Yes** (AsyncLocalStorage) | **No** — use separate `Database` instances | +| Concurrent trx + non-trx on one `Database` | Isolated contexts | Risk of cross-talk — prefer separate instances | +| Mechanism | `AsyncLocalStorage` per async context | Single slot until step Promise settles | + +--- + +## 8. Cluster and server limits + +Stream transaction behaviour on the **server** is unchanged. The driver fix is client-side only. + +| Topic | Notes | +|-------|--------| +| **Idle timeout** | See [Idle timeout between operations](#81-idle-timeout-between-operations) below. | +| **Cluster** | Multi-document ACID has cluster limitations; see [ArangoDB transaction docs](https://docs.arango.ai/arangodb/stable/develop/transactions/). | +| **`poolSize`** | In cluster with load balancing, you may need a higher `config.poolSize` for many parallel transactions. See README “Streaming transactions timeout in cluster”. | +| **ArangoDB 4.0** | JavaScript transactions (`executeTransaction`) removed; **stream transactions** are the supported multi-step API. | +| **Begin options** | `skipFastLockRound`, `maxTransactionSize`, `lockTimeout`, `allowImplicit`, `waitForSync` — passed to `beginTransaction` / `withTransaction`. | + +### 8.1 Idle timeout between operations + +ArangoDB enforces a **maximum idle time** between operations in a single stream transaction (on coordinators and single servers). This prevents abandoned transactions from holding locks indefinitely. + +| Setting | Value | +|---------|--------| +| **Default** | **60 seconds** between operations | +| **Maximum (configurable)** | **Up to 120 seconds** via the server startup option `--transaction.streaming-idle-timeout` | +| **Reset behaviour** | Each operation sent while the transaction is still valid **resets** the idle timer to the configured timeout | + +**Plain language:** If you begin a transaction and then wait too long before the next `trx.step()` (or commit/abort), the server may expire the transaction. The clock resets every time you perform an operation inside that transaction. + +**Example:** With the default 60 s timeout, a gap of 90 s between two steps can fail with `transaction not found` or similar. An administrator can raise the limit to at most 120 s on the server; the driver cannot change this — it is enforced by ArangoDB. + +Official reference: [Stream Transactions — timeout and transaction size](https://docs.arango.ai/arangodb/stable/develop/transactions/stream-transactions/) and [Transaction limitations](https://docs.arango.ai/arangodb/stable/develop/transactions/limitations/). + +--- + +## 9. API reference (summary) + +| Method | Description | +|--------|-------------| +| `db.beginTransaction(collections, options?)` | Starts a stream transaction; returns `Transaction`. | +| `trx.step(callback)` | Runs `callback` in transaction context; callback must return a Promise. | +| `trx.commit(options?)` | Commits the transaction. | +| `trx.abort(options?)` | Aborts and rolls back. | +| `trx.get()` | Returns `{ id, status }`. | +| `db.withTransaction(collections, callback, options?)` | Begin + run callback with `step` + commit; abort on throw. | +| `db.transaction(id)` | Returns a `Transaction` handle for an existing server-side ID. | +| `db.listTransactions()` | Lists running stream transactions for this database. | + +**Collection options** for `beginTransaction`: + +```js +await db.beginTransaction({ + read: ["readonly_col"], + write: ["orders", "inventory"], + exclusive: ["locked_col"], // optional exclusive lock +}); +``` + +Or pass a single collection / array of collections as shorthand for `{ write: [...] }`. + +--- + +## 10. Migration from the old behaviour + +### If you followed old “one DB call per step” guidance + +That pattern still works and remains a good style for clarity. You are **not** required to merge steps — you **may** now use async/multi-call steps when it simplifies your code. + +### If you worked around the async bug + +Patterns like splitting every `await` into its own `trx.step` or moving async logic **outside** `step` were compensating for the old leak. You can simplify: + +```js +// Old workaround +await loadData(); +await trx.step(() => collection.save(data)); + +// New — optional simplification +await trx.step(async () => { + await loadData(); + return collection.save(data); +}); +``` + +### If you relied on concurrent transactions on one `Database` (Node.js) + +Previously unreliable; now supported. No code change required if you already structured steps correctly — behaviour becomes correct instead of racy. + +### Internal / advanced integrations + +If you called `Connection#setTransactionId` or `#clearTransactionId` directly (internal API), those methods are **removed**. Use public `trx.step()` only. + +--- + +## 11. Troubleshooting + +| Symptom | Likely cause | What to do | +|---------|--------------|------------| +| `abort()` did not roll back changes | Old driver version, or DB call **outside** `trx.step` | Upgrade; wrap every transactional call in `step`; await all steps before abort | +| `transaction not found` / `expired` | Idle timeout between steps exceeded (default **60 s**, max configurable **120 s** on server) | Keep steps within the idle window; ask ops to set `--transaction.streaming-idle-timeout` (≤ 120 s); check cluster coordinator stickiness | +| Non-transactional update appears transactional | Update called **inside** `trx.step` callback | Move non-transactional calls outside `step` | +| Concurrent transactions interfere (browser) | Browser single-slot fallback | Use one `Database` per concurrent transaction | +| `Transaction callback was not an async function...` | `step()` callback did not return a Promise | Use `() => collection.save(...)` or an `async` function | +| Changes visible before commit | Reading **outside** the transaction | Reads without trx ID see committed data only; use `step` for reads that must see uncommitted trx data | + +--- + +## 12. Further reading + +- [Stream Transactions — ArangoDB HTTP API](https://docs.arango.ai/arangodb/stable/develop/http-api/transactions/stream-transactions/) +- [Stream Transactions — ArangoDB developer guide](https://docs.arango.ai/arangodb/stable/develop/transactions/stream-transactions/) +- [Node.js AsyncLocalStorage](https://nodejs.org/api/async_context.html#class-asynclocalstorage) +- [ArangoDB 4.0 removed methods](./arangodb-v4-removed-methods.md) — `executeTransaction` removed; use stream transactions +- Generated API docs: [`Transaction#step`](https://arangodb.github.io/arangojs/latest/classes/transaction.Transaction.html#step), [`Database#withTransaction`](https://arangodb.github.io/arangojs/latest/classes/database.Database.html#withTransaction) + +--- + +## Glossary + +| Term | Meaning | +|------|---------| +| **Stream transaction** | Multi-step transaction: begin on server, attach ID to each operation, then commit or abort. | +| **`x-arango-trx-id`** | HTTP header that tells ArangoDB which transaction an operation belongs to. | +| **`trx.step(callback)`** | Driver API that runs `callback` while the transaction ID is active for that async context. | +| **AsyncLocalStorage** | Node.js feature that stores data per async context (used to isolate concurrent transactions). | +| **Idle timeout** | Maximum time with no operations before the server expires a stream transaction. Default **60 s**; configurable up to **120 s** (`--transaction.streaming-idle-timeout`). | +| **Commit** | Apply all transaction changes permanently. | +| **Abort** | Discard all transaction changes. | diff --git a/src/connection.ts b/src/connection.ts index e8b6dd709..08a6ce94e 100644 --- a/src/connection.ts +++ b/src/connection.ts @@ -12,6 +12,7 @@ import * as configuration from "./configuration.js"; import * as databases from "./databases.js"; import * as errors from "./errors.js"; import { ERROR_ARANGO_CONFLICT } from "./lib/codes.js"; +import { getActiveTransactionId } from "./lib/transaction-context.js"; import * as util from "./lib/util.js"; import { LinkedList } from "./lib/x3-linkedlist.js"; @@ -687,7 +688,6 @@ export class Connection { protected _hostUrls: string[] = []; protected _activeHostUrl: string; protected _activeDirtyHostUrl: string; - protected _transactionId: string | null = null; protected _onError?: (err: Error) => void | Promise; protected _precaptureStackTraces: boolean; protected _queueTimes = new LinkedList<[number, number]>(); @@ -1027,32 +1027,6 @@ export class Connection { return cleanUrls; } - /** - * @internal - * - * Sets the connection's active `transactionId`. - * - * While set, all requests will use this ID, ensuring the requests are executed - * within the transaction if possible. Setting the ID manually may cause - * unexpected behavior. - * - * See also {@link Connection#clearTransactionId}. - * - * @param transactionId - ID of the active transaction. - */ - setTransactionId(transactionId: string) { - this._transactionId = transactionId; - } - - /** - * @internal - * - * Clears the connection's active `transactionId`. - */ - clearTransactionId() { - this._transactionId = null; - } - /** * @internal * @@ -1245,8 +1219,9 @@ export class Connection { headers.set("content-length", "0"); } - if (this._transactionId) { - headers.set("x-arango-trx-id", this._transactionId); + const transactionId = getActiveTransactionId(); + if (transactionId) { + headers.set("x-arango-trx-id", transactionId); } if (allowDirtyRead) { diff --git a/src/databases.ts b/src/databases.ts index 350b0395b..7f31f3078 100644 --- a/src/databases.ts +++ b/src/databases.ts @@ -2838,7 +2838,7 @@ export class Database { * ``` */ listTransactions(): Promise { - return this._connection.request( + return this.request( { pathname: "/_api/transaction" }, (res) => res.parsedBody.transactions ); diff --git a/src/lib/transaction-context.ts b/src/lib/transaction-context.ts new file mode 100644 index 000000000..c31a8eb29 --- /dev/null +++ b/src/lib/transaction-context.ts @@ -0,0 +1,102 @@ +/** + * Transaction ID scope for stream transaction steps. + * + * Node.js uses AsyncLocalStorage so concurrent transactions on the same + * Connection carry independent transaction IDs through async continuations. + * + * In browsers (no AsyncLocalStorage), a single slot is held until the step + * promise settles. Use separate `Database` instances for concurrent + * transactions in the browser. + * + * @packageDocumentation + * @internal + */ + +let nodeAls: + | { + run(store: string, callback: () => T): T; + getStore(): string | undefined; + } + | null = null; +let nodeAlsReady: Promise | null = null; +let fallbackSlot: string | undefined; + +function isNodeRuntime(): boolean { + return ( + typeof process !== "undefined" && + typeof process.versions === "object" && + process.versions !== null && + typeof process.versions.node === "string" + ); +} + +function ensureNodeAls(): Promise { + if (!isNodeRuntime()) { + return Promise.resolve(); + } + if (nodeAls) { + return Promise.resolve(); + } + if (!nodeAlsReady) { + nodeAlsReady = (async () => { + try { + const spec = ["node:", "async_hooks"].join(""); + const { AsyncLocalStorage } = (await import( + spec + )) as typeof import("node:async_hooks"); + nodeAls = new AsyncLocalStorage(); + } catch { + nodeAls = null; + } + })(); + } + return nodeAlsReady; +} + +/** + * Returns the active stream transaction ID for the current async context, if any. + */ +export function getActiveTransactionId(): string | undefined { + if (nodeAls) { + return nodeAls.getStore(); + } + return fallbackSlot; +} + +/** + * Runs `callback` while `transactionId` is the active transaction ID. + * + * On Node.js the scope is preserved until a returned Promise settles. + * In the browser the slot is cleared when the returned Promise settles. + */ +export async function runTransactionStep( + transactionId: string, + callback: () => Promise +): Promise { + await ensureNodeAls(); + + const invoke = (): Promise => { + const promise = callback(); + if (!promise || typeof promise.then !== "function") { + throw new Error( + "Transaction callback was not an async function or did not return a promise!" + ); + } + return Promise.resolve(promise); + }; + + if (nodeAls) { + return nodeAls.run(transactionId, invoke); + } + + const previous = fallbackSlot; + fallbackSlot = transactionId; + try { + return await invoke().finally(() => { + fallbackSlot = previous; + }); + } catch (err) { + fallbackSlot = previous; + throw err; + } +} diff --git a/src/test/04-transactions.ts b/src/test/04-transactions.ts index e29f41c51..0498210f0 100644 --- a/src/test/04-transactions.ts +++ b/src/test/04-transactions.ts @@ -189,5 +189,276 @@ describe("Transactions", function () { const doc = await collection.document("test"); expect(doc).to.have.property("_key", "test"); }); + + it("keeps async work inside the transaction", async () => { + const sleep = (millis: number) => + new Promise((resolve) => setTimeout(resolve, millis)); + const trx = await db.beginTransaction(collection); + allTransactions.push(trx); + + await trx.step(async () => { + await sleep(50); + return collection.save({ _key: "async-test" }); + }); + + const existsInTrx = await trx.step(() => + collection.documentExists("async-test"), + ); + expect(existsInTrx).to.equal(true); + + let leaked = false; + try { + await collection.document("async-test"); + leaked = true; + } catch {} + expect(leaked).to.equal(false); + + await trx.abort(); + + const existsAfterAbort = await collection.documentExists("async-test"); + expect(existsAfterAbort).to.equal(false); + }); + + it("supports multiple DB calls in one async step", async () => { + const trx = await db.beginTransaction(collection); + allTransactions.push(trx); + await trx.step(async () => { + await collection.save({ _key: "multi-a" }); + await collection.save({ _key: "multi-b" }); + }); + const a = await trx.step(() => collection.documentExists("multi-a")); + const b = await trx.step(() => collection.documentExists("multi-b")); + expect(a).to.equal(true); + expect(b).to.equal(true); + await trx.commit(); + expect(await collection.documentExists("multi-a")).to.equal(true); + expect(await collection.documentExists("multi-b")).to.equal(true); + }); + + it("supports concurrent transactions on the same database", async () => { + const sleep = (millis: number) => + new Promise((resolve) => setTimeout(resolve, millis)); + + const trx1 = await db.beginTransaction(collection); + const trx2 = await db.beginTransaction(collection); + allTransactions.push(trx1, trx2); + + const [meta1, meta2] = await Promise.all([ + trx1.step(() => + sleep(100).then(() => collection.save({ _key: "concurrent-1" })), + ), + trx2.step(() => collection.save({ _key: "concurrent-2" })), + ]); + + expect(meta1).to.have.property("_key", "concurrent-1"); + expect(meta2).to.have.property("_key", "concurrent-2"); + + const doc1InTrx1 = await trx1.step(() => + collection.documentExists("concurrent-1"), + ); + expect(doc1InTrx1).to.equal(true, "doc1 should exist within trx1"); + + const doc2InTrx2 = await trx2.step(() => + collection.documentExists("concurrent-2"), + ); + expect(doc2InTrx2).to.equal(true, "doc2 should exist within trx2"); + + const doc1OutsideTrx1 = await collection.documentExists("concurrent-1"); + expect(doc1OutsideTrx1).to.equal( + false, + "doc1 should not exist outside trx1", + ); + + await trx1.abort(); + await trx2.abort(); + }); + + it("withTransaction commits on success and returns the result", async () => { + const meta = await db.withTransaction(collection, async (step) => + step(() => collection.save({ _key: "with-tx-ok" })), + ); + expect(meta).to.have.property("_key", "with-tx-ok"); + expect(await collection.documentExists("with-tx-ok")).to.equal(true); + }); + + it("withTransaction aborts when the callback throws", async () => { + try { + await db.withTransaction(collection, async (step) => { + await step(() => collection.save({ _key: "with-tx-fail" })); + throw new Error("deliberate failure"); + }); + expect.fail("Expected withTransaction to throw"); + } catch (e: any) { + expect(String(e)).to.include("deliberate failure"); + } + expect(await collection.documentExists("with-tx-fail")).to.equal(false); + }); + + it("supports sequential transactions on the same database", async () => { + await db.withTransaction(collection, async (step) => { + await step(() => collection.save({ _key: "sequential-1" })); + }); + await db.withTransaction(collection, async (step) => { + await step(() => collection.save({ _key: "sequential-2" })); + }); + expect(await collection.documentExists("sequential-1")).to.equal(true); + expect(await collection.documentExists("sequential-2")).to.equal(true); + }); + + it("does not attach a transaction id to concurrent non-transactional writes", async () => { + const sleep = (millis: number) => + new Promise((resolve) => setTimeout(resolve, millis)); + const trx = await db.beginTransaction(collection); + allTransactions.push(trx); + + await Promise.all([ + trx.step(async () => { + await sleep(100); + await collection.save({ _key: "concurrent-in-trx" }); + }), + (async () => { + await sleep(20); + await collection.save({ _key: "concurrent-outside-trx" }); + })(), + ]); + + expect(await collection.documentExists("concurrent-outside-trx")).to.equal( + true, + "non-transactional write should commit immediately", + ); + expect(await collection.documentExists("concurrent-in-trx")).to.equal( + false, + "transactional write should not be visible outside the transaction", + ); + + await trx.abort(); + expect(await collection.documentExists("concurrent-in-trx")).to.equal( + false, + ); + expect(await collection.documentExists("concurrent-outside-trx")).to.equal( + true, + "non-transactional write should survive trx abort", + ); + }); + + it("aborts all saves from one async step", async () => { + const trx = await db.beginTransaction(collection); + allTransactions.push(trx); + await trx.step(async () => { + await collection.save({ _key: "abort-multi-a" }); + await collection.save({ _key: "abort-multi-b" }); + }); + expect(await trx.step(() => collection.documentExists("abort-multi-a"))).to + .equal(true); + expect(await trx.step(() => collection.documentExists("abort-multi-b"))).to + .equal(true); + await trx.abort(); + expect(await collection.documentExists("abort-multi-a")).to.equal(false); + expect(await collection.documentExists("abort-multi-b")).to.equal(false); + }); + + it("can update a document inside a transaction", async () => { + await collection.save({ _key: "update-me", value: 1 }); + const trx = await db.beginTransaction(collection); + allTransactions.push(trx); + await trx.step(() => collection.update("update-me", { value: 2 })); + const inTrx = await trx.step(() => collection.document("update-me")); + expect(inTrx).to.have.property("value", 2); + const outsideTrx = await collection.document("update-me"); + expect(outsideTrx).to.have.property("value", 1); + await trx.commit(); + const committed = await collection.document("update-me"); + expect(committed).to.have.property("value", 2); + }); + + it("can remove a document inside a transaction", async () => { + await collection.save({ _key: "remove-me" }); + const trx = await db.beginTransaction(collection); + allTransactions.push(trx); + await trx.step(() => collection.remove("remove-me")); + expect( + await trx.step(() => collection.documentExists("remove-me")), + ).to.equal(false); + expect(await collection.documentExists("remove-me")).to.equal(true); + await trx.commit(); + expect(await collection.documentExists("remove-me")).to.equal(false); + }); + + it("can run an AQL query inside a transaction step", async () => { + await collection.save({ _key: "aql-doc", label: "hello" }); + const trx = await db.beginTransaction(collection); + allTransactions.push(trx); + const label = await trx.step(async () => { + const cursor = await db.query( + "FOR d IN @@col FILTER d._key == @key RETURN d.label", + { "@col": collection.name, key: "aql-doc" }, + ); + return cursor.next(); + }); + expect(label).to.equal("hello"); + await trx.commit(); + }); + + it("can use multiple collections in one transaction", async () => { + const other = await db.createCollection(`other-${Date.now()}`); + await db.waitForPropagation( + { pathname: `/_api/collection/${other.name}` }, + propagationForResourceMs, + ); + const trx = await db.beginTransaction([collection, other]); + allTransactions.push(trx); + await trx.step(() => collection.save({ _key: "multi-col-a" })); + await trx.step(() => other.save({ _key: "multi-col-b" })); + await trx.commit(); + expect(await collection.documentExists("multi-col-a")).to.equal(true); + expect(await other.documentExists("multi-col-b")).to.equal(true); + await other.drop(); + }); + + it("lists a running transaction via listTransactions", async () => { + const trx = await db.beginTransaction(collection); + allTransactions.push(trx); + try { + const running = await db.listTransactions(); + expect( + running.some( + (t) => String(t.id) === String(trx.id) && t.state === "running", + ), + ).to.equal(true); + } finally { + await trx.commit(); + } + }); + + it("throws when step callback does not return a promise", async () => { + const trx = await db.beginTransaction(collection); + allTransactions.push(trx); + try { + await trx.step( + (() => undefined) as unknown as () => Promise, + ); + expect.fail("Expected step to throw"); + } catch (e: any) { + expect(String(e)).to.include("did not return a promise"); + } + await trx.abort(); + }); + + it("supports helper functions that receive step", async () => { + async function saveViaStep( + step: Transaction["step"], + key: string, + value: number, + ) { + await step(() => collection.save({ _key: key, value })); + } + + await db.withTransaction(collection, async (step) => { + await saveViaStep(step, "helper-a", 1); + await saveViaStep(step, "helper-b", 2); + }); + expect(await collection.document("helper-a")).to.have.property("value", 1); + expect(await collection.document("helper-b")).to.have.property("value", 2); + }); }); }); diff --git a/src/transactions.ts b/src/transactions.ts index 7f222187f..31a3c86e3 100644 --- a/src/transactions.ts +++ b/src/transactions.ts @@ -9,9 +9,9 @@ * @packageDocumentation */ import * as collections from "./collections.js"; -import * as connection from "./connection.js"; import * as databases from "./databases.js"; import * as errors from "./errors.js"; +import { runTransactionStep } from "./lib/transaction-context.js"; import { TRANSACTION_NOT_FOUND } from "./lib/codes.js"; //#region Transaction operation options @@ -350,22 +350,21 @@ export class Transaction { } /** - * Executes the given function locally as a single step of the transaction. + * Executes the given function as a single step of the transaction. * * @param T - Type of the callback's returned promise. * @param callback - Callback function returning a promise. * - * **Warning**: The callback function should wrap a single call of an async - * arangojs method (e.g. a method on a `Collection` object of a collection - * that is involved in the transaction or the `db.query` method). - * If the callback function is async, only the first promise-returning (or - * async) method call will be executed as part of the transaction. See the - * examples below for how to avoid common mistakes when using this method. + * The callback must return a `Promise`. The callback may be `async` and may + * perform multiple arangojs calls (with `await` between them). All requests + * made while the callback's returned Promise is pending are sent with this + * transaction's ID. * - * **Note**: Avoid defining the callback as an async function if possible - * as arangojs will throw an error if the callback did not return a promise. - * Async functions will return an empty promise by default, making it harder - * to notice if you forgot to return something from the callback. + * On Node.js, concurrent steps of **different** transactions on the same + * {@link databases.Database} are supported because the driver tracks the + * transaction ID per async context ({@link https://nodejs.org/api/async_context.html | AsyncLocalStorage}). + * + * In browsers, use separate `Database` instances for concurrent transactions. * * **Note**: Although almost anything can be wrapped in a callback and passed * to this method, that does not guarantee ArangoDB can actually do it in a @@ -392,133 +391,103 @@ export class Transaction { * data: "potato" * })); * - * // Transaction must be committed for changes to take effected + * // Transaction must be committed for changes to take effect * // Always call either trx.commit or trx.abort to end a transaction * await trx.commit(); * ``` * * @example * ```js - * // BAD! If the callback is an async function it must only use await once! + * // Async work before a DB call stays inside the transaction: * await trx.step(async () => { - * await collection.save(data); - * await collection.save(moreData); // WRONG - * }); - * - * // BAD! Callback function must use only one arangojs call! - * await trx.step(() => { - * return collection.save(data) - * .then(() => collection.save(moreData)); // WRONG + * await loadDataFromExternalApi(); + * return collection.save({ _key: "x" }); * }); - * - * // BETTER: Wrap every arangojs method call that should be part of the - * // transaction in a separate `trx.step` call - * await trx.step(() => collection.save(data)); - * await trx.step(() => collection.save(moreData)); + * await trx.abort(); // the save above is rolled back * ``` * * @example * ```js - * // BAD! If the callback is an async function it must not await before - * // calling an arangojs method! + * // Multiple DB calls in one step: * await trx.step(async () => { - * await doSomethingElse(); - * return collection.save(data); // WRONG - * }); - * - * // BAD! Any arangojs inside the callback must not happen inside a promise - * // method! - * await trx.step(() => { - * return doSomethingElse() - * .then(() => collection.save(data)); // WRONG + * const a = await collection.save({ _key: "a" }); + * const b = await collection.save({ _key: "b" }); + * return b; * }); + * ``` * - * // BETTER: Perform any async logic needed outside the `trx.step` call - * await doSomethingElse(); - * await trx.step(() => collection.save(data)); + * @example + * ```js + * // Concurrent transactions on one Database (Node.js): + * const [r1, r2] = await Promise.all([ + * trx1.step(() => collection.save({ _key: "a" })), + * trx2.step(() => collection.save({ _key: "b" })), + * ]); + * ``` * - * // OKAY: You can perform async logic in the callback after the arangojs - * // method call as long as it does not involve additional arangojs method - * // calls, but this makes it easy to make mistakes later - * await trx.step(async () => { - * await collection.save(data); - * await doSomethingDifferent(); // no arangojs method calls allowed + * @example + * ```js + * // Prefer db.withTransaction for automatic commit/abort: + * await db.withTransaction(collection, async (step) => { + * await step(() => collection.save({ _key: "a" })); + * await step(() => collection.save({ _key: "b" })); * }); * ``` * * @example * ```js - * // BAD! The callback should not use any functions that themselves use any - * // arangojs methods! + * // BAD! The callback should not use helper functions that call arangojs + * // methods without going through `trx.step` themselves! * async function saveSomeData() { * await collection.save(data); * await collection.save(moreData); * } * await trx.step(() => saveSomeData()); // WRONG * - * // BETTER: Pass the transaction to functions that need to call arangojs - * // methods inside a transaction - * async function saveSomeData(trx) { - * await trx.step(() => collection.save(data)); - * await trx.step(() => collection.save(moreData)); + * // BETTER: Pass the transaction (or its step function) to helpers + * async function saveSomeData(step) { + * await step(() => collection.save(data)); + * await step(() => collection.save(moreData)); * } - * await saveSomeData(); // no `trx.step` call needed + * await saveSomeData(trx.step.bind(trx)); * ``` * * @example * ```js - * // BAD! You must wait for the promise to resolve (or await on the - * // `trx.step` call) before calling `trx.step` again! - * trx.step(() => collection.save(data)); // WRONG + * // BAD! You must await each `trx.step` before starting the next step on + * // the same transaction! + * trx.step(() => collection.save(data)); // WRONG — not awaited * await trx.step(() => collection.save(moreData)); * - * // BAD! The trx.step callback can not make multiple calls to async arangojs - * // methods, not even using Promise.all! - * await trx.step(() => Promise.all([ // WRONG - * collection.save(data), - * collection.save(moreData), - * ])); - * - * // BAD! Multiple `trx.step` calls can not run in parallel! - * await Promise.all([ // WRONG - * trx.step(() => collection.save(data)), - * trx.step(() => collection.save(moreData)), - * ])); - * - * // BETTER: Always call `trx.step` sequentially, one after the other + * // BETTER: Always await sequential steps on one transaction * await trx.step(() => collection.save(data)); * await trx.step(() => collection.save(moreData)); * - * // OKAY: The then callback can be used if async/await is not available + * // OKAY: Chain with `.then` if async/await is not available * trx.step(() => collection.save(data)) * .then(() => trx.step(() => collection.save(moreData))); * ``` * * @example * ```js - * // BAD! The callback will return an empty promise that resolves before - * // the inner arangojs method call has even talked to ArangoDB! + * // BAD! The callback must return a promise — async functions must return + * // or await the arangojs call! * await trx.step(async () => { - * collection.save(data); // WRONG + * collection.save(data); // WRONG — missing return/await * }); * * // BETTER: Use an arrow function so you don't forget to return * await trx.step(() => collection.save(data)); * - * // OKAY: Remember to always return when using a function body + * // OKAY: Remember to return when using a function body * await trx.step(() => { - * return collection.save(data); // easy to forget! - * }); - * - * // OKAY: You do not have to use arrow functions but it helps - * await trx.step(function () { * return collection.save(data); * }); * ``` * * @example * ```js - * // BAD! You can not pass promises instead of a callback! + * // BAD! You cannot pass a promise instead of a callback! * await trx.step(collection.save(data)); // WRONG * * // BETTER: Wrap the code in a function and pass the function instead @@ -527,29 +496,28 @@ export class Transaction { * * @example * ```js - * // WORSE: Calls to non-async arangojs methods don't need to be performed - * // as part of a transaction - * const collection = await trx.step(() => db.collection("my-documents")); + * // BAD! Non-async arangojs methods do not perform HTTP requests and must + * // not be wrapped in `trx.step` — the callback must return a promise! + * await trx.step(() => db.collection("my-documents")); // WRONG — throws * - * // BETTER: If an arangojs method is not async and doesn't return promises, - * // call it without `trx.step` + * // BETTER: Resolve collection handles outside the transaction step * const collection = db.collection("my-documents"); + * const trx = await db.beginTransaction(collection); + * await trx.step(() => collection.save(data)); + * ``` + * + * @example + * ```js + * // OKAY: Async logic after the arangojs call is fine as long as it does + * // not invoke additional arangojs methods (easy to break later) + * await trx.step(async () => { + * await collection.save(data); + * await doSomethingDifferent(); // no arangojs method calls + * }); * ``` */ step(callback: () => Promise): Promise { - const conn = (this._db as any)._connection as connection.Connection; - conn.setTransactionId(this.id); - try { - const promise = callback(); - if (!promise) { - throw new Error( - "Transaction callback was not an async function or did not return a promise!" - ); - } - return Promise.resolve(promise); - } finally { - conn.clearTransactionId(); - } + return runTransactionStep(this.id, callback); } } //#endregion