Internals

Architecture#

Data is a headless modular monolith with an optional separate worker process and no user interface; splitting it into more services is not required. Studios, websites, scripts and services are all API clients, and none has a privileged database connection. Data never reads Access tables, never mints an identity (it holds only the gateway's public key), and never creates a session or runs a login flow (spec §1.2).

Code layout#

Only src/runtime/config.ts reads environment variable names; everything else receives typed settings (Configuration).

Domains and import rules#

The required domains are gateway-context, workspaces, schema, authorization, catalog, records, audit and events, plus model for the workspace model.

1. A domain's core imports no Hono, no Bun API, no application handler, no runtime and no concrete adapter.
2. The application imports domains' public entry points, never their internals.
3. Only the composition root constructs concrete adapters.
4. Worker orchestration lives in the runtime and calls the domains' public use cases.
5. Audit and events accept primitive-typed envelopes and injected persistence ports, and do not import records.
6. A unit-of-work port lets records coordinate its repositories, audit sink and outbox without importing the concrete SQL adapters.
7. No cycle is allowed, including a type-only cycle, and no catch-all directory of shared implementations.

bun run architecture enforces the matrix and these rules, and test/architecture checks that the matrix names every domain.

Request flow#

1. The gateway signs the caller's identity into x-gateway-signature. Data rebuilds the canonical payload from method, path, query, host and headers and verifies it with the route's public key; it never trusts a transmitted payload.
2. One statement reads the workspace and its catalog (collections and policy bundle).
3. Access answers the principal's permission keys for subject, tenant, workspace and route, or a refusal, outside the business-data transaction.
4. The scoped transaction projects only granted fields, authorizes a write by one complete policy, and commits record, audit entry, domain event and receipt together (spec §10).

Storage#

1. Deployments run PostgreSQL 15 or later; development, tests and CI use 18 (decision 32).
2. Business tables carry tenant_id and workspace_id under row-level security, scoped per transaction with set_config(…, true).
3. There are five roles: migration owner, API runtime, registry reader, delivery and maintenance. The API never holds the credentials of the other four (Operations).
4. Tables come only from reviewed migrations. A declared history table mirrors its collection table and is written by the change statement (spec §11.6).

Catalog, model and events#

1. The catalog (registered collections and policy bundle) is a logical registry over the global physical schema (spec §7.5). It is read uncached, so a published bundle applies on the next request.
2. The model overlays the manifest in realtime (spec §7.6). Its domain owns the model store, table column reader and workspace collection registrar ports.
3. Subscribers registered in the composition root receive each committed event at least once, possibly out of order. Workers claim deliveries with leases and act with a fresh context for their service principal; the originating user is only an attribution.

The process roles (api, worker, all) are described in Operations.

Development#

Installation and the platform-wide commands are in platform development.

Test suites#

The differential tests (test/integration/filter.property.test.ts and write-guard.test.ts) compare generated predicates and write policies evaluated in memory and in the database. Fuzzing (test/unit/fuzz.test.ts) proves caller input never reaches a statement as an identifier. Integration tests start their own PostgreSQL 18 server (decision 16).

Release gates#

bun run typecheck
bun run lint
bun run format:check
bun run architecture
bun run dead
bun test test/unit test/contract test/integration test/architecture packages/client/test
bun run test:mutation
bun run test:compat

The operational gates come on top: the backup rehearsal, the capacity measurement, the declared rate and latency objective, and CI's dependency audit. The mutation gate is textual, so a refactor that moves a mutation's text must update the mutation, or the gate fails it as stale.

Scripts#

Operator commands run on the deployment machine. They are never HTTP routes and never bypass Access. Their credentials are listed in Configuration.

bun run dead finds unreachable modules, unread exported names and unused dependencies. bun run release:tag tags a release whose version increased.

To add a collection to this deployment, follow Modeling.

Release acceptance#

No requirement is complete until an automated check demonstrates its behavior.

Mandatory adversarial cases:

1. A grant for workspace A cannot read, write, count, reference or expand workspace B's data.
2. An Access grant for one tenant does not make another tenant's registered workspace accessible.
3. A forged tenant, forged route, e-mail address match or stale role claim cannot influence authorization.
4. Group removal, grant revocation and expiry, a disabled principal and a disabled tenant take effect within the documented freshness bounds.
5. Filtering and sorting by a hidden field are refused, and counts and relation expansions cannot bypass row policies.
6. Two policies with complementary field sets cannot authorize a synthetic write neither permits.
7. An update that moves a row outside its check is refused atomically.
8. Concurrent updates with one ETag yield one successful version change and one precondition refusal.
9. A version hidden by the deployment's policies is not found in the history, and a rollback they forbid is refused: history is not a path around authorization.
10. A POST retried after a committed but lost response creates one record and returns its original receipt.
11. Reusing a create key with another body cannot create another record within the retention window.
12. An expired or malformed Access context, wrong context identifiers and incomplete capability sets never allow a data operation.
13. A pooled connection never keeps a previous tenant, workspace or principal scope.
14. Runtime credentials cannot disable row-level security, alter the schema, edit audit history, or query Access's tables.
15. A subscriber cannot turn an actor's attribution into current authorization.
16. All backend functionality and API tests run without any Studio process or package.

Capacity#

bun run scripts/ops/capacity.ts starts PostgreSQL, a fixture Access, Data and concurrent clients that sign like the pinned gateway, then runs the scenarios of spec §15.5. Tune it with CAPACITY_CLIENTS (default 16) and CAPACITY_DURATION_MS (default 10000 per scenario). The numbers describe one machine at one moment; a deployment measures its own with the same command.

Results#

Release candidate 1.0.0, guarded write path:

1. Reads hold up with a context fetch per request: the fetch precedes the statement and is the cheap part.
2. No update is lost: an update without the current validator gets 412 PRECONDITION_FAILED, and its latency is the row lock's.
3. With Access down, requests fail closed in about two milliseconds (503 AUTHORIZATION_UNAVAILABLE), with no queue.
4. Version 1.0.0 on PostgreSQL 15.19 measured 991/s, 1251/s, 313/s and 4280/s (p50 15.8, 11.6, 50.8 and 3.6 ms). Runs on different database versions are not comparable.
5. Three consecutive read-heavy runs gave 1510/s, 1571/s and 1371/s, a spread of about a fifth.

Statement shape at scale#

bun run scripts/ops/scale.ts writes a million-row collection, reads a page statement's plan, walks every page and compares the rows with those of a single statement. Pages are selected first and projected afterwards (decision 2), in an index order (decision 3).

The walk over 200 000 rows read 4 000 pages, every row, in database order. Sorting or filtering on a field needs that field's index in the collection's reviewed migration.

Not measured:

1. Several replicas: reads scale with replicas, writes with the database.
2. Worker capacity: integration tests cover claims, leases and retries; watch data_outbox_backlog and data_outbox_oldest_age_seconds.
3. Maximum bounds (a 200-record page, a 50-predicate filter or a 1000-row expansion): contract tests check them and refuse an excess with 422 RESULT_TOO_LARGE.

Deployment declaration#

A deployment declares these items, measures them on its own hardware and publishes the result.

The reference objectives add a 50% margin to the measurement (1 510 and 2 149 requests per second at a p95 of 13.4 and 9.4 ms). A deployment that declares more than its machine holds answers 503 when full.

Compatibility#

The harness runs this repository's real gateway, Access and Data against a real PostgreSQL server.

Integration baseline#

compatibility.json at the repository root names the three service workspaces and the revision they are tested against (spec §1.1).

{
  "version": 2,
  "repository": "https://github.com/ggpwnkthx/platform.git",
  "workspaces": {
    "gateway": {
      "path": "packages/gateway",
      "source": {
        "repository": "https://github.com/ggpwnkthx/platform.git",
        "commit": "dc3d421a5f490594fe012d04eaaef21733736563"
      }
    },
    "access": {
      "path": "packages/access",
      "source": {
        "repository": "https://github.com/ggpwnkthx/platform.git",
        "commit": "dc3d421a5f490594fe012d04eaaef21733736563"
      }
    },
    "data": {
      "path": "packages/data"
    }
  }
}

References, candidates and overrides work as described in platform compatibility: here the gateway and Access are references and Data is the candidate. bun run check:compatibility proves the spec's baseline block agrees with the manifest, and refuses a CI workflow that pins its own revision.

Harness#

bun run --cwd packages/data test:compat   # Data harness
bun run compatibility                     # Access contract and platform acceptance suites
GATEWAY_REPOSITORY=/srv/mirrors/gateway ACCESS_REPOSITORY=/srv/mirrors/access \
  bun run --cwd packages/data test:compat

Through the gateway, a client discovers a collection, creates a record under its own identity, reads the fields it may read, updates with a current validator, observes the audit metadata, and proves another tenant cannot reach the data.

The harness takes each reference's source from the first of these that is available:

1. GATEWAY_REPOSITORY and ACCESS_REPOSITORY: a git URL or a local path, cloned without network,
2. the COMPAT_MIRROR directory of bare gateway.git and access.git mirrors,
3. the local repository,
4. the manifest URL.

A URL with or without .git names the same mirror. GATEWAY_COMMIT and ACCESS_COMMIT must be full revisions; anything else is an error, with no fallback. COMPAT_KEEP keeps the run's temporary directory.

Access extension#

Without Access answering /v1/access-context, Data fails closed: the extension is a required dependency, and it lives upstream (decision 15). To change it, land the change in the Access workspace (keeping the gateway decision contract), then pass the harness against the new revision.

Signature fixtures#

GATEWAY_CHECKOUT=/path/to/platform/gateway \
  bun run --cwd packages/data scripts/dev/generate-fixtures.ts

Gateway code at the repository revision produces the canonical payload and signature, and the fixture records that revision. Legacy v4 is included; since the gateway stopped signing it in 1.0.0, only its key derivation is kept locally.

Platform release 1.0.0#

Platform release 1.0.0 is a major change to the gateway and Access contracts. The Data contract (v1 routes, v5 identity, access context 1), the route table format, the decision protocol and the identity headers are unchanged. Impact on Data:

Release readiness#

A gate is done when it is a repository fact (a running check, a published measurement, a written decision), and outside when the repository cannot hold it. 1.0.0 is published but stays pre-production until both outside gates are met. The suites make requirements checked, not proved: the author wrote every check.

Releasing#

Follow the platform release steps. The tag's workflows publish the image, binaries for seven platforms, the client and checksums. For Data:

1. Update README.md, docs/data/README.md, CHANGELOG.md and docs/data/internals.md: retire the pre-production wording, and write "proved" only where someone else's check proves it.
2. bun run release:tag compares versions numerically (0.10.0 is greater than 0.9.0) and falls back to the workflow's release identity when the checkout has no committer identity.
3. The changelog's "Unreleased" section lists client-visible changes: the database evaluates a write's predicate, legacy identity is refused by default, the client is on a registry and runs on node, and backup works with the deployment's roles.

Security#

The rules are spec §2.3 and §6.4. This section reviews them and everything around them.

Review#

Reviewed with no finding:

Fixed:

Open findings#