AHRI_TRE_RS Documentation

This book is the narrative entry point for AHRI_TRE_RS system documentation. It is published beside generated Rust API documentation so contributors can move between architecture notes and crate-level APIs without switching sites.

The first documentation slices establish the local build, publication channels, implemented architecture, adapter boundaries, governance model, provenance model, data contracts, and the Configuration and Secrets model. Start with AHRI_TRE As A Trusted Research Environment for the high-level TRE/SDE framing and Five Safes mapping. The documentation status matrix marks major areas as implemented, partial, planned, or pre-release so readers can separate current behavior from roadmap intent.

Documentation Channels

The assembled site is channel-shaped from the start:

• dev is for documentation generated from the main development branch.
• stable is reserved for release or version-tag documentation.

Local builds default to the dev channel. The stable channel is reserved for release or version-tag builds and may not exist before the first release.

Use the documentation status matrix when deciding whether a page describes stable release-facing behavior, current main-branch development behavior, or planned work that is not finished yet.

Existing Source Material

The current repository documentation remains the source of truth while the book is expanded:

• Architecture brief
• Database design
• Tabular contract
• Testing
• Ordered backlog

API Docs

The public TRE API contract is the protocol module interface. Start with the Public Protocol API guide or open the generated protocol rustdoc. Workflow implementors can still use the generated app rustdoc for the in-process application service layer.

AHRI_TRE As A Trusted Research Environment

AHRI_TRE_RS is being built as the technical core of a trusted research environment (TRE), also called a secure data environment (SDE). Its purpose is to let approved research happen close to sensitive data while preserving the governance, provenance, and operational controls needed to keep that work accountable.

In this documentation, “trusted” does not mean that the software asks operators or researchers to trust it blindly. It means the system is designed so trust can be made concrete: users authenticate through a known identity path, study access is checked at workflow boundaries, data movement is recorded as provenance, datasets use stable tabular contracts, and public outputs are separated from credential-bearing or sensitive operational state.

AHRI_TRE_RS is therefore not a whole institutional governance programme by itself. Ethics review, data-owner approval, researcher accreditation, disclosure review, and incident response still belong to the institution operating the environment. The system supplies the technical spine that lets those institutional controls be expressed consistently in software.

The Five Safes

The Five Safes framework is a common way to reason about controlled research access to sensitive data. UK Data Service summarises the five dimensions as safe data, safe projects, safe people, safe settings, and safe outputs, and describes the framework as a set of principles for safe research access. The Office for National Statistics also frames the model as a way to maximise research use while keeping data secure and preventing identification.

The five dimensions are deliberately interdependent:

This framing is useful because it prevents over-reliance on any single control. Anonymisation or de-identification is not enough on its own. A TRE needs an approved purpose, trusted users, controlled execution, appropriate data treatment, and governed release of outputs.

AHRI_TRE And Data Visiting

The article “Data visiting governance: a conceptual framework” uses “data visiting” for a model where analysis occurs in the data provider’s controlled computing environment rather than moving the data to the researcher. It argues that data visiting governance should be configurable across several dimensions, including researcher autonomy, data location, data visibility, shared-data nature, output governance, trust and control model, and auditability and traceability.

AHRI_TRE_RS fits naturally into that discussion. It is not only a file store or a metadata catalogue. It is a workflow-oriented environment in which technical choices act as governance levers:

The key design implication is that AHRI_TRE should be read as a configurable research environment, not a single access mode. A high-trust internal analyst may need interactive session-backed commands. A batch workflow may need stateless automation with explicit inputs and JSON output. A future remote researcher may need a narrow protocol or HTTP surface. These modes should vary the amount of autonomy and visibility without changing the underlying governance and provenance model.

How The Technical Design Supports Trust

Separate Governance Concerns

The governance model separates authentication, datastore entry, study authorisation, and DUO restrictions. This separation matters for both the Five Safes and data visiting. A person can be authenticated but not authorised for a study. A study can carry data-use restrictions without those restrictions being treated as access grants. A custodian can administer access without becoming the only representation of research legitimacy.

The current model records study access grants and study custodianship in PostgreSQL metadata. Study custodians can manage ordinary access grants through workflow paths, and destructive or lifecycle operations can record the authenticated TRE user as the actor. This creates an audit trail around who did what, under which study context, instead of leaving those facts in terminal history or informal operating notes.

Keep Data Near The Provider

AHRI_TRE_RS uses a dual-store runtime:

• PostgreSQL stores metadata, governance, access control, provenance, and domain structure.
• DuckDB plus DuckLake stores and queries analytical datasets in the mounted lake filesystem.

This structure supports data visiting because analytical work can happen inside the provider-controlled runtime. The researcher or workflow interacts with controlled commands, protocol requests, Arrow batches, or approved exports rather than receiving broad direct access to every backing store.

The system deliberately treats cross-store work as workflow orchestration rather than pretending PostgreSQL and DuckLake are one distributed transaction manager. That keeps failure handling, cleanup, and provenance visible.

Use Stable Data Contracts

Trusted research environments need predictable boundaries between tools. AHRI_TRE_RS uses:

• Arrow RecordBatch values as the canonical in-memory tabular model.
• Parquet as the canonical persisted analytical dataset format.
• Arrow IPC for binary tabular transport.
• JSON over HTTP for control-plane messages.

These contracts help safe data and safe settings at the same time. They allow Python, R, Julia, CLI, daemon, and future HTTP surfaces to share the same core tabular and control-plane shapes without making any one client dataframe model the system of record.

Keep Infrastructure Behind Adapters

PostgreSQL metadata access is behind the libpq-based metadata adapter. OAuth and OIDC logic are separated from libpq wiring. DuckDB, DuckLake, and Lake location logic are behind the lake adapter. Runtime configuration is typed and explicit.

Those boundaries are safety controls as much as engineering controls. They make it harder for a CLI command, language binding, or future HTTP handler to bypass policy by reaching directly into PostgreSQL or DuckLake. They also help operators reason about where credentials, live handles, Restricted local references, and query execution can appear.

Treat Public Output As A Boundary

Safe outputs are not just final tables in a publication. In a software system, public output includes CLI text, JSON responses, logs, diagnostics, generated artifacts, archive manifests, provenance summaries, exported metadata, and error messages.

AHRI_TRE_RS has a shared sensitive-material policy for this boundary. Secret material is write-only at credential ingress points and must not appear in public responses. Safe credential metadata can be shown when useful. Restricted local references are redacted or bounded by context. Leak-canary tests give this contract executable evidence.

This does not replace human disclosure control for research results. It does make the software boundary safer by default, so operational output review is not fighting accidental credential or path leakage at the same time.

What This Means For Readers

When you read the rest of this book, the details should connect back to this TRE/SDE model:

• System Architecture explains the dual-store runtime and crate boundaries that support safe settings.
• Adapters And Runtime Boundaries explains where PostgreSQL, OAuth, DuckDB, DuckLake, and runtime configuration are contained.
• Governance And Provenance explains study access, custodianship, DUO restrictions, governed querying, and transformation lineage.
• Sensitive Material Handling explains the public-output contract for secrets, credentials, diagnostics, and logs.
• Data Contracts explains why Arrow, Parquet, Arrow IPC, and JSON are the stable interfaces across tools and languages.
• Documentation Status Matrix separates implemented behavior from planned or pre-release surfaces.

The short version is: AHRI_TRE_RS supports the Five Safes by making governance, identity, controlled execution, data treatment, provenance, and safe output boundaries part of the system architecture rather than optional conventions around it.

References

• UK Data Service, What is the Five Safes framework?
• Office for National Statistics, The “Five Safes” - Data Privacy at ONS
• Donrich Thaldar, Data visiting governance: a conceptual framework, Human Genomics, 2025

Documentation Channels

The published documentation site has two public channels:

• dev is built from the latest successful documentation workflow on main.
• stable is built only from release or version-tag events.

The channels are intentionally separate. Development documentation may describe mainline behavior that has not been released yet. Stable documentation should match a released source revision or a version tag and should not drift on every main-branch push.

Pre-release State

Before the first release or version tag is published, the stable channel may not exist on GitHub Pages. In that state, use the dev channel for current project documentation and treat the docs stable badge as the reserved release-facing URL.

The stable publishing workflow preserves the existing dev channel when it publishes stable, and the development publishing workflow preserves the existing stable channel when it publishes dev.

Local builds and CI select the channel through DOCS_CHANNEL; see the documentation and CI controls.

Channel Metadata

Each channel includes channel-metadata.json beside the book and API landing page. The metadata records:

• documentation channel
• source branch or Git ref
• source commit
• source tag when the build came from a tag or release
• release name when GitHub release metadata is available
• build time
• workflow run URL when GitHub Actions provides it
• relative book, API, and rustdoc paths

The metadata is for traceability. It does not replace the repository source or the GitHub release/tag history.

Documentation Status Matrix

This matrix shows whether each major documentation area describes implemented behavior, partially implemented behavior, planned work, or pre-release channel state. Use it with the documentation channel guide: dev documentation may describe current main branch behavior and roadmap context, while stable documentation is reserved for release or version-tag builds and may not exist before the first release.

Status Definitions

Matrix

Channel Choice

Use dev when you need the latest documented repository behavior, current workflow status, or roadmap notes for partial and planned surfaces such as the local daemon, the bounded web control plane, and language bindings.

Use stable when you need documentation that matches a release or version tag. Before the first release, the stable channel may be missing; in that case, read the dev channel and check the matrix status before relying on incomplete surfaces as operator-facing behavior.

System Architecture

AHRI TRE is a governance-aware, provenance-first research environment. A Deployment’s Trusted runtime owns live datastore capabilities, authentication, Secret resolution, governed workflows, and content admission. User-side clients and the Web control plane reach those capabilities through authenticated interfaces.

The datastore combines two stores:

• PostgreSQL holds metadata, governance, access control, provenance, and domain records, as well as the DuckLake catalog.
• DuckDB and DuckLake manage analytical data in configured Lake storage, with Arrow RecordBatch as the in-memory tabular boundary, Parquet for persisted datasets, and Arrow IPC for tabular transfer.

Cross-store work uses explicit admission and compensating cleanup. A PostgreSQL transaction cannot atomically commit Lake files and metadata together.

This chapter describes the implemented runtime and Client parity architecture. The parity completion audit records the qualified CLI/C and Browser artifacts. The final section separates the broader Governed-compute design from those delivered capabilities.

Architecture Views

User View

Researchers, custodians, and automation use authenticated Sessions and Study context. Datastore entry, Study access, custodianship, DUO restrictions, and Asset risk classification remain distinct concerns. Discovery metadata does not grant access to protected content.

flowchart LR
    user["Researcher / custodian / automation"]
    browser["TRE Browser<br/>Discovery and access requests"]
    client["CLI / language binding"]
    session["Authenticated Trusted Session<br/>Execution profile and Current study"]
    workflow["Governed application workflow"]
    catalog["Catalogue and provenance<br/>Studies, Domains, Assets, semantic model"]
    source["Source files / HTTPS / SQL / REDCap"]
    assets["Governed Assets<br/>Datafiles and Datasets"]
    disclosure["Authorized low-risk disclosure"]
    destination["Explicit user-controlled copy"]

    user --> browser
    user --> client --> session --> workflow
    browser --> catalog
    source -->|authorized acquisition and ingest| workflow
    workflow --> catalog
    workflow --> assets
    assets --> disclosure --> destination

System Integrator View

There are three separate entry paths: ordinary user Clients, the Browser’s Web control plane, and trusted operator administration. Their credentials and permissions are not interchangeable.

flowchart TB
    subgraph clients["User-controlled clients"]
        cli["CLI and automation"]
        daemon["ahri-tred<br/>local lifecycle and Managed forwarding"]
        wrappers["Python / Julia / R / other bindings"]
        cabi["Stable C ABI"]
        managed["Managed runtime<br/>Client Session context and HTTPS transport"]
        cli --> daemon --> managed
        cli -->|content and acquisition| managed
        wrappers --> cabi --> managed
    end

    browser["TRE Browser<br/>separate TypeScript application"]
    web["Web control plane<br/>OIDC, cookies, CSRF, HTTP projection"]
    idp["ORCID / OIDC provider"]
    operator["Trusted operator client"]

    subgraph trusted["Deployment-scoped Trusted runtime"]
        api["Authenticated interfaces<br/>user, Web service, local operator"]
        sessions["Runtime logins and live Sessions"]
        app["App-owned workflows<br/>governance, writers, disclosure"]
        adapters["Metadata, Lake and source adapters"]
        api --> sessions --> app --> adapters
        api -->|scoped service / operator capability| app
    end

    pg["PostgreSQL<br/>TRE metadata and DuckLake catalog"]
    lake["Lake storage<br/>Datafiles and Parquet"]
    sources["External sources"]

    browser --> web
    web <-->|browser authentication| idp
    web -->|mTLS service identity and scoped protocol| api
    managed -->|Runtime credential over HTTPS| api
    operator -->|protected local Unix socket| api
    sessions <-->|server-owned OAuth lifecycle| idp
    adapters --> pg
    adapters --> lake
    adapters --> sources

Web has no PostgreSQL or Lake credential, mount, direct adapter dependency, or deployed datastore route. Its readiness check authenticates to the Trusted runtime and checks capabilities. The user-side Managed runtime likewise has no TRE datastore or Lake authority. Client acquisition can access an external source using the user’s source credentials; that is separate from datastore execution. See ADR-0010 and ADR-0015.

Developer View: High-Level Layers

The four parity modules are behavioral boundaries within the existing crates. They concentrate shared decisions without requiring four new crates.

The decisions are ADRs 0020, 0021, 0022, and 0023.

flowchart TB
    cli["CLI / local daemon"]
    ffi["C ABI / language bindings"]
    client["ClientSessionContext<br/>Managed transport and safe context"]
    web["Web HTTP adapter<br/>service / visitor / custodian context"]
    protocol["ahri_tre_protocol<br/>versioned DTOs, references, errors"]
    runtime["Trusted runtime composition<br/>authenticate, retain capabilities, dispatch"]
    app["ahri_tre_app<br/>workflow decisions and disclosure admission"]
    domain["ahri_tre_types + ahri_tre_core<br/>records, policies, repository contracts"]
    metadata["ahri_tre_pgmeta / runtime_pgmeta"]
    lake["ahri_tre_lake / ahri_tre_tabular"]
    sources["ahri_tre_sqlmeta / ahri_tre_redcap"]

    cli --> client
    ffi --> client
    client -->|user protocol| runtime
    web -->|service protocol| runtime
    protocol -.-> client
    protocol -.-> web
    protocol -.-> runtime
    runtime --> app
    app --> domain
    app --> metadata
    app --> lake
    app --> sources

The arrows show responsibility flow. Configuration, Secret capabilities, and runtime composition supply dependencies; workflows do not reopen configuration documents or Secret stores. The protocol crate contains serializable contracts and compatibility rules, with no workflow execution or datastore connections.

Developer View: Protocol Layer Detail

Ordinary JSON control-plane operations use protocol 2.0.0. The active Browser contract is v2 and shares canonical object references. Unsupported protocol majors fail before request-body decoding. Protocol, C ABI, library package, and content-frame versions are independent compatibility dimensions.

An ordinary CLI Session workflow follows:

CLI command
  -> ProtocolRequestEnvelope
  -> local daemon socket
  -> shared ClientSessionContext / Managed transport
  -> authenticated Trusted-runtime HTTPS interface
  -> exact owner-scoped live Session
  -> AppService workflow
  -> safe protocol response

The daemon owns local process lifecycle and forwarding. Only daemon.version and daemon.doctor use local public-protocol dispatch; private shutdown and lifecycle frames remain adapter-local. Missing configuration, an unavailable Trusted runtime, and unsupported remote operations fail without local datastore execution. Automation uses the same trusted workflow authority, with explicit Session and Study scope where required.

The C ABI route follows:

host-language wrapper
  -> opaque C client / Session handle
  -> shared ClientSessionContext / Managed transport
  -> authenticated Trusted-runtime HTTPS interface
  -> AppService workflow
  -> result handle containing response JSON and optional content payload

CLI content/upload/acquisition adapters and C payload calls use separate bounded interfaces on the same Trusted HTTPS origin. They do not put binary content or source credentials into ordinary JSON envelopes. C handle ownership and CLI presentation remain adapter concerns. Full host-language packages live in separate repositories under ADR-0005.

Configuration, Secrets, and Authentication

ahri_tre_config owns versioned Application and Client document parsing, validation, selection, and Effective configuration projection. Application configuration belongs to trusted operators. A generated Client bootstrap contains only schema version, Deployment identity, the canonical Trusted HTTPS origin, and public CA trust. Execution profiles, datastore topology, policies, and Secret references remain server-side.

ahri_tre_secrets owns Secret material, immutable Injected snapshots, and the encrypted Managed-secret store. Configuration contains logical references and declared Injected versions, never resolved material.

flowchart LR
    application["Application configuration"]
    config["ahri_tre_config<br/>validation and projection"]
    effective["Effective configuration<br/>selected graph and fingerprint"]
    bootstrap["TrustedRuntimeBootstrap<br/>immutable retained state"]
    secrets["ahri_tre_secrets<br/>Injected snapshot / Managed store"]
    capabilities["Resolved runtime and operation capabilities"]
    clientdoc["Generated Client bootstrap<br/>Deployment, origin, public trust"]

    application --> config --> effective --> bootstrap
    config --> clientdoc
    secrets -->|verified startup snapshots| bootstrap
    bootstrap --> capabilities
    secrets -->|newly authorized Managed-secret resolution| capabilities

Trusted bootstrap verifies the complete Managed store and retains Effective configuration, the Injected-secret snapshot, and startup-required Managed-secret snapshots before exposing listeners or authentication capabilities. Configuration and Injected changes require restart. A newly authorized operation may resolve its permitted Managed reference again; existing capabilities keep the material they opened. Configuration fingerprints describe non-secret configuration, not Secret values or mutable active Managed versions.

Runtime login is deployment-wide and precedes profile-specific Session opening. The Trusted runtime retains reusable upstream OAuth/OIDC artifacts as encrypted Managed secrets. Clients receive only a short-lived opaque Runtime client credential. Rotation, reauthentication, expiry, and logout preserve login-owner binding and revocation; they do not transfer a live capability to another user. A client-visible credential or reference is never a PostgreSQL credential.

Web receives its own Effective service graph and authentication capabilities; it does not receive the Trusted runtime’s datastore graph or server private key. The operator handler resolves privileged dependencies inside Trusted authority. There is no user-client fallback to administration or infrastructure repair.

Version 1 uses one active Trusted runtime per Deployment. Administration uses the fixed /run/ahri-tre/admin.sock Unix socket, protected by filesystem permissions and kernel peer identity; a Runtime client credential cannot authorize it. Durable Dataset coordination state is separate from disposable trusted scratch, so replacing scratch does not erase writer ownership or unresolved cleanup evidence.

Development tooling, Local TRE, and Integration fixtures have separate lifecycles and authority. Opening the development container starts tools only; the durable Local TRE deployment and disposable Integration services are started explicitly with separate networks, credentials, and storage. See the development environment guide.

Sessions and Public Identity

A live Session exists only while the Trusted runtime retains its capability. The exact client reference is {deployment_id, runtime_login_id, session_id}. A reused friendly name cannot redirect an old reference to a new Session. Current study belongs to that live Session and is resolved and authorized by the application; it is not an executable client-side default.

CLI invocations share safe registry state within one local user, Deployment, and Runtime login. Each constructed C client starts with an independent registry shared by its handles. List and close-all operate on the invoking registry, not every Session owned by the same human. Freeing a local C handle releases memory; explicit remote close ends the shared capability for all attached clients. A lost response never licenses transparent replay.

The client resolves an Execution profile once from an explicit choice, saved preference, or advertised configured default and sends the explicit choice to Trusted open. A failed choice does not fall back. Restart does not recreate live Sessions or their Current study. See the Client Session contract.

Public datastore object references use {datastore_id, kind, id}. Here datastore_id is the durable Datastore UUID, not a configuration key or a connection address. Dataset and Datafile references identify logical Assets; an asset_version reference identifies an immutable revision. References carry scope and type, but grant no access. Names remain convenience selectors: ambiguous names and conflicting Study, Domain, type, or version constraints fail explicitly. Permitted latest-version selections resolve once before planning or admission. See ADR-0025.

Workflow Ownership

Developer View: Application Layer Detail

ahri_tre_app owns the substantive Session and Browser use cases. Trusted adapters authenticate callers, retain capabilities, validate transport framing, and map requests and results. PostgreSQL adapters own statements, row decoding, transactions, and persistence mechanics. The Lake adapter owns DuckDB, container-visible Lake locations, trusted scratch, and physical cleanup.

The workflows cover catalogue, Study/Asset governance, semantic definitions and instances, mappings and links, dictionaries, provenance, archive-first deletion, acquisition, Dataset writers, and governed content. Semantic values and inferred dictionary content use the same disclosure rules as Dataset content; metadata read permission alone does not admit content-derived results. Browser Discovery uses its own restricted safe projections and cannot borrow analyst authority.

Study Access-request Workflow

Web validates its browser session and request-security context, then forwards protocol intent over its authenticated service connection. Trusted validates ready-Datastore scope and supplies narrow application ports through ahri_tre_runtime_pgmeta:

• Discovery uses a read-only metadata authority.
• Requester submission, listing, detail, and withdrawal use a separate least-privilege access-request authority and the server-derived visitor.
• Custodial queue, approval, and rejection use the matching live custodian Session’s libpq OAuth connection and authenticated PostgreSQL actor.

For approval, the application preflights the pending request, provisions or reuses the requester’s workflow entry in that exact Datastore, then records the Study grant and approved decision through the authenticated metadata port. Trusted resolves the workflow-entry Secret at the newly authorized operation boundary. Web owns neither that Secret nor a database connection.

Workflow entry is an idempotent prerequisite, not a Study grant. If provisioning succeeds and the final decision fails, the entry remains for a safe retry; compensating deletion cannot prove that another authority does not use it. Submission and withdrawal grant neither Study access nor workflow entry.

Dataset Writers and Tracked Operations

All supported Dataset writers share durable Output reservations for the Datastore, Output study, and logical Dataset. They contend for the same target across users, Sessions, and name/reference selectors before version allocation or materialization.

flowchart LR
    authorize["Resolve and authorize intent"]
    reserve["Reserve logical Dataset<br/>persist attempt ownership"]
    prepare["Prepare immutable sources<br/>materialize Lake output"]
    admit["Atomic PostgreSQL admission<br/>metadata, provenance, latest version"]
    success["Consume reservation<br/>return safe result"]
    cleanup["Compensate known failed work"]
    release["Release failed attempt's reservation"]
    retain["Retain ownership<br/>until cleanup and executor exclusion are proven"]

    authorize --> reserve --> prepare --> admit --> success
    prepare -->|failure| cleanup
    admit -->|known rollback| cleanup
    admit -->|unknown commit outcome| retain
    cleanup -->|incomplete or executor still active| retain
    cleanup -->|complete and executor excluded| release

Final Dataset metadata and successful provenance commit together. Lake work remains outside that transaction. Unknown commit outcomes retain output and ownership; cleanup must not destroy possibly admitted data. A failed or expired operation alone does not make the target reusable. Successful admission consumes the reservation only when remaining work cannot mutate its committed target.

Managed Datafile-to-Dataset materialization has public Operation tracking, including admission idempotency, status, cancellation, history, and durable result readback. Start acceptance persists its reservation, attempt, pending Operation, and optional idempotency binding atomically; final admission records the typed result in the metadata transaction. Cancellation is cooperative and cannot retract a committed result. Restart interruption requires reconciliation and explicit new work, not automatic Session restoration or materialization replay. Other synchronous writers reuse reservation and compensation ownership without pretending to be detached Operations. See ADR-0018.

Source Acquisition and Governed Derivation

Clients may read an external source with their own authority and upload it, or securely delegate a bounded acquisition to the Trusted runtime. Destination Study authorization is independent of source authentication. A workstation path is opened by the client and never becomes a server path to reopen.

The implemented acquisition families include local files and Arrow/table uploads, HTTPS, client-local DuckDB/SQLite, client or Trusted PostgreSQL/MSSQL, and offline or live REDCap. Input formats and provider restrictions are defined by the upload, HTTPS, SQL source, and REDCap contracts.

User-supplied source credentials travel separately from public JSON and are retained only for that acquisition. They are discarded on completion, failure, cancellation, or expiry, with no Session credential cache or authenticated pool for later work. Named shared sources have separate configured grants and logical Secret references; ordinary Study access does not grant their credentials. Outbound policy, TLS verification, finite budgets, and parser limits apply before admission. TRE metadata credentials never substitute for source credentials.

Ordinary ingest requires explicit Asset risk classification. Low classification requires the owning Study’s custodian and justification; later versions must respect the logical Asset’s classification. Uploaded or client-produced content is not proof of a governed derivation.

dataset.transform is a bounded synchronous governed SQL writer. It resolves all input dependencies and an independent Output Study, evaluates captured immutable inputs in the restricted query worker, and admits high-risk output through the shared writer. It returns safe identities and provenance, not rows or derived counts. It is distinct from both external SQL acquisition and direct low-risk query disclosure. See the Dataset SQL transform contract.

Trusted Content Disclosure

The application resolves complete immutable inputs, including dependencies behind query views and aliases, checks authorization and risk, and captures the admission snapshot. Unknown dependencies, missing classification, unauthorized inputs, or any high-risk input prevent content disclosure to a user-controlled client.

Authorized low-risk content may be returned completely within finite service budgets; those budgets protect reliability rather than create a second disclosure grant. Dataset data, previews, exports, Datafile bytes, query results, and content-bearing semantic responses share this boundary.

Required admission/start evidence is durable before the first byte leaves Trusted control. The protected Governance Study’s append-only, non-content ledger records governance evidence. Delivery completion records outcomes and known rows/bytes without copying content. An unresolved older completion retains recovery state; each new disclosure still needs its own durable admission. Reclassification or revocation blocks new admissions but does not retroactively truncate an already admitted finite stream or recall an earlier client copy.

Trusted transport delivers framed payloads with identity, sequencing, digest, and terminal-evidence validation. EOF without a verified successful terminal is failure. CLI file destinations publish atomically only after verified completion; stdout may already contain partial bytes when a transfer fails. C results support bounded incremental reads or explicit bounded materialization, with independent result-buffer ownership. Managed buffering creates no implicit persistent cache. See the content transfer contract.

Lake locations, storage credentials, and trusted scratch stay inside the trusted boundary. Clients identify returned copies as low-risk / user-controlled; this notice is not another approval step. Legacy unclassified Assets require the explicit preserving governance upgrade, which classifies them high and records evidence while retaining safe metadata access.

Crate Boundaries

PostgreSQL OAuth retains libpq. The existing password-authenticated Browser Discovery adapter may use Rust postgres, but remains behind ahri_tre_pgmeta and is invoked from Trusted composition. Neither Web nor domain types acquire a database backend through that exception.

Datastore Identity Binding

There is no central registry database. A datastore-local binding records its Datastore identity, generated durable UUID, owned physical stores, encryption policy, credential references, and lifecycle state. The Datastore identity is the PostgreSQL metadata database name; the Datastore configuration ID selects an Application declaration; the public Datastore identifier is the generated UUID. These values have different purposes.

An Execution profile selects one configured Datastore. Trusted constructs the Session plan from retained configuration, opens the selected metadata database, checks its binding and DuckLake state, and passes the resolved capability to application workflows. A client supplies profile intent, not a physical database or Lake profile. Creation, adoption, and storage maintenance remain privileged workflows. Lake locations come from Effective configuration and verified binding state and must be container-visible; host-side Restricted local references belong to deployment tooling.

Metadata Identity Ownership

Ordinary new persistent identifiers are datastore-owned. Creation requests describe the object; metadata adapters omit generated IDs and return the persisted record. PostgreSQL identity sequences and UUID defaults provide the identities. Dataset and Datafile subtype rows use the datastore-assigned Asset-version ID rather than another application-generated identity.

Fixtures, import/replay paths, reference seeds, natural keys such as DUO codes, composite joins, and temporary staging names are deliberate exceptions. Public Datastore-bound references encode existing identities without requiring a wholesale primary-key migration.

Diagnostics and Qualification

Operational diagnostics report bounded safe observations with acquisition time and evidence scope. A retained Session-open observation is historical evidence, not a fresh database or Lake probe. Correlation IDs connect requests and Operations without granting authority or replay rights. Operational events omit credentials, content, raw SQL, private paths, and arbitrary adapter errors. Their bounded best-effort delivery is separate from the durable evidence required for governance and disclosure. See Diagnostics and Observability.

The completed parity PRD qualifies the integrated workflows through the recorded installed Linux aarch64 CLI/C artifacts and matching protocol-v2 Browser companion. The inventory distinguishes ordinary-user, Browser, operator, local, and bounded transfer surfaces. Actual capability advertisement also reflects deployment configuration, including whether tracked operation execution is enabled.

This evidence does not qualify every native platform or deployment image. Release handoff, large/slow ingest resume, and the separately planned concurrent Asset-access hardening retain their own backlog and validation scope.

Target Architecture: Trusted Runtime and Governed Compute

The Trusted boundary, disclosure policy, shared writer ownership, protected Governance ledger, and bounded restricted SQL evaluation above are implemented. The broader Governed-compute system in ADR-0009 remains a distinct target: a general job scheduler, arbitrary pipeline workers in a separate trust zone, reviewed fixed transformations, and governed release/publication workflows are not implied by Client parity or the current private query worker.

flowchart LR
    client["User-controlled Client / JupyterHub server"]
    trusted["Trusted runtime<br/>authorize and admit governed work"]
    worker["Governed compute worker<br/>isolated pipeline job"]
    output["Declared governed outputs"]
    release["Authorized Export / Repository transformation"]
    external["Approved external destination"]

    client -->|job intent| trusted
    trusted -->|explicit inputs and short-lived Job capability| worker
    worker --> output --> trusted
    trusted --> release --> external

The accepted target gives workers explicit read-only inputs and private staging, without raw Lake mounts, datastore credentials, Managed secrets, or unrestricted network access. Arbitrary user pipelines produce high-risk outputs; a reviewed fixed trusted transformation may derive low-risk output only under its immutable input/output and risk rule. High-risk content leaves TRE control only through an authorized release workflow. A low-risk input does not weaken the boundary when another input is high risk.

JupyterHub is a user-side client environment. Its trusted identity broker may project a bounded Runtime credential into the user server, while upstream OAuth artifacts and datastore authority remain protected. Image publication and qualification are separate delivery gates. Pilot, test, staging, and production are separate Deployments with independently provisioned identities, Datastores, Lake storage, and Secrets; promoting immutable software does not promote data or credentials.

Adapters And Runtime Boundaries

AHRI_TRE_RS keeps infrastructure behind adapter crates so the workflow model can stay stable while PostgreSQL, DuckDB, OAuth, and runtime packaging details evolve.

PostgreSQL Metadata

PostgreSQL is the metadata store. The metadata adapter boundary is owned by ahri_tre_pgmeta.

The adapter is responsible for:

• connecting from explicit PostgreSQL server and authentication inputs resolved by the Trusted runtime
• bootstrapping and checking the canonical metadata schema
• exposing typed read models for studies, governance, domains, variables, assets, versions, datasets, datafiles, transformations, tags, and mappings
• exposing write primitives with operation-specific errors
• keeping SQL and schema details out of domain records and application request types

Application workflows should call metadata capabilities through the app and adapter boundaries. Shared type crates should not hold live connections, transactions, libpq handles, or connection pools.

Datastore identity binding is datastore-local metadata, not a separate repository service. Ordinary opens connect to the PostgreSQL database named by logical Datastore identifier, read the binding from that database, and verify the binding before returning a runtime capability. Privileged creation obtains physical database authority only from Application configuration inside the Trusted runtime.

PostgreSQL OAuth

OAuth/OIDC session semantics are separated from PostgreSQL wiring.

ahri_tre_orcid owns pure OAuth/OIDC contract types and validation semantics. The Trusted runtime owns reusable authentication artifacts through owner-bound Managed-secret capabilities; public records retain only safe identity and exact-version provenance.

ahri_tre_libpq_oauth owns the PostgreSQL 18 libpq bearer-token bridge. It installs the required authdata hook, passes the selected ID token to libpq, and returns a redacted PostgreSQL metadata connection only after the connection has succeeded.

This split is deliberate. Authentication flow code should not leak into the metadata read/write model, and metadata SQL should not know how a token was obtained.

DuckDB, DuckLake, And Lake Locations

DuckDB plus DuckLake is the analytical lake side of the runtime. The lake adapter boundary is owned by ahri_tre_lake.

Runtime code must not embed Restricted local references. Configured creation passes the container-visible filesystem base or object namespace to the Lake adapter, which derives the Datastore-owned path. Ordinary identity-bound opens receive the persisted Lake location instead of reading ambient path variables.

The lake adapter is responsible for:

• creating and validating the local lake layout
• loading DuckDB extensions needed by the Lake location
• attaching DuckLake through PostgreSQL-backed catalog credentials, preferably materialized as session-local managed secrets resolved from the datastore binding
• detecting persisted DuckLake encryption behavior
• returning health information and diagnostics to the application layer
• executing lake operations without moving DuckDB/DuckLake details into domain crates

The application layer coordinates metadata and lake work. It must treat cross-store operations as workflow steps with observable outcomes, not as implicit ACID transactions across PostgreSQL and DuckLake.

Configuration and Secret capabilities

ahri_tre_config owns Application and Client parsing, validation, selection, Effective projection, origin tracking, and fingerprints. The Trusted runtime retains one immutable bootstrap state containing the Effective configuration, the startup Injected-secret snapshot, and the startup Managed-secret snapshot. A separate narrow Managed-store capability permits re-resolution only for a newly authorized operation. The runtime passes resolved settings and narrow Secret capabilities to adapters; adapters do not reopen documents or recover authority from process environment.

Ordinary operations use logical Datastore and Execution-profile identifiers. The Trusted runtime resolves those identifiers against its Application snapshot, authorizes the request, and supplies only the selected physical inputs. User-side Managed runtimes and bindings receive the Client bootstrap, not server topology or Secret references.

See Configuration And Secrets for selection, projection, fingerprint, and ownership rules.

Authentication

Authentication is split by trust boundary.

• Browser identity is validated by the Web service using configured issuer, client, redirect, TLS, and service Secret capabilities.
• Direct-IDE clients obtain a short-lived runtime credential through the local login socket; Managed runtime renewal is bounded and coordinated.
• Brokered clients receive a non-renewable projected credential with an absolute lifetime.
• OAuth and runtime authentication artifacts are stored as owner-bound Managed secrets. Public records retain safe identity and exact version provenance.

The Client bootstrap contains only Deployment identity, Trusted-runtime origin, and TLS trust. Tokens, client secrets, passwords, caches, credential file paths, and live handles never enter shared configuration or public protocol types.

Replacement is an owner transaction. Session and Datastore reference evidence blocks removal while a persisted authority still depends on a version. Unknown or unavailable reference inspection fails closed.

User Guide

Choose the route that matches your responsibility. These pages prefer concrete commands and decision points over API internals.

For data analysts

Start with Getting Started when you have received access to an AHRI TRE and need to select a profile, authenticate, open a Session, discover governed Studies and assets, and run basic Datastore commands. CLI How-To and the HDSS workflow provide end-to-end examples.

Analysts should not need Docker, PostgreSQL administrator credentials, direct Lake access, or the development environment. Ask the TRE operator for the installed Client bootstrap, admitted Runtime login, profile names, and Study access.

For system integrators

Use the System Integrator Guide to distinguish repository testing, non-production Local TRE evaluation, and the controls that an independently designed staging or production environment must supply.

For developers

Use the Developer Guide for the VS Code container, ordinary Rust validation, disposable Integration fixtures, Local TRE profiles, and the replacement-environment migration boundary.

Installed package workflows

Start with Installation when you have received an AHRI TRE runtime package and need to install the CLI, daemon, runtime libraries, and generated Client bootstrap.

Use Getting Started when you have an installed AHRI TRE CLI package and need to select an operator-published profile, use an authorized Session, and run basic Datastore commands.

Use CLI How-To when you need deeper command examples from a checkout, local Managed-runtime diagnostics, or authenticated protocol operations against a configured datastore.

Use Importing A REDCap Project when you need to bring a complete REDCap project into an existing study and domain, including source artifact preservation, dictionary registration, and form-dataset materialization.

Use Creating Datastores when you need to provision a new datastore identity and its underlying metadata database, DuckLake catalog database, and lake directory for examples or validation.

For reference tables, see Configuration And Secrets. For workflow output details, see Example Workflows. For package maintainers, see the Developer Installation Package Guide.

Installation

This chapter is for people installing an AHRI TRE package provided by an operator, release maintainer, or language-wrapper package. It does not require a Rust checkout, Cargo, or the release packaging toolchain.

AHRI TRE installation gives you the local runtime:

The package does not install or create PostgreSQL, DuckLake catalog databases, lake storage, Docker services, or organisation-specific credentials. Your operator owns those infrastructure details and should tell you which datastore name to open.

Choose The Right Package

Use the package that matches the computer or container where you will run the CLI:

Native Windows packages are not currently provided. Windows users should use a Linux environment such as WSL2 or a supported Linux container runtime, according to local operator guidance.

Unpack The Package

Create a private working directory and unpack the tar archive:

mkdir -p ~/Downloads/ahri-tre
tar -xf ahri-tre-<version>-<target>.tar -C ~/Downloads/ahri-tre
cd ~/Downloads/ahri-tre/ahri-tre-<version>-<target>

Replace <version> and <target> with the values in the package filename. The unpacked directory should contain bin/, lib/, include/, share/ahri-tre/, and install.sh.

Install To A Prefix

Choose an install prefix that your user account can write to. For a personal install, ~/.local/ahri-tre is usually enough:

./install.sh "$HOME/.local/ahri-tre"

The installer copies package files into the prefix, stores the runtime binaries under libexec/ahri-tre, and writes launchers into bin/. The launchers set the packaged runtime library path before starting ahri-tre or ahri-tred, so you should not need to point commands at Cargo target directories or source checkout paths.

Add the installed bin directory to your shell PATH:

export PATH="$HOME/.local/ahri-tre/bin:$PATH"

To make that change persistent, add the same line to your shell profile, for example ~/.profile, ~/.bashrc, or ~/.zshrc.

Check The Install

Run local checks before connecting to a datastore:

ahri-tre version
ahri-tre doctor --format json
ahri-tre schema list --format json
ahri-tred --help

If your shell cannot find ahri-tre, check that the install prefix bin directory is on PATH. If the command reports that a shared library cannot be loaded, run the installed launcher from <prefix>/bin/ahri-tre rather than a binary copied from the unpacked package.

doctor should finish cleanly or return an actionable warning. If it reports that a retired operational environment variable is process-visible, unset it. Configuration and credentials must come from the operator-provided Client bootstrap and the Injected/Managed Secret subsystem.

Install The Client Bootstrap

Ask your operator for the projected Client bootstrap and an admitted Runtime login. The Client document contains only Deployment identity, the canonical Trusted-runtime HTTPS origin, and Deployment CA trust. It contains no credentials or physical Datastore/Lake configuration.

Deployment tooling normally installs the document at /etc/ahri-tre/client.toml. If your operator provides a different protected path, select it explicitly on each command:

ahri-tre --config /protected/handoff/client.toml daemon start

Explicit selection never falls back. The package does not contain or load profile dotenv files, secret overlays, token files, passfiles, or endpoint overrides.

Start Using AHRI TRE

After installation and configuration, continue with Getting Started to start the daemon, open a session, and run datastore commands.

For language-wrapper packages, use the wrapper’s own installation instructions. Wrappers may reuse this installed runtime by locating the packaged C ABI library in <prefix>/lib, the public header in <prefix>/include, and the daemon launcher at <prefix>/bin/ahri-tred.

Getting started

Prerequisites

Ask your TRE operator for a projected Client bootstrap and confirmation that your OIDC identity is admitted. The Client document contains Deployment identity, Trusted-runtime origin, and TLS trust, but no credentials.

Check the local installation

ahri-tre version
ahri-tre doctor --strict

Top-level version and doctor work without configuration, a daemon or login. doctor reports read-only local build and filesystem observations, their times and evidence limits. A successful local report does not establish remote readiness. Skipped checks suggest existing diagnostic commands for the next step; doctor does not run them. Neither command contacts the Trusted runtime.

doctor --format json includes the same findings and next steps as text. Any error exits 2; a required warning exits 1 under --strict; other reports exit 0.

Authenticate and start the Managed runtime

ahri-tre --config /etc/ahri-tre/client.toml auth login
ahri-tre --config /etc/ahri-tre/client.toml auth status

Open the authorization URL printed by auth login. The Trusted runtime owns the callback, validates the OIDC response, and retains the upstream OAuth artifacts. The Managed runtime receives only its short-lived opaque credential at the fixed ephemeral path.

ahri-tre --config /etc/ahri-tre/client.toml daemon start
ahri-tre daemon version
ahri-tre daemon doctor
ahri-tre daemon status

Explicit selection never falls back to a different document, local daemon endpoint, dotenv file, or process environment.

The daemon commands after start report the local Managed-runtime lifecycle, protocol compatibility, socket, state, and Session journals. A rendered or parseable Client bootstrap is not evidence that the client can reach the Trusted runtime. Local diagnostics deliberately do not claim client ready; that requires a successful authenticated remote protocol operation using the operator-admitted Runtime login.

Work in a configured profile

Select an execution profile through the authenticated Trusted-runtime protocol. The profile determines the allowed Datastore and workflow dependencies. Once a Session is authorized, ordinary domain, study, asset, dataset, transformation, and ingest commands use that Session capability.

Use ahri-tre --help and ahri-tre <command> --help for the installed command surface. Stop the managed daemon with ahri-tre daemon stop when finished. End the Runtime login with ahri-tre --config /etc/ahri-tre/client.toml auth logout.

CLI how-to

The CLI is a protocol client. It does not build database connections, discover local runtimes, load dotenv profiles, cache bearer tokens, or accept credential and endpoint overrides.

Client-side checks

ahri-tre version --format json
ahri-tre doctor --strict --format json
ahri-tre --config /etc/ahri-tre/client.toml auth login --format json
ahri-tre --config /etc/ahri-tre/client.toml auth status --format json
ahri-tre --config /etc/ahri-tre/client.toml daemon start --format json
ahri-tre daemon version --format json
ahri-tre daemon doctor --format json
ahri-tre daemon status --format json
ahri-tre --config /etc/ahri-tre/client.toml auth logout --format json

auth login prints the browser authorization URL to standard error so JSON standard output remains one final safe status envelope. Upstream OIDC tokens, the polling capability, and the Runtime credential are excluded from normal output. auth status reports only safe Deployment, OIDC identity, and expiry metadata; auth logout revokes server authority before removing the ephemeral credential file.

Top-level version and doctor inspect only the local installation and do not select the Client document. daemon start selects and retains the immutable Client bootstrap; later daemon readiness commands inspect that local Managed runtime. Neither rendering the document nor local daemon diagnostics proves Client readiness. Record that only after a successful authenticated stable-protocol operation reaches the configured Trusted runtime. Failures are explicit; there is no local execution fallback.

Authenticated Sessions

With Runtime login and the Managed runtime active, select only logical profile and Session names:

ahri-tre execution-profile list
ahri-tre execution-profile select research
ahri-tre session open analysis
ahri-tre session status analysis
ahri-tre session close analysis

The Trusted runtime resolves the Execution profile and Datastore binding. The CLI receives safe identities, not database topology, Lake locations, Secret references, tokens, passwords, or reusable store handles.

Operator checks

Trusted operators use an Application document:

ahri-tre --config application.toml config validate
ahri-tre --config application.toml config show-effective \
  --for execution-profile --profile research
ahri-tre --config application.toml config preflight \
  --for execution-profile --profile research

Preflight reads only the target’s required Secrets and performs bounded checks. It never creates or repairs infrastructure.

Workflow commands

After profile selection and Session authorization, workflow commands operate on logical identifiers. Use the checked-in examples/cli_validation/governed-lifecycle-smoke.sh only against a disposable configured Datastore.

Creating Datastores

Datastore creation is a privileged Trusted-runtime operation. PostgreSQL, Storage, Lake, scratch, identity, and Secret requirements must already exist in the selected versioned Application document.

Validate and preflight one logical Datastore before mutation:

ahri-tre --config application.toml config validate
ahri-tre --config application.toml config preflight \
  --for datastore-create --datastore research
ahri-tre datastore create research --format json

The request carries only the Datastore configuration ID. It cannot override physical topology, paths, roles, credentials, Secret roots, or reset policy. Version 1 refuses existing, failed, or partially created components instead of adopting or force-resetting them.

Creation establishes the PostgreSQL identity binding and durable Managed-secret reference evidence before publishing readiness. Retry and compensation follow the authoritative owner state; uncertain commits fail closed.

For remote deployments, mount Storage at the exact container-visible path named by Application configuration before preflight. Restricted local references are never sent to runtime code.

Datastore schema migrations

The clean version-1 baseline owns one canonical metadata schema marker. Privileged Datastore creation bootstraps that schema through the libpq adapter.

Production clients do not receive direct migration, adoption, or reset authority. A database with missing, unexpected, or historical migration state fails preflight or startup. Repair it through a reviewed deployment migration, or provision a new configured Datastore and move governed data explicitly.

This keeps schema mutation behind Trusted-runtime authority and prevents user processes from recovering administrator credentials or direct database handles.

Moving Datastore Lake locations

The legacy in-process Lake move command has been removed. A Datastore binding owns its persisted canonical namespace, and runtime opens verify it against Effective configuration.

Changing an Application document does not move existing data. Treat a physical move as a reviewed deployment migration: quiesce writers, copy and verify data with infrastructure-owned tools, establish a new immutable binding or Datastore, and update configuration only after the authoritative state agrees. Runtime code never accepts a caller-selected source or destination path.

System Integrator Guide

This guide separates repository testing from the work of creating AHRI TRE testing, staging, and production environments. This repository provides application artifacts, protocol and configuration contracts, a disposable Integration fixture, and a durable non-production Local TRE. This repository does not provide production infrastructure or claim that Local TRE is a deployment template.

Choose the environment

Do not promote Integration volumes or Local TRE volumes into another environment. Promote reviewed, immutable release artifacts and versioned configuration instead.

Inputs supplied by this project

• the Rust binaries, libraries, public protocol, and C ABI described in the installation and API guides;
• versioned Application and Client configuration contracts;
• PostgreSQL metadata and DuckLake adapter behavior;
• configured container-visible Lake location as the canonical container-visible filesystem Lake location;
• JSON-over-HTTP control-plane behavior and OAuth/OIDC boundaries;
• release-versioned TRE Browser compatibility metadata; and
• validation commands and synthetic fixtures that contain no production data.

The Configuration And Secrets, Authentication, Sensitive Material Handling, and System Architecture pages define the application-facing boundaries. They are not infrastructure-as-code.

Create a testing, staging, or production environment

AHRI TRE does not select Kubernetes, virtual machines, a cloud, or an infrastructure-as-code tool for the integrator. The procedure below is the provider-neutral deployment plan that the integrator must express in the organization’s chosen platform. A deployment is not made by copying Local TRE: it is made by provisioning the following boundaries and satisfying the same Application configuration contract.

Create testing, staging, and production as separate logical Deployments. Each must have its own Deployment UUID, DNS names, PKI, OIDC registration, Application configuration, Injected secrets, Managed-secret root identity and store, PostgreSQL databases and roles, Lake storage, audit history, and backup policy. There is no testing, staging, or production switch in the application.

1. Record the deployment specification

Before creating resources, put a reviewed, non-secret specification under change control. At minimum, record:

• the environment name and immutable Deployment UUID;
• release artifact versions, checksums or image digests, and supported CPU architecture;
• the Trusted-runtime and Web HTTPS origins and certificate authorities;
• the OIDC issuer, public client identifiers, exact Web callback URI, and allowed audiences;
• PostgreSQL server, database, TLS, role, and backup design;
• the container-visible Lake base path, storage provider, encryption, capacity, retention, and recovery design;
• execution profiles, compute and image policy, scratch capacity, ingress, egress, and network-policy decisions; and
• service ownership, monitoring, recovery objectives, maintenance windows, and approval evidence.

Use synthetic data and test identities in testing. Staging should reproduce the production topology, trust path, policy, and upgrade procedure without sharing production data or credentials. Production uses only organization-approved services and controls. Capacity may differ, but security boundaries must not be silently removed in a lower environment.

2. Provision the platform boundaries

Translate the specification into reviewed infrastructure code and create:

1. Private data services. Provision TLS-enabled PostgreSQL, the metadata and DuckLake catalog databases, narrowly scoped administration/runtime/Web roles, and durable Lake storage. Mount the Lake at the exact absolute path that will be declared in Application configuration; inside trusted workloads this is the canonical configured container-visible Lake location boundary. Do not expose PostgreSQL or the Lake publicly.
2. Trusted service capacity. Provide persistent encrypted storage for /var/lib/ahri-tre/secrets, separately replaceable storage for /var/lib/ahri-tre/secrets-rotation, and encrypted local scratch. Only trusted runtime and operator workloads may receive these mounts or datastore administration capability.
3. Secret projections. Configure the platform secret store to project each injected://namespace/name beneath /run/secrets/namespace/name/ as a protected value file and a non-secret version file. The version must exactly match expected_version in configuration. Separately generate and back up the Deployment’s X25519 root identity, then project its value at /run/secrets/ahri-tre/root-identity/value. Never store the identity beside a backup of the encrypted Managed-secret store.
4. Network and Web edge. Put the static TRE Browser and ahri-tre-web behind one public HTTPS origin. Route the Web service through a private TLS listener, validate its certificate at the proxy, preserve Origin, X-Request-Id, and X-Protocol-Version, and allow only explicitly reviewed service-to-service and egress paths.
5. Operator and user separation. Give deployment automation a distinct trusted operator identity. User-controlled development, Jupyter, and pipeline workloads must not receive the Application configuration, Managed-secret store, root identity, raw Lake authority, or datastore-internal credentials.

Runtime mount contract

Project only the capability each workload needs. Platform-specific source names are deployment details; the container-visible targets and trust split are the application contract.

Run the Trusted runtime and trusted Secret-administration workloads under one deployment-dedicated service UID and GID. The effective service UID must own the Managed store; use the dedicated group only where read sharing is required. Managed-store directories may grant at most 0750, and regular files at most 0640. Neither may grant world access or group write. Unexpected ownership, excessive modes, inaccessible required paths, or unsupported locking is fatal; AHRI TRE reports safe metadata and never changes ownership or permissions.

Both Secret tiers limit each decrypted value to 64 KiB. Oversized Injected values are rejected before they are returned, and oversized Managed values are rejected before persistence. Treat this as an interface bound, not a prompt to split a larger credential across multiple Secret references.

Root rotation is the only workflow that also receives the next identity at /run/secrets/ahri-tre/root-identity-next/value and an empty writable staging store at /var/lib/ahri-tre/secrets-rotation. Long-running services must never receive either capability.

PKI and public trust

Keep the CA signing key outside AHRI TRE workloads. Inject only each service’s leaf certificate and private key into the service that terminates that identity. Place the public Client CA chain in Application configuration so config render-client can project it into the Client bootstrap. The Web edge has its own reviewed public/private TLS termination path; private service hops still validate certificate chains and hostnames. Development-generated Local certificates are not production trust material.

The Web Control Plane Datastore Deployment guide gives the current Web-to-Runtime topology, identity and authority split, same-origin proxy rules, and single-instance session-store limitation. If the intended service topology requires a capability that the selected AHRI TRE release does not implement, treat that as a release blocker rather than inventing a privileged side path.

3. Install one reviewed release

Build or obtain every executable and frontend artifact from one reviewed release. Verify checksums, signatures where supplied by the release process, architecture, compatibility metadata, dependency inventory, and vulnerability policy before installation. Pin immutable versions or image digests; never deploy latest, a developer worktree, a Local TRE volume, or an Integration fixture.

The Developer Installation Package Guide documents the current runtime package artifacts and their verification surface. The TRE Browser is a separately versioned companion artifact and must match the compatibility metadata of the selected backend release.

4. Author and validate Application configuration

Create one authoritative /etc/ahri-tre/config.toml for the Deployment. Start with the generated schema and minimal document; use the maintained complete example only as a structural reference because its endpoints, certificates, identities, and Secret references are placeholders:

ahri-tre config schema --format json > application-v1.schema.json
ahri-tre config init --output config.toml
# Edit config.toml from the reviewed deployment specification.
ahri-tre --config config.toml config validate
ahri-tre --config config.toml config show-effective \
  --for trusted-runtime
ahri-tre --config config.toml config show-effective --for web
ahri-tre --config config.toml config show-effective \
  --for execution-profile --profile research

The complete reference is examples/config/application-v1.toml. Replace every placeholder and remove unused declarations. Review the safe Effective output and configuration fingerprint, but keep the authored document and its provenance as the authority. Do not replace configuration with a set of environment variables.

After the required Secret projections and dependency endpoints have been staged, preflight each path that will be started or administered:

ahri-tre --config config.toml config preflight --for trusted-runtime
ahri-tre --config config.toml config preflight --for web
ahri-tre --config config.toml config preflight \
  --for execution-profile --profile research
ahri-tre --config config.toml config preflight \
  --for datastore-create --datastore research
ahri-tre --config config.toml config preflight --for secret-administration

This command uses the same selected Effective configuration and completeness rules as startup. It reads only the target’s required Secret material and performs bounded, safe dependency checks; it never creates, repairs, migrates, or otherwise mutates infrastructure. A successful result means those deployment-owned prerequisites were observable at that moment. It does not authenticate a TRE user, authorize a workflow, or prove service readiness. Trusted-runtime preflight reports authenticated client transport separately as not applicable, so it cannot be interpreted as client ready. Use --format json for exactly one machine-readable result document.

Render the public Client bootstrap from that same document and distribute it read-only to managed clients. It contains the Deployment identity, canonical Trusted-runtime origin, and public CA chain, not service topology or secrets:

ahri-tre --config config.toml config render-client --output client.toml

5. Bootstrap secrets and datastores

Run the following as a trusted, auditable operator workload with the final configuration, Injected-secret projection, root identity, and persistent Managed-secret volume mounted at their production paths:

ahri-tre --config /etc/ahri-tre/config.toml secrets init
ahri-tre --config /etc/ahri-tre/config.toml \
  secrets generate managed://postgresql/runtime-password --generator password
# Add every other managed:// reference declared by the selected service graphs.
ahri-tre --config /etc/ahri-tre/config.toml secrets verify --all

Use secrets put <reference> with its protected terminal prompt, or secrets put <reference> --stdin from a bounded secret-delivery mechanism, when a value is supplied externally. Do not pass a value as an argument or capture it in logs. secrets init is create-only and must not be rerun against an existing store.

Credential replacement is owner-routed. Rotate a generated Datastore database and DuckLake-catalog credential with only its logical identifier:

ahri-tre datastore rotate-lake-credential research

The Trusted runtime stages the new version, applies and verifies it at PostgreSQL, then activates it and removes the prior ciphertext. To remove an obsolete Managed Secret, select the intended Application configuration explicitly:

ahri-tre --config /etc/ahri-tre/config.toml secrets remove <managed://reference>

The request is bound to the selected Deployment UUID. Removal fails closed unless the immutable Application, durable and currently inspected Datastore bindings, durable Session records, and authentication-artifact state all prove the reference is absent; there is no force bypass.

Start the Trusted runtime with the final mounts and configuration, then use its protected local administration interface to reconcile each predeclared Datastore. For the research declaration in the reference configuration:

ahri-tre-runtime --config /etc/ahri-tre/config.toml
# From the trusted operator workload that can reach the runtime's local socket:
ahri-tre-runtime datastore reconcile research

Reconciliation creates an absent Datastore or validates its retained ready binding. It does not authorize copying a database or Lake from another logical Deployment. Apply release-supported migrations and datastore role provisioning before making a datastore visible to Web.

6. Start services and qualify the deployment

Start trusted services only after configuration validation, complete secret verification, PostgreSQL TLS, writable trusted scratch, Lake mounts, and datastore reconciliation succeed. Start ahri-tre-web on a private address, then enable the same-origin proxy and Browser only after /health, /ready, and safe /diagnostics checks pass. Follow the Web deployment guide for exact readiness semantics: /health alone does not prove datastore access.

Startup and readiness gates

The diagnostics operator guide documents request joins, evidence fields, exit/status meanings, bounded partial results and the tested Local TRE investigations for these gates.

These gates answer different questions and are not interchangeable:

A rendered Client bootstrap is only a validated artifact. Distribution, reload, and a successful authenticated remote operation are required before recording Client readiness. Likewise, ./dev-env doctor checks the contributor host and tooling; it is not product preflight, recovery verification, or service readiness.

Before accepting the environment, exercise with non-production records:

• TLS chain and hostname failures as well as the successful paths;
• OIDC login, callback, logout/expiry, and audience enforcement;
• expected datastore discovery and rejection of undeclared or unauthorized datastores;
• creation/opening of a Session and the configured Lake and catalog boundary;
• Study access request, custodial decision, audit, and provenance behavior;
• backup and exact same-Deployment restore of PostgreSQL, Lake data, configuration, encrypted Managed secrets and audit journal, with the root identity restored through its separate channel; and
• monitoring, alerting, restart, rollback, credential rotation, capacity exhaustion, and disaster-recovery runbooks.

Record the release identities, Deployment UUID, configuration fingerprint, test results, approvers, and exceptions without recording credentials, tokens, personal identities, Restricted local references, or restricted data.

7. Promote releases, not environments

Promote the same reviewed artifact digests from testing to staging and then to production. For each target, independently provision infrastructure, author and validate its configuration, inject its secrets, initialize its Deployment-bound Managed store, reconcile its datastores, and run its qualification gates. Never promote a Deployment UUID, root identity, secret-store ciphertext, database volume, Lake volume, OIDC credential, or private key between environments.

An update follows the same path: qualify the candidate in testing, rehearse the upgrade and rollback in staging from a representative non-production backup, approve the change, apply it to production, and verify readiness plus critical user journeys. Rollback must use the release’s documented compatibility and data-migration rules; replacing binaries alone is not a recovery plan.

Credential and trust rotation

Managed and Injected credentials

Use only the owning workflow for a Managed credential. secrets put and secrets generate are create-only; there is no generic overwrite or version selector. For a generated Datastore database and DuckLake catalog credential, invoke the protected owner route:

ahri-tre datastore rotate-lake-credential research

The Trusted runtime stages the successor, updates and verifies PostgreSQL, activates the new ciphertext, and removes the prior active ciphertext. A failure or uncertain commit fails closed and preserves recovery evidence. Already-open Sessions retain the exact versions with which they opened; new authorized operations resolve the new active version. Authentication artifacts are replaced through their authentication owner transaction, not by editing store files.

For an Injected secret, deployment tooling creates a new external version, updates the corresponding expected_version in Application configuration, projects the matching value and version files, validates/preflights the selected paths, and restarts every consumer. Processes snapshot Injected material at startup and never live-reload it.

Client CA rotation

Rotate the Client trust CA with an overlap:

1. Add the new public CA chain beside the old chain in Application configuration, validate it, render the replacement Client bootstrap, and distribute it atomically.
2. Restart every Managed runtime so its immutable bootstrap trusts both roots. Confirm distribution; rendering alone is not reachability evidence.
3. Issue and project a new Trusted-runtime leaf certificate and key chaining to the new CA, update its declared Injected version, and restart the Trusted runtime.
4. Prove authenticated Client operations through the new service certificate.
5. Remove the old CA from Application configuration, render and distribute the reduced bootstrap, restart clients again, and retire the old CA under the organization’s PKI policy.

A routine leaf renewal beneath an unchanged trusted CA does not require a bootstrap change, but the service still needs the newly versioned Injected projection and restart. A client that missed an advance trust update fails closed; the server cannot replace trust anchors over the connection they authenticate.

Managed-store root rotation and rollback

Root rotation is offline and non-destructive:

1. Quiesce every Secret consumer and writer. Revoke temporary diagnostic access, take and verify the current backup, and keep the old store and old identity as a matching recovery pair.

2. Mount the active store read-only at /var/lib/ahri-tre/secrets, an empty distinct writable staging volume at /var/lib/ahri-tre/secrets-rotation, and the current/next identities at their fixed paths.

3. Run:

   ahri-tre --config /etc/ahri-tre/config.toml secrets rotate-root
   

   Success means only that the staged store was completely re-encrypted and verified. The command neither activates mounts nor authorizes startup.

4. While services remain stopped, have deployment tooling switch the staged store and next identity together. Never form a mixed old/new pair.

5. In the actual new active topology, run:

   ahri-tre --config /etc/ahri-tre/config.toml secrets verify --all
   

   Start services only after this post-cutover gate passes.

If the new active pair fails verification, keep services stopped. Restore the retained old store and old identity together, repeat the same verify --all gate against that restored active topology, and start only if it passes. A double failure remains an operator recovery incident. AHRI TRE does not perform automatic rollback, remove a failed staging tree, resume a partial rotation, or export plaintext.

Backup and exact same-Deployment recovery

The initial supported backup is an offline, application-consistent procedure:

1. Quiesce every service and process capable of reading or writing the Managed store, PostgreSQL, or Lake state.
2. Snapshot the complete Application configuration, encrypted Managed-store directory and audit journal, PostgreSQL state, and Lake state at one reviewed recovery point. Record release identities and safe fingerprints.
3. Back up the matching X25519 identity through a separate external secret-custody channel. Never place it in or beside the ciphertext backup.
4. Mount the copied Application document, store, and separately retrieved identity into an isolated offline verification workload at the same fixed container paths and run secrets verify --all. Keep writers quiesced until this succeeds. A copied but unverified snapshot is not a completed backup.
5. Test PostgreSQL/Lake restoration and application qualification in a controlled non-production recovery exercise.

Recovery restores only the same logical Deployment: the same Deployment UUID, Application configuration, complete encrypted store/audit history, matching identity, and consistent PostgreSQL/Lake state. Deployment tooling restores files and mounts; AHRI TRE has no secrets restore command. Before any Secret-consuming service starts, run offline secrets verify --all in the restored active topology, then repeat dependency, service-readiness, and authenticated Client gates.

Changing the Deployment UUID, substituting another identity, copying selected ciphertexts into a new store, or using this procedure to promote environments fails closed and is unsupported. A new Deployment receives new trust roots and credentials.

Audit integrity and failure handling

Managed mutations append integrity-protected, safe audit records. The journal contains operation, logical reference, version, time, Deployment identity, and safe actor metadata—never Secret material. secrets verify --all checks the entire chain and every active envelope; it repairs nothing. List, inspect, human, JSON, log, and failure output remain safe to retain.

Treat missing, unreadable, permission-invalid, corrupt, identity-mismatched, Deployment-mismatched, lock-unsupported, or inspection-unavailable state as a hard failure. Do not delete staged material after an uncertain owner update, edit manifests, rename ciphertexts, skip an authority during removal, or use a force bypass. Preserve failed root-rotation staging for incident evidence and provide a newly empty destination only after reviewed disposal. Infrastructure break-glass identities and native recovery tools remain institution-owned and outside AHRI TRE configuration and Secret references.

Troubleshooting without weakening boundaries

• A retired-variable error names the variable only. Remove it from the process and deployment manifest; do not translate its value into another implicit input.
• An explicit configuration failure is final for that path. Correct the selected document, ownership, permissions, or schema; do not fall back to the canonical file.
• A preflight failure is safe selected-path evidence, not permission to create or repair a dependency. Inspect the named category through the owning platform and adapter.
• A TLS or Client-origin failure requires checking the distributed bootstrap, Deployment identity, CA overlap, DNS, and server leaf identity. There is no insecure or endpoint-override mode.
• A Managed-store integrity or identity failure keeps consumers stopped. Use offline secrets verify --all and an exact verified restore; never edit the store or audit journal.
• A root-rotation interruption keeps services stopped at the deployment tooling’s recovery marker. Resume the reviewed switch/verification procedure or restore and verify the old matching pair. Local dev-env recovery commands apply only to Local TRE.

Controls owned by the integrator

Before staging or production, design and review:

1. Identity and trust: approved OAuth/OIDC registration, redirect origins, certificate issuance and rotation, hostname verification, operator identities, and break-glass access.
2. Secret delivery: separate bootstrap, runtime, Web, Datastore, and operator capabilities; external secret custody; audit and rotation. Never bake credentials into an image or configuration document.
3. Network isolation: no public PostgreSQL or Lake endpoint; explicit workload-to-workload policy; controlled ingress and egress; separate administration and client paths.
4. Persistent data: PostgreSQL and Lake placement, encryption, retention, capacity, consistent backup, verified restore, disaster recovery, and data deletion policy.
5. Operations: health, metrics, logs, traces, alerts, on-call ownership, change control, rollback, dependency updates, and vulnerability response.
6. Governance: Study authorization, custodianship, audit/provenance, disclosure review, and any governed-compute isolation required by local policy.

Local TRE deliberately supplies no production backup, observability platform, RustFS/S3 service, JupyterHub, governed-compute worker, or container orchestrator. Its local CA and deterministic identities are test material.

Cut over retained Local TRE state

The replacement development-container migration is complete. Its legacy Compose/editor-attach and Git-volume instructions remain only in the historical one-time retirement section of README-dev-env.md; they are not a current Local, Integration, staging, or production operating path.

For the configuration-and-Secrets cutover, inspect this checkout’s retained Local TRE state through the supported interface:

./dev-env doctor
./dev-env local status

doctor checks host prerequisites only. local status reports exactly one safe action:

• no state change required: no Local deployment exists, or retained state already matches the current configuration/Secret and service fingerprint;
• service rebuild required: run ./dev-env local up to rebuild immutable services while retaining current configuration, roots, and durable volumes;
• guarded ./dev-env local reset: the retained Local deployment predates the cutover. Run the ownership-checked reset, type the printed Local identifier, then run ./dev-env local up to create the new authority layout.

Neither status nor up converts old state. Do not edit the root selector, rename volumes, copy old roots, or invoke raw Compose to bypass classification. Local up reports client ready only after an isolated workload uses the rendered Client bootstrap and an ephemeral Runtime credential for an authenticated stable-protocol HTTPS probe.

Local TRE has no supported backup, snapshot, restore, export, import, production isolation, or disaster-recovery guarantee. The Integration fixture is even narrower: ./dev-env integration run creates a command-scoped disposable PostgreSQL/Lake test environment, permits its fixture-only inputs only inside that runner, and removes successful state. Neither scope is a deployment template or production validation result.

Qualification gates

For every candidate release:

• run formatting, Clippy with warnings denied, and the full workspace tests;
• run the same Integration contract natively on linux/amd64 and linux/arm64, without QEMU or another architecture emulator;
• verify PostgreSQL TLS and the writable Lake mount at the configured container-visible Lake location through the Integration probes;
• verify declared image versions and compatibility labels, never latest;
• test invalid CA, hostname, and expiry failures;
• inspect images and runtime projections for source, private keys, credentials, and Restricted local references; and
• record platform, Docker/Compose, artifact revisions, profiles, and results without recording credentials or user identities.

Use README-dev-env.md for the repository-owned test and Local lifecycle. Production incidents and recovery must use the integrator’s reviewed runbooks, not Local TRE commands or raw Docker volume copies.

Run repository integration reproducibly from the checkout:

./dev-env doctor
./dev-env integration run

Arguments after -- may narrow Cargo tests for development, for example ./dev-env integration run -- -p ahri_tre_pgmeta; release qualification uses the complete runner. This fixture is the only documented command-scoped direct dependency exception and is not a production smoke test.

For an installed deployment, use release-packaged binaries and the final mounts—not dev-env—to record:

ahri-tre --config /etc/ahri-tre/config.toml config validate
ahri-tre --config /etc/ahri-tre/config.toml config show-effective \
  --for trusted-runtime --format json
ahri-tre --config /etc/ahri-tre/config.toml config preflight \
  --for trusted-runtime --format json
ahri-tre --config /etc/ahri-tre/config.toml secrets verify --all --format json

Repeat show-effective and preflight for Web, every admitted Execution profile, Datastore creation, and Secret administration path. Then record each running service’s documented readiness and an authenticated Client stable-protocol operation. Keep the exactly one JSON document from each reporting command as safe evidence, but do not record credentials, personal identity, private paths, or dependency-native errors.

Diagnostics And Observability

Use the reported request UUID, the owning service’s operational events, and the check’s evidence basis together. A successful transport, readable Lake directory, or previously opened Session does not establish current Datastore readiness. These diagnostics use existing process streams and Local TRE commands; no collector, dashboard, telemetry database, or external account is required.

P2.7 is complete. The coverage ledger records the delivered initial paths and validation. The fresh all-five acceptance passed Lake cases L1–L3, absent OAuth artifact case A1, expired artifact case A2, and verified cleanup. Under ADR-0019, A1 proves actual absence causes generic credential rejection. The rejected caller receives no owner-specific absence finding.

Find a reported request

From the checkout owning the Local TRE, inspect the named services:

./dev-env local status
./dev-env local logs web
./dev-env local logs trusted-runtime

Use the UUID from the Browser response’s x-request-id header or the protocol response’s request_id. The following UUID is a recorded metadata-failure example; replace it with the reported UUID for your deployment:

request_id=2581c020-a54b-48d3-8ecd-5dd10fa8363c
./dev-env local logs | rg --fixed-strings "$request_id"

local logs returns the last 200 redacted lines per service. local logs --follow trusted-runtime streams subsequent lines. Output includes an environment identifier and may include Compose service prefixes; it is not a raw JSON file. No match can mean the event is older than that window, was dropped, or belongs to another process/deployment. It does not prove that the request never ran. Neither command requires container IDs. Local status/log selection can initialize the non-secret checkout identifier; it does not start services. Exit 69 means logs or prerequisites are unavailable; 78 can mean incompatible retained checkout state. Follow the Local lifecycle guide for that state; diagnostic inspection does not authorize a reset.

Group matching events by service_role and process_instance_id, then compare operation, stage, outcome, failure_category and duration_ms. Within one process, parent_span_id points to a local span_id. Across processes, join by request_id; there is no distributed span transport or global event ordering. Callers can reuse UUIDs, so also use the operation, process and time window. HTTP input UUIDs are validated/replaced and the resolved value is returned. Correlation grants neither identity, replay rights, nor mutation idempotency.

Tracked Dataset admission adds operation_id. Join detached worker events to admission with that ID and the initiating request ID; later polls have their own request IDs. accepted means an Operation was admitted or read back, not that a Dataset committed. Recovery joins by Operation ID without inventing the lost request ID. A killed process cannot emit a trustworthy completion or orderly stop.

For a CLI workflow already authorized in an existing Session, retain stdout and stderr separately. Here client.toml and analysis stand for your existing Client document and Session name, not server configuration or a new Session:

umask 077
ahri-tre --config ./client.toml --session analysis domain list --format json > /tmp/tre-domain-result.json 2> /tmp/tre-domain-events.log

The result contains the protocol correlation; stderr contains CLI events. The Managed runtime has its own process stream. Do not expect it in Web’s logs. Keep authorized result payloads local; they can contain information excluded from general operational telemetry. Library and C ABI hosts own their logging sink.

Choose the diagnostic boundary

The configuration path, profile and Session names above are placeholders for already configured selections. Use config preflight --help to choose other supported targets; do not supply an Application document to a user-side client. Web paths refer to the Web service: the Local Browser image does not proxy /ready. Use the deployed Web route and configured TLS trust, or the existing ./dev-env local test smoke wrapper for its defined Browser/Web checks.

Preflight’s filesystem_lake is directory access; lake_storage_endpoint is remote TCP reachability; trusted_scratch is independent local posture. None proves an authenticated object read, write success or usable catalog. lake_catalog_authentication and lake_catalog_read remain optional not_applicable/not_checked in operator preflight because that path has no resolved catalog capability and opening the adapter can create scratch/TLS files. Metadata administrator success cannot upgrade them.

authentication_provider checks TCP reachability only, with no discovery, token exchange or validation. authentication_material, authentication_expiry and authentication_validation stay not_checked without owner authority. Selected client Secrets are opaque; token-shaped bytes do not become user OAuth evidence. Preflight neither acquires a user’s credentials nor repairs permissions, migrates metadata, writes Lake data, rotates Secrets, renews tokens or retries mutations.

Read observation fields and time

Existing check envelopes retain their own statuses and stable id, summary, severity and safe evidence. Optional observation adds:

Preflight also retains its outer check scope and optional category; use observation.scope for the common authority vocabulary above. Diagnostic categories and operational-event categories are separate closed vocabularies.

Calculate age from the original observed_at_utc and your display time; don’t substitute the time of the status response. Session-open catalog evidence is historical even in the opening response. Current workflow failures can report their own authenticated observations; subsequent Session inspection preserves their acquisition time and presents them as historical. Missing evidence stays skipped, not passed. Session findings use passed, warning, failed, skipped.

OAuth evidence at data.oauth_artifact_observation has a finding and separate evaluated_at_utc: status may evaluate expiry now while retaining the original artifact capture time (origin=runtime_login, scope=owner_login). This does not validate the token with its provider. Runtime credential validity and OAuth artifact expiry are distinct. Only the owning login can receive the observation.

Retrieve the shipped JSON contracts without login:

ahri-tre schema get cli.doctor.output.v1 --format json
ahri-tre schema get protocol.daemon.readiness.v2 --format json
ahri-tre schema get cli.config.preflight.v1 --format json
ahri-tre schema get protocol.session.lifecycle.v2 --format json
ahri-tre schema get cli.auth.status.v1 --format json
ahri-tre schema get protocol.runtime.credential.status.v2 --format json

cli.daemon.lifecycle.v1 describes private local lifecycle output and cli.diagnostics.output.v1 generic local diagnostics. Consumers must accept the documented optional observation fields; older reports may omit them. Fixed actions are ahri-tre doctor, ahri-tre daemon doctor, ahri-tre config preflight --help, ahri-tre auth status, ahri-tre session status, and ./dev-env local logs trusted-runtime. Supply your existing configuration and Session selection where required. Advice to reauthenticate is an explicit next step, never an automatic action taken by inspection.

Deadlines and partial results

Preflight allows five seconds per complete probe and fifteen seconds per active report, including DNS. Four process-wide worker permits bound outstanding work; timed-out native calls retain permits until they really finish. Independent checks and completed metadata stages survive another stage’s timeout. Unperformed checks retain timeout, admission or not-checked explanations. These budgets do not promise preemption of arbitrary native filesystem/configuration loading.

daemon doctor bounds the complete local exchange to five seconds, including silent, trickled and oversized responses. Its transport also enforces the one-MiB response-frame limit before waiting for a newline. Reused synchronous local filesystem operations still depend on OS responsiveness.

Web uses the existing ten-second Runtime deadline for DNS, TLS, exchange and validation. Readiness and diagnostics share the configured anonymous Browser read budget regardless of cookies: defaults are six concurrent checks, burst 30 and 300 checks/minute. Six resolver slots remain occupied by uninterruptible lookups after timeout. /health remains independent. A timeout or cancelled forwarding event does not assert that remote/native work stopped or rolled back. The Managed Runtime HTTP attempt also retains its ten-second deadline; no new general workflow deadline is introduced by logging.

Representative investigations

These examples are verified against the recorded public acceptance runs linked below, not faults to inject into a retained Local TRE. Begin with local status, search the reported request, then inspect the owning service:

L1 successes carry catalog findings in data.session.adapter_observations; L2/L3 failures use error.adapter_observations, HTTP 200 protocol failure, operation_failed, CLI 3. No Session is manufactured for a failed open. The ticket 16 authorization trace explains why explicit owner Session opening can acquire catalog evidence while later inspection cannot open adapters. The ticket 19 walkthrough provides the automated disposable reproduction, fault setup, public text/JSON commands, protected-state checks and verified cleanup. The fresh report’s ok=true and five_case_acceptance=PASS require all five cases and cleanup. A1 combines validated exact-owner absence with first-request HTTP 401 and CLI exit 6, preserving generic rejection and expected security cleanup. Earlier reports retain their original A1 BLOCKED result; they are historical evidence.

Operational event contract, version 1

ahri_tre_observability owns the event contract and bounded executable sink; ahri_tre_security owns sensitive-material policy. CLI/Managed runtime/Web/Trusted runtime initialize newline-delimited JSON stderr logging before bootstrap. Stdout remains command results or the defined data stream. Human usage errors may still appear on stderr. Libraries and the C ABI emit passive tracing events and never replace the host subscriber. No ambient logging filter such as RUST_LOG changes these fixed defaults; logging setup failure leaves execution available without this subscriber.

Unknown provenance is omitted, especially before configuration resolution. Client processes do not claim an Application fingerprint. Enrichment neither reopens documents/Secret stores nor hashes Secret material. Version 1 has no Datastore identity field; do not infer one from a resource name. timestamp locates event emission; observed_at records acquisition before delivery; duration_ms uses a monotonic clock. Nested stage times can overlap and must not be added to invent total elapsed time. Runtime total starts before credential validation, after HTTP decoding. An attempt number describes an existing retry under the request; telemetry never adds retries.

Operation names are the 20 fixed Discovery/search actions, browser_access_request_submit/list/get/withdraw and browser_custodian_access_request_list/approve/reject (each slash denotes a separate suffix), domain_list, session_open/status/close, runtime_login_start/callback/poll/acknowledge, runtime_credential_status/rotate/logout, dataset_from_datafile, operation_list/get/cancel/result_get, runtime_capabilities, and other_request. Unlisted request kinds use the last fixed name, never input-derived text. The crate reference explains stage placement and governance outcomes for each implemented workflow.

Starts, success, accepted and skipped outcomes are info; internal failures are error; other completion outcomes are warn. process_ready is info and the other standalone process signals are warn. Successful probe/status traffic suppresses redundant successful child stages and emits at most one summary at its owning boundary. Detailed workflow events are bounded by operation/stage/attempt, not rows, chunks, Variables or search matches. A successful child never proves the parent mutation committed. Human summaries are not the machine contract.

Only allowlisted fields are serialized. Credentials, cookies, tokens, private keys, personal identities, Session references, Restricted local references, raw targets/query strings, SQL, resource names, request/rejection text and governed content are excluded. Pipeline logs remain governed content and never enter operational telemetry. Configuration/Secret ownership, OAuth/libpq separation and owner authorization remain unchanged; there is no diagnostic login/repair, new Web datastore probe or new user-side server authority.

Delivery limits

Each JSON event is at most 8192 bytes, excluding its newline. Optional numeric evidence is limited to 256 measurements on ingress. Oversize envelopes omit all optional evidence and set truncated=true; required JSON and typed IDs are never byte-truncated. The writer has 256 queued records plus at most one in flight. Producers enqueue without blocking and drop the newest event when full or disconnected. A blocked stderr stalls only the dedicated writer. Failed writes are discarded without retries. Loss counts saturate at u64::MAX; after a successful write, events_dropped reports accumulated loss in dropped_events. A failed recovery report retains the count for a later successful recovery.

Executable normal return attempts a bounded 100 ms flush; destruction never joins a blocked writer. Abrupt termination can lose queued records. A partial sink write can damage a line; recovery terminates that prefix before further JSON, and readers must discard damaged lines. There is no durable delivery, retention, replay or audit guarantee. Delivery failures do not determine workflow or governance results. The pressure acceptance record covers blocked, failed and recovering sinks, concurrent requests, host-owned FFI logging, and unchanged CLI data stdout with constant event counts.

MinisForum Test Datastore Server Installation

This guide records the confirmed MinisForum installation and its original deployment path. The server is now maintained with the separate server update procedure, whose current worked example uses server/validator v0.10.8 and site kit 0.3.14. A blank MinisForum uses the fresh server installation procedure. Sections 1–12 preserve the superseded v0.10.3-based server and 0.2.0 site-bundle procedure as historical evidence; do not execute them for a new or resumed installation. Resume only with the release-bound v3 generated runbook described in Section 13.

Do not apply or edit the original fixed-site bundle for svrlducklakedev01. Ticket 12 must generate a new bundle for the MinisForum profile before site provisioning can continue.

Confirmed profile

The Runtime certificate must contain runtime.svrltreapcc02.home.arpa as a DNS Subject Alternative Name. The port is not part of the certificate name. Never replace name resolution and certificate validation with an insecure exception.

The generated server package contains no deployment Secret values. Never add TLS private keys, OIDC client secrets, passwords, tokens, or managed-store keys to the repository, transfer archive, or retained command output.

The earlier v0.10.3-based server and 0.2.0 site bundle are not an upgrade baseline. Release v0.10.4 cleanly replaces them. Do not install directly from deployment/test-datastore-kit/server-package; that directory contains templates, not a compiled server package.

Historical record: Sections 1–12 describe work already performed for the superseded artifacts. Keep them for provenance only. The runnable source of truth for the replacement is the v3 runbook generated by Ticket 12.

1. Prepare the WSL2 builder

Run the builder commands in the main checkout:

cd /home/kobus/repos/ahri-tre-rs
./dev-env doctor

Temporary inputs go below tmp/kit-build and the generated kit goes below dist. Both locations are ignored by Git.

2. Download and verify the release

mkdir -p tmp/kit-build/downloads

gh release download v0.10.3 \
  --repo AHRIORG/ahri-tre-rs \
  --pattern 'ahri-tre-0.10.3-x86_64-unknown-linux-gnu.tar' \
  --pattern 'ahri-tre-0.10.3-x86_64-unknown-linux-gnu.tar.sha256' \
  --dir tmp/kit-build/downloads

cd tmp/kit-build/downloads
sha256sum --check \
  ahri-tre-0.10.3-x86_64-unknown-linux-gnu.tar.sha256
cd /home/kobus/repos/ahri-tre-rs

The checksum must report OK. The frozen expected SHA-256 is cad4efb729b09f5c6371639a758ee6b872e22315f52cb5550caad331ca75b5c2.

3. Create the pinned source checkout

git clone --no-checkout . tmp/kit-build/source-v0.10.3

git -C tmp/kit-build/source-v0.10.3 checkout --detach \
  56cbd62b900f7bfb36544eb1ac9b6a4a69bdf565

git -C tmp/kit-build/source-v0.10.3 rev-parse HEAD
git -C tmp/kit-build/source-v0.10.3 status --porcelain

The revision command must print 56cbd62b900f7bfb36544eb1ac9b6a4a69bdf565. The status command must print nothing.

4. Build the maintainer image

Run this from /home/kobus/repos/ahri-tre-rs, not from the pinned source checkout:

docker build \
  --file .devcontainer/Dockerfile \
  --target maintainer \
  --tag ahri-tre-maintainer:local \
  .

docker run --rm ahri-tre-maintainer:local bash -lc \
  'zig version && cargo-zigbuild --version'

Both probe commands must print version information.

5. Build the server programs

The explicit CARGO_TARGET_DIR keeps the completed programs outside the container-private /tmp directory.

docker run --rm \
  --env CARGO_TARGET_DIR=/workspaces/ahri-tre-rs/tmp/kit-build/source-v0.10.3/target \
  --volume "$PWD:/workspaces/ahri-tre-rs" \
  --workdir /workspaces/ahri-tre-rs/tmp/kit-build/source-v0.10.3 \
  ahri-tre-maintainer:local \
  bash -lc '
    test "$(git rev-parse HEAD)" = \
      56cbd62b900f7bfb36544eb1ac9b6a4a69bdf565
    test -z "$(git status --porcelain)"
    RUSTFLAGS="-C link-arg=-Wl,--build-id=0x56cbd62b900f7bfb36544eb1ac9b6a4a69bdf565" \
      cargo zigbuild --locked --release \
        --target x86_64-unknown-linux-gnu \
        -p ahri_tre_trusted_runtime \
        -p ahri_tre_web
  '

test -x tmp/kit-build/source-v0.10.3/target/x86_64-unknown-linux-gnu/release/ahri-tre-runtime
test -x tmp/kit-build/source-v0.10.3/target/x86_64-unknown-linux-gnu/release/ahri-tre-web

6. Generate and verify the server package

docker run --rm \
  --env CARGO_TARGET_DIR=/workspaces/ahri-tre-rs/tmp/kit-build/packager-target \
  --volume "$PWD:/workspaces/ahri-tre-rs" \
  --workdir /workspaces/ahri-tre-rs \
  ahri-tre-maintainer:local \
  bash -lc '
    cargo run --locked -p xtask -- \
      test-datastore-kit package-server \
      --release-archive \
        tmp/kit-build/downloads/ahri-tre-0.10.3-x86_64-unknown-linux-gnu.tar \
      --release-checksum \
        tmp/kit-build/downloads/ahri-tre-0.10.3-x86_64-unknown-linux-gnu.tar.sha256 \
      --runtime-binary \
        tmp/kit-build/source-v0.10.3/target/x86_64-unknown-linux-gnu/release/ahri-tre-runtime \
      --web-binary \
        tmp/kit-build/source-v0.10.3/target/x86_64-unknown-linux-gnu/release/ahri-tre-web \
      --artifact-root \
        dist/ahri-tre-test-datastore-deployment-kit-0.1.0 \
      --source-revision \
        56cbd62b900f7bfb36544eb1ac9b6a4a69bdf565
  '

The packager validates the release checksum, source revision, architecture, embedded build identity, dependencies, launch behaviour, and public schemas.

KIT_ARTIFACT_ROOT="$PWD/dist/ahri-tre-test-datastore-deployment-kit-0.1.0"

test -x "$KIT_ARTIFACT_ROOT/server/install.sh"
test -x "$KIT_ARTIFACT_ROOT/server/bin/ahri-tre-runtime"
test -x "$KIT_ARTIFACT_ROOT/server/bin/ahri-tre-web"
test "$(find "$KIT_ARTIFACT_ROOT/server" -type f | wc -l)" -eq 28

7. Archive and copy the package

cd "$KIT_ARTIFACT_ROOT"
tar -czf ahri-tre-server-0.1.0.tar.gz server
sha256sum ahri-tre-server-0.1.0.tar.gz \
  > ahri-tre-server-0.1.0.tar.gz.sha256

Replace with the ordinary Ubuntu login name. First confirm the destination:

ssh <minisforum-user>@192.168.31.75

grep '^PRETTY_NAME=' /etc/os-release
uname -m
hostnamectl --static
ip -4 -brief address
exit

The relevant values must be Ubuntu 26.04 LTS, x86_64, svrltreapcc02, and 192.168.31.75. Transfer the archive from WSL2:

The repeated SSH and SCP commands can use public-key authentication instead of asking for the MinisForum account password each time. In WSL2, first check for an existing Ed25519 key:

ls -l ~/.ssh/id_ed25519 ~/.ssh/id_ed25519.pub

If both files exist, reuse them. If they do not exist, create a key and choose whether to protect it with a passphrase when prompted:

ssh-keygen -t ed25519 -a 100 -C "wsl2-to-minisforum"

Copy only the public key to the MinisForum, then verify that a new SSH connection succeeds:

ssh-copy-id <minisforum-user>@192.168.31.75
ssh <minisforum-user>@192.168.31.75
exit

Never copy ~/.ssh/id_ed25519, the private key, to another computer. If the key has a passphrase, cache it for the current WSL2 session:

eval "$(ssh-agent -s)"
ssh-add ~/.ssh/id_ed25519

ssh <minisforum-user>@192.168.31.75 \
  'umask 077; mkdir -p ahri-tre-transfer'

scp ahri-tre-server-0.1.0.tar.gz \
    ahri-tre-server-0.1.0.tar.gz.sha256 \
    <minisforum-user>@192.168.31.75:ahri-tre-transfer/

Using the reserved IP address for this SSH transfer does not weaken the later HTTPS hostname validation.

8. Verify, extract, and install on the MinisForum

ssh <minisforum-user>@192.168.31.75
cd ~/ahri-tre-transfer

sha256sum --check ahri-tre-server-0.1.0.tar.gz.sha256
tar -xzf ahri-tre-server-0.1.0.tar.gz

test -x server/install.sh
test "$(find server -type f | wc -l)" -eq 28

Stop if the checksum fails or the folder does not contain all 28 files. Check for an unexpected earlier installation:

systemctl list-unit-files 'ahri-tre*' --no-legend
sudo test ! -e /usr/libexec/ahri-tre

Stop rather than overwrite or adopt unexpected AHRI TRE files or units. On a clean host, install and diagnose the package:

cd ~/ahri-tre-transfer/server
sudo ./install.sh

sudo /usr/libexec/ahri-tre/diagnose.sh dependencies
sudo /usr/libexec/ahri-tre/diagnose.sh verify-web-binary

Each diagnostic must return JSON containing “status”:“ok”. Run the Web-binary check even for this CLI-only installation: the package includes the Web program, but its service remains disabled.

systemctl show ahri-tre-runtime.service \
  --property=ActiveState,UnitFileState

systemctl show ahri-tre-web.service \
  --property=ActiveState,UnitFileState

Do not enable or start either service yet.

9. Prepare the Runtime certificate inputs

For the current 0.3.14 installed-conformance candidate, follow Recovering a Conformance Candidate After Runtime Key Loss. That self-contained procedure gives every WSL2 and development-container command needed to create the protected Runtime key, issue its certificate, regenerate the site, and compose the checksum-bound replacement candidate.

The server software is installed, but the MinisForum is not provisioned yet. Ticket 12 supplies the v2 profile and packager; Kobus still owns the external certificate and Secret work.

Prepare these two public files on the WSL2 build machine:

• pki/runtime-certificate-chain.pem, whose leaf certificate SAN contains runtime.svrltreapcc02.home.arpa; and
• pki/deployment-ca-chain.pem, containing the public issuing CA chain.

Keep the Runtime private key on the MinisForum and provision it only at the path named by the generated injected-secrets.json. Do not copy a private key, CA signing key, password, OAuth secret, token, or Runtime credential into the repository or site bundle.

10. Generate the MinisForum site bundle

In WSL2, open the current Ticket 12 repository root, not the detached v0.10.3 source checkout used to build the server binaries. Cargo is supplied by the maintainer image built in step 4; it is not expected to be installed directly in WSL2. Do not install an unpinned Ubuntu, Snap, or Rustup Cargo merely to run this step.

cd ~/repos/ahri-tre-rs

test -f pki/runtime-certificate-chain.pem
test -f pki/deployment-ca-chain.pem
docker image inspect ahri-tre-maintainer:local >/dev/null

docker run --rm \
  --env CARGO_TARGET_DIR=/workspaces/ahri-tre-rs/tmp/kit-build/packager-target \
  --volume "$PWD:/workspaces/ahri-tre-rs" \
  --workdir /workspaces/ahri-tre-rs \
  ahri-tre-maintainer:local \
  bash -lc '
    cargo run --locked -p xtask -- test-datastore-kit package-site \
      --inputs deployment/test-datastore-kit/site-package/site-inputs.minisforum.json \
      --runtime-certificate-chain pki/runtime-certificate-chain.pem \
      --public-ca-chain pki/deployment-ca-chain.pem \
      --server-component-manifest \
        dist/ahri-tre-test-datastore-deployment-kit-0.1.0/server/component-versions.json \
      --artifact-root dist/ahri-tre-test-datastore-deployment-kit-0.2.0
  '

This CLI-only profile deliberately has no --web-certificate-chain. A successful command reports the v2 schema, host=svrltreapcc02, datastore=ahri-tre-test, and artifacts=21.

Before copying anything, confirm the generated identity:

KIT_ARTIFACT_ROOT="$PWD/dist/ahri-tre-test-datastore-deployment-kit-0.2.0"

grep -F 'f2ef37c5-7430-468a-a439-b3ba1b0527c1' \
  "$KIT_ARTIFACT_ROOT/site/site-inputs.json"
grep -F 'runtime.svrltreapcc02.home.arpa' \
  "$KIT_ARTIFACT_ROOT/site/client.toml"
grep -F '"enabled": false' \
  "$KIT_ARTIFACT_ROOT/site/web-ingress-requirements.json"

Review site/firewall-plan.json, site/filesystem-plan.json, and both site/client-publication/ files. Stop if any name, address, mount, client network, or Deployment UUID differs from the confirmed site record. Regenerate from corrected reviewed inputs; do not edit generated files.

11. Publish the hosts-file entry

Open site/name-resolution.md and apply its instructions to the MinisForum, Windows, WSL2, and macOS. Each hosts file must contain the same mapping:

192.168.31.75 runtime.svrltreapcc02.home.arpa

First check for an existing entry before appending. If the name already exists with another address, stop and correct that line instead of creating conflicting entries.

On the MinisForum, run:

grep -n 'runtime.svrltreapcc02.home.arpa' /etc/hosts
grep -Eq \
  '^[[:space:]]*192\.168\.31\.75[[:space:]]+runtime\.svrltreapcc02\.home\.arpa([[:space:]]|$)' \
  /etc/hosts || \
  printf '%s\n' '192.168.31.75 runtime.svrltreapcc02.home.arpa' | \
  sudo tee -a /etc/hosts >/dev/null
getent ahostsv4 runtime.svrltreapcc02.home.arpa

getent must return 192.168.31.75.

On Windows, open PowerShell as Administrator. Check for an existing entry, append the mapping only when the name is absent, flush the cache, and use the normal Windows resolver to verify it:

$hostsPath = "$env:SystemRoot\System32\drivers\etc\hosts"
$existing = Select-String -LiteralPath $hostsPath -Pattern `
  '^\s*[^#\s]+\s+runtime\.svrltreapcc02\.home\.arpa(?:\s|$)'
$existing

if (-not $existing) {
    Add-Content -LiteralPath $hostsPath `
      -Value "`r`n192.168.31.75`t runtime.svrltreapcc02.home.arpa" `
      -Encoding ascii
}

Clear-DnsClientCache
[System.Net.Dns]::GetHostAddresses(
  'runtime.svrltreapcc02.home.arpa'
).IPAddressToString

If $existing shows another address, correct that line instead of running Add-Content. The final command must include 192.168.31.75.

Windows and WSL2 have separate hosts files. In the WSL2 Ubuntu terminal, run:

grep -n 'runtime.svrltreapcc02.home.arpa' /etc/hosts
grep -Eq \
  '^[[:space:]]*192\.168\.31\.75[[:space:]]+runtime\.svrltreapcc02\.home\.arpa([[:space:]]|$)' \
  /etc/hosts || \
  printf '%s\n' '192.168.31.75 runtime.svrltreapcc02.home.arpa' | \
  sudo tee -a /etc/hosts >/dev/null
getent ahostsv4 runtime.svrltreapcc02.home.arpa

To keep this entry when WSL2 would otherwise regenerate /etc/hosts, edit /etc/wsl.conf without replacing any existing settings:

sudo nano /etc/wsl.conf

Add this section if it is absent, or add the setting to its existing [network] section:

[network]
generateHosts = false

From Windows PowerShell, run wsl --shutdown, reopen Ubuntu, and repeat the grep and getent checks. Do not run wsl --shutdown while other important WSL2 work is still running.

On macOS, run:

grep -n 'runtime.svrltreapcc02.home.arpa' /etc/hosts
grep -Eq \
  '^[[:space:]]*192\.168\.31\.75[[:space:]]+runtime\.svrltreapcc02\.home\.arpa([[:space:]]|$)' \
  /etc/hosts || \
  printf '%s\n' '192.168.31.75 runtime.svrltreapcc02.home.arpa' | \
  sudo tee -a /etc/hosts >/dev/null
sudo dscacheutil -flushcache
sudo killall -HUP mDNSResponder
dscacheutil -q host -a name runtime.svrltreapcc02.home.arpa

The final command must report 192.168.31.75. A failed ping does not by itself mean that name resolution failed; use the resolver checks above. Do not use the IP address in HTTPS commands or accept an insecure certificate exception.

12. Archive and copy the generated site folder

From WSL2:

cd ~/repos/ahri-tre-rs
KIT_ARTIFACT_ROOT="$PWD/dist/ahri-tre-test-datastore-deployment-kit-0.2.0"

printf 'Using site artifact root: %s\n' "$KIT_ARTIFACT_ROOT"
test -d "$KIT_ARTIFACT_ROOT/site"
test -x "$KIT_ARTIFACT_ROOT/site/provision.sh"

cd "$KIT_ARTIFACT_ROOT"
test "$PWD" = \
  "$HOME/repos/ahri-tre-rs/dist/ahri-tre-test-datastore-deployment-kit-0.2.0"

tar -czf ahri-tre-site-0.2.0.tar.gz site &&
  sha256sum ahri-tre-site-0.2.0.tar.gz \
    > ahri-tre-site-0.2.0.tar.gz.sha256 &&
  sha256sum --check ahri-tre-site-0.2.0.tar.gz.sha256

scp ahri-tre-site-0.2.0.tar.gz \
    ahri-tre-site-0.2.0.tar.gz.sha256 \
    <minisforum-user>@192.168.31.75:ahri-tre-transfer/

The checksum command must report ahri-tre-site-0.2.0.tar.gz: OK before the transfer. The older 0.1.0 artifact root contains server/, not site/. If tar reports site: Cannot stat, stop: do not create a checksum or transfer that archive. Return to the repository root, reset KIT_ARTIFACT_ROOT exactly as above, and repeat the two test commands.

On the MinisForum:

cd ~/ahri-tre-transfer
sha256sum --check ahri-tre-site-0.2.0.tar.gz.sha256
tar -xzf ahri-tre-site-0.2.0.tar.gz

test -x site/provision.sh
test -f site/site-inputs.json
test -f site/name-resolution.md

Stop if the checksum or any file check fails.

13. Replace with the v0.10.4 server and v3 site bundle

Do not run the copied 0.2.0 bundle’s site/provision.sh. Do not install PostgreSQL directly on Ubuntu. Preserve the existing server package and Runtime certificate work as evidence until the v0.10.4 runbook validates and replaces the package; do not assume either is reusable. The current site bundle is superseded for provisioning.

The default design is PostgreSQL 18 in a managed container on the MinisForum. This is the first live-site and real-identity qualification target for the production validator. Only after it passes does the separate svrlducklakedev01 rollout begin.

13.1 Understand why this step has stopped

Preparing the original Step 13 exposed three problems that an operator should not work around manually:

1. PostgreSQL 18 provides the OAuth protocol hook but does not include a production ORCID token validator.
2. The current repository validator under .devcontainer/local/ trusts the deterministic Local OIDC fixture. It cannot validate real ORCID tokens.
3. The 0.2.0 bundle uses host=127.0.0.1 sslmode=verify-full with a DNS-only certificate and runs administrator commands through the host postgres account. Those assumptions do not work safely for both a container and an external PostgreSQL server.

The shared correction is specified in the Production PostgreSQL ORCID OAuth Validator PRD. Its first three implementation issues are complete and provide:

• the hardened production ORCID validator;
• a pinned PostgreSQL 18 image and compatible standalone artifact; and
• one portable managed-container/external deployment interface.

Ticket 12 packages those artifacts into v0.10.4, delivers the image as a checksummed OCI archive, and consumes them for this MinisForum. A later issue adopts the qualified release on svrlducklakedev01.

13.2 Record the MinisForum PostgreSQL choice

Use this installation record when Ticket 12’s corrected site-input contract is implemented:

Deployment mode: managed-container
PostgreSQL logical TLS name: postgres.svrltreapcc02.home.arpa
PostgreSQL routing address: 127.0.0.1
PostgreSQL port: 5432
Persistent data path: /data/ahri-tre/postgresql
Direct workstation PostgreSQL access: no
ORCID environment: Sandbox only for the first live qualification
ORCID Sandbox issuer: exact confirmed HTTPS issuer
ORCID Sandbox audience: exact Sandbox-issued client ID

The logical TLS name is the name checked against the PostgreSQL certificate. The routing address tells the local Runtime where to send the connection. They are deliberately separate:

host=postgres.svrltreapcc02.home.arpa
hostaddr=127.0.0.1
sslmode=verify-full

Do not reuse runtime.svrltreapcc02.home.arpa for PostgreSQL. Runtime HTTPS and PostgreSQL are different authorities and should have separate private keys and certificates.

For a future separate database server, select external instead. Its certificate must contain that server’s PostgreSQL logical name, and Runtime must route to its recorded address. The server must permit installation of the released PostgreSQL 18 validator artifact. Moving an existing database also requires a tested backup, restore, and cutover; changing the address alone does not move the data.

13.3 Preserve the current MinisForum state

Do not mutate the completed server installation or Runtime key work before the v0.10.4 replacement runbook has validated it. On the MinisForum, run these read-only checks:

hostname -s
findmnt /data
df -h /data
sudo systemctl is-enabled ahri-tre-runtime.service || true
sudo systemctl is-active ahri-tre-runtime.service || true
sudo systemctl is-active ahri-tre-web.service || true
dpkg-query -W -f='${binary:Package} ${Version}\n' 'postgresql*' 2>/dev/null || true
docker --version 2>/dev/null || true
docker compose version 2>/dev/null || true

Expected results:

• the hostname is svrltreapcc02;
• /data is the separate approximately 1 TB filesystem;
• Runtime and Web are not active;
• it is acceptable for PostgreSQL and Docker to be absent.

If PostgreSQL is already installed, do not remove or reconfigure it from this guide. Record the package output for the corrected installer to inspect. If an AHRI TRE service is active, stop and investigate why before doing anything else.

Keep the installed v0.10.3-based package and copied 0.2.0 archive/checksum as superseded evidence until the v0.10.4 runbook performs its clean replacement. Do not patch generated files inside the bundle. Do not create the PostgreSQL data directory yet; the released image must declare the numeric account and exact permissions that own it.

13.4 Prepare the ORCID Sandbox registration information

The Runtime login callback for this CLI-only site is:

https://runtime.svrltreapcc02.home.arpa/v1/runtime-login/callback

In the ORCID Sandbox developer registration:

1. confirm that exact callback is allowed;
2. record the exact Sandbox issuer, JWKS URI, and Sandbox-issued client ID;
3. keep the client secret in a password manager or root-only Secret file; and
4. record Kobus as the registration and callback owner.

The client ID in the current bundle is ahri-tre-test. It is an invalid placeholder until it exactly matches the value issued by ORCID Sandbox. The v3 validator configuration requires that client ID as the token audience and one exact Sandbox issuer. If either differs, correct the non-secret site record and regenerate the entire bundle. Never configure Sandbox and production ORCID as simultaneously trusted issuers.

Do not paste the client secret into this repository, this guide, a shell command, a generated archive, or retained terminal output. Do not guess or add scopes by hand; use only the scope emitted by the corrected, ORCID-qualified bundle.

13.5 Prepare the two remaining external inputs

Validator issues 01–03, P2.11c, the v0.10.4 release, and the v3 packager are complete. Before final site generation, the operator must provide:

1. the actual ORCID Sandbox-issued client ID for the registered Runtime callback; and
2. a separate PostgreSQL public CA chain after securely retaining its signing key outside the repository.

Copy site-inputs.minisforum.v3.template.json to a private working path, replace its two REPLACE_... values, and change nothing else. The corrected kit provides all of the following:

• a checksummed PostgreSQL 18 OCI archive pinned by image digest;
• validator source revision, dependency provenance, and artifact checksums;
• a managed-container installation and lifecycle runbook;
• the exact persistent-data owner and permissions;
• a PostgreSQL-specific DNS name and certificate requirements;
• logical host plus optional hostaddr connection rendering;
• hostssl ... oauth and explicit non-TLS reject rules;
• portable administrator, admission, and removal commands that do not call runuser -u postgres;
• validator configuration for the exact ORCID Sandbox issuer, JWKS/trust inputs, bounded policy, and mandatory Sandbox-issued audience;
• safe readiness checks for TLS, HBA ordering, validator loading, and routing; and
• positive and negative OAuth qualification checks that retain no token or Secret value.

The Local fixture validator, a source file copied manually to the MinisForum, or an unpinned generic PostgreSQL image does not satisfy this checkpoint.

13.6 Regenerate and replace the site bundle

After those two inputs are ready, return to the repository root in WSL2 and follow the package command in the Developer Installation Package Guide. Generate the new 0.3.0 artifact root; do not overwrite dist/ahri-tre-test-datastore-deployment-kit-0.2.0.

Before copying the replacement bundle, verify that its generated manifests record all of these values:

Runtime: runtime.svrltreapcc02.home.arpa:443
PostgreSQL mode: managed-container
PostgreSQL TLS name: postgres.svrltreapcc02.home.arpa
PostgreSQL route: 127.0.0.1:5432
PostgreSQL data: /data/ahri-tre/postgresql
Web: disabled
ORCID environment: Sandbox
ORCID issuer: the exact confirmed Sandbox issuer
ORCID client ID: the actual Sandbox-issued value
Validator artifact: released revision and digest/checksum

Use the archive and checksum commands emitted by the corrected guide. Verify the checksum before copying it to ~/ahri-tre-transfer/. Keep versioned bundles side by side; never merge files from 0.2.0 into the replacement.

13.7 Resume installation from the generated runbook

The corrected generated runbook, not this provisional guide, owns the exact commands. Its expected sequence is:

1. install the supported container engine only if its prerequisite check says it is absent;
2. run server/upgrade.sh to replace the superseded server package, verify its component manifest, then verify the OCI archive checksum, load it locally, and confirm that the loaded image digest and recorded provenance match the v0.10.4 manifest;
3. create a PostgreSQL private key on the MinisForum and have only its public signing request signed by the deployment CA;
4. verify that the certificate matches postgres.svrltreapcc02.home.arpa and its local private key;
5. create /data/ahri-tre/postgresql with the generated ownership and permissions;
6. deploy the PostgreSQL container with port 5432 published only on 127.0.0.1, persistent data under /data, and Secrets mounted read-only;
7. prove PostgreSQL TLS, validator loading, HBA ordering, health, restart, and persistent-data behavior;
8. project the ORCID Sandbox client secret and PostgreSQL administrator Secret at the exact generated paths;
9. run the corrected provision.sh and verify-readiness.sh;
10. admit the intended canonical orcid_... role; and
11. prove that a real admitted ORCID Sandbox identity succeeds while a real valid but unadmitted Sandbox identity fails. Cite the release-bound deterministic rejection matrix for invalid-token classes; do not mint or retain synthetic tokens on the MinisForum.

Only after server readiness succeeds should you publish the corrected client.toml and public CA through the WSL2 and macOS packages.

13.8 Do not use these shortcuts

Do not:

• run the copied 0.2.0/site/provision.sh;
• install the Local fixture validator on the MinisForum;
• install PostgreSQL directly merely to satisfy runuser -u postgres;
• change sslmode=verify-full to require, prefer, or disable;
• add an insecure certificate exception;
• use a PostgreSQL password as a substitute for ORCID Session authentication;
• publish port 5432 on 0.0.0.0 or the LAN address;
• expose PostgreSQL or Lake storage to Windows, WSL2, or macOS clients; or
• hand-edit generated configuration to make readiness pass.

Your current operator action is to complete Section 13.5, generate the final bundle, and then follow its short site/HITL.md. Complete Ticket 12 only after the live MinisForum qualification evidence is reviewed.

Related material

• Developer Installation Package Guide
• System Integrator Guide
• Ticket 02 server-package handoff
• Ticket 12 configurable LAN site
• Production PostgreSQL ORCID OAuth Validator PRD

Installing the Test Datastore Server

This procedure installs the confirmed AHRI TRE Test Datastore on an empty MinisForum. It does not upgrade or adopt an earlier installation. For an existing server, use Updating the Test Datastore Server. For clean-host installed-package qualification, use Preparing Secrets for Clean-Host Installed-Package Conformance instead; its harness must remain the only package installer.

The worked example installs:

• Ubuntu MinisForum svrltreapcc02 at 192.168.31.75;
• AHRI TRE server v0.10.8;
• PostgreSQL ORCID validator v0.10.8; and
• release-bound MinisForum site kit 0.3.14.

Publication gate: do not begin this procedure until the GitHub v0.10.8 release actually lists both the 0.3.14 archive and its .sha256 asset. The download in Section 4 must fail closed while those assets are unpublished.

There are three command locations:

• WSL2 is the Ubuntu terminal on the Windows 11 desktop. Downloads and certificate signing happen there.
• MinisForum is the Ubuntu server shell whose prompt starts with sysadmin@svrltreapcc02.
• Browser is a Windows or MacBook browser used to retrieve the existing ORCID Sandbox application credential and later test login.

Copy each command as one complete line. Do not copy a displayed prompt. If less opens a file, press q to return to the command prompt.

Kit 0.3.14 includes the root-only persistent Injected-secret authority and boot-time projector qualified in kit 0.3.6, plus corrected canonical ORCID admission. PostgreSQL and the Trusted runtime start only after the projector has recreated and verified their files under Ubuntu’s ephemeral /run filesystem.

The server directory is the release-bound v0.10.8 package composed for kit 0.3.14. Its server manifest, site configuration, validator image, and checksums are one unit; do not combine them with an earlier kit.

Never put a password, private key, client secret, authorization code, or token in the repository, release kit, command-line argument, screenshot, ticket, or retained terminal log.

What “empty MinisForum” means

This procedure requires all of the following to be absent on the MinisForum:

• an ahri-tre-postgresql container;
• /data/ahri-tre/postgresql;
• /etc/ahri-tre/config.toml; and
• an existing AHRI TRE Managed-secret store or Datastore.

An empty server does not mean an empty certificate and credential history. The published site kit already contains the Runtime public certificate, the PostgreSQL public CA, and the public ORCID client ID. Before starting, the site operator must still possess:

1. the Runtime private key matching the certificate in kit 0.3.14;
2. the PostgreSQL CA private key matching the public CA in kit 0.3.14; and
3. the ORCID Sandbox client secret for client ID APP-267KB7OA1UIVOI14.

For this installation, place the two retained private keys at these protected paths in WSL2, outside the repository:

/home/kobus/ahri-tre-pki/private/runtime-private-key.pem
/home/kobus/ahri-tre-pki/private/postgresql-ca-private-key.pem

If either private key is unavailable, stop. Do not generate a replacement and pair it with the published public material. Issue new certificates and build a new immutable site kit instead. If the intent is to restore an old Datastore, stop and use a recovery procedure with its original Deployment root identity; this fresh-install procedure creates a new identity.

1. MinisForum: verify the host foundation

Install Ubuntu 26.04 LTS x86-64, reserve 192.168.31.75 for this host, set the hostname to svrltreapcc02, and mount the persistent data filesystem at /data. Then connect by SSH and run:

hostname -s

Expected: svrltreapcc02.

ip -4 -o address show | grep -F ' 192.168.31.75/'

The command must print the assigned address.

findmnt /data

df -h /data

Both commands must show the intended persistent data filesystem. Do not continue if /data is merely a directory on the Ubuntu root filesystem.

Confirm that this really is a fresh installation:

sudo test ! -e /data/ahri-tre/postgresql && echo 'PostgreSQL data path is unused'

sudo test ! -e /etc/ahri-tre/config.toml && echo 'AHRI TRE configuration is absent'

sudo docker inspect ahri-tre-postgresql >/dev/null 2>&1; test $? -ne 0 && echo 'PostgreSQL container is absent'

All three commands must print the stated confirmation. If docker is not yet installed, the final command may instead report that sudo: docker is not found; that is acceptable at this point.

2. MinisForum: install operating-system prerequisites

Install the utilities consumed by the release-bound scripts:

sudo apt-get update

sudo apt-get install -y age ca-certificates curl jq openssl

The host-side PostgreSQL operator uses PostgreSQL 18 psql, pg_dump, and pg_restore even though the database server itself runs in Docker. Configure the PostgreSQL project’s repository and install only its client package by following the official Ubuntu package instructions:

sudo apt-get install -y postgresql-common

sudo /usr/share/postgresql-common/pgdg/apt.postgresql.org.sh

sudo apt-get update

sudo apt-get install -y postgresql-client-18

psql --version

pg_dump --version

Both version commands must report PostgreSQL 18. Do not install an Ubuntu PostgreSQL server package; the released managed container owns the server.

Install Docker Engine using Docker’s current official Ubuntu instructions. Use one supported Docker installation; do not mix Ubuntu’s conflicting Docker packages with Docker’s official packages.

Verify the result on the MinisForum:

sudo systemctl enable --now docker.service

sudo docker version

sudo docker network inspect bridge --format '{{(index .IPAM.Config 0).Gateway}}'

For kit 0.3.14 the last command must print 172.17.0.1. The generated HBA policy is restricted to that exact bridge gateway. Stop rather than editing the generated policy if the address differs.

3. WSL2: verify retained certificate authority material

Run these commands in WSL2, not on the MinisForum:

sudo test -s /home/kobus/ahri-tre-pki/private/runtime-private-key.pem

sudo test -s /home/kobus/ahri-tre-pki/private/postgresql-ca-private-key.pem

test -s /home/kobus/ahri-tre-pki/public/postgresql-ca-chain.pem

Confirm that the PostgreSQL CA private key matches its public certificate without displaying the key:

sudo openssl pkey -in /home/kobus/ahri-tre-pki/private/postgresql-ca-private-key.pem -pubout -outform DER | sha256sum

openssl x509 -in /home/kobus/ahri-tre-pki/public/postgresql-ca-chain.pem -pubkey -noout | openssl pkey -pubin -outform DER | sha256sum

The two SHA-256 values must be identical. Stop if they differ.

4. WSL2: download and verify the published kit

Create a version-specific download directory:

mkdir -p ~/ahri-tre-install/v0.10.8

Download the complete site kit and checksum from GitHub:

gh release download v0.10.8 --repo AHRIORG/ahri-tre-rs --pattern 'ahri-tre-test-datastore-deployment-kit-0.3.14.tar.gz' --pattern 'ahri-tre-test-datastore-deployment-kit-0.3.14.tar.gz.sha256' --dir ~/ahri-tre-install/v0.10.8

Verify the archive:

cd ~/ahri-tre-install/v0.10.8

sha256sum --check ahri-tre-test-datastore-deployment-kit-0.3.14.tar.gz.sha256

Stop unless the result is:

ahri-tre-test-datastore-deployment-kit-0.3.14.tar.gz: OK

5. WSL2: transfer the public kit

Create a protected transfer directory on the MinisForum:

ssh sysadmin@192.168.31.75 'umask 077; mkdir -p ~/ahri-tre-transfer/v0.10.8'

Copy the verified release files:

scp ahri-tre-test-datastore-deployment-kit-0.3.14.tar.gz ahri-tre-test-datastore-deployment-kit-0.3.14.tar.gz.sha256 sysadmin@192.168.31.75:ahri-tre-transfer/v0.10.8/

The kit contains only public configuration, certificates, binaries, the validator image, and provenance evidence. Secret material is projected separately below.

6. MinisForum: verify and extract the kit

Connect from WSL2:

ssh sysadmin@192.168.31.75

On the MinisForum, run:

cd ~/ahri-tre-transfer/v0.10.8

sha256sum --check ahri-tre-test-datastore-deployment-kit-0.3.14.tar.gz.sha256

Stop unless the result says OK. Then extract and enter the kit:

tar -xzf ahri-tre-test-datastore-deployment-kit-0.3.14.tar.gz

cd ~/ahri-tre-transfer/v0.10.8/ahri-tre-test-datastore-deployment-kit-0.3.14

All relative server/... and site/... paths from this point refer to this directory on the MinisForum.

Confirm the public site identity:

jq '{hostname, ipv4_address, kit_version, datastore_id, runtime_dns_name, client_id: .oidc.client_id, validator: .validator_release.image_reference}' site/site-inputs.json

Expected values include svrltreapcc02, 192.168.31.75, 0.3.14, ahri-tre-test, runtime.svrltreapcc02.home.arpa, APP-267KB7OA1UIVOI14, and validator 0.10.8.

7. MinisForum: install local name resolution

The Runtime HTTPS name resolves to the MinisForum LAN address. PostgreSQL’s TLS name resolves locally while the connection route remains loopback-only.

Check for conflicting entries:

grep -nE 'runtime\.svrltreapcc02\.home\.arpa|postgres\.svrltreapcc02\.home\.arpa' /etc/hosts || true

If either name already maps to another address, correct that entry instead of adding a duplicate. Otherwise add the two mappings:

printf '%s\n' '192.168.31.75 runtime.svrltreapcc02.home.arpa' '127.0.0.1 postgres.svrltreapcc02.home.arpa' | sudo tee -a /etc/hosts >/dev/null

Verify them:

getent ahostsv4 runtime.svrltreapcc02.home.arpa

getent ahostsv4 postgres.svrltreapcc02.home.arpa

The first output must include 192.168.31.75; the second must include 127.0.0.1.

Before client qualification, also apply the Runtime mapping from site/name-resolution.md to Windows, WSL2, and the MacBook. Do not map the PostgreSQL name on clients; PostgreSQL is not exposed to the LAN.

Review the generated firewall plan:

less site/firewall-plan.json

It requires default-deny inbound traffic, SSH administration, Runtime HTTPS from 192.168.31.0/24, and no non-loopback PostgreSQL access. Press q, then apply that policy with Ubuntu’s firewall. Allow SSH before enabling the firewall so the current connection is not locked out:

sudo apt-get install -y ufw

sudo ufw default deny incoming

sudo ufw default allow outgoing

sudo ufw allow OpenSSH

sudo ufw allow from 192.168.31.0/24 to 192.168.31.75 port 443 proto tcp

sudo ufw deny 5432/tcp

sudo ufw enable

sudo ufw status verbose

Do not continue unless the existing SSH session remains connected and the reported policy matches site/firewall-plan.json.

8. MinisForum: install the AHRI TRE server package

For a blank host use install.sh, never upgrade.sh:

sudo server/install.sh

This installs the v0.10.8 binaries and creates the service identities, but it does not start the Runtime before its configuration and Secrets exist.

Verify the identities and component manifest:

id ahri-tre-runtime

getent group ahri-tre-oidc

sudo test -s /usr/share/ahri-tre/server/component-versions.json && echo 'Server manifest installed'

All three checks must succeed.

Create the root-owned shared projection namespaces before creating Secret leaves. Mode 0711 permits traversal to separately restricted leaf directories without making any Secret value readable:

sudo install -d -o root -g root -m 0755 /run/secrets
sudo install -d -o root -g root -m 0711 \
  /run/secrets/ahri-tre \
  /run/secrets/oidc \
  /run/secrets/postgres \
  /run/secrets/postgres/tls \
  /run/secrets/runtime

Verify the shared namespace contract:

test "$(sudo stat -c '%U:%G:%a' /run/secrets)" = root:root:755
for namespace in ahri-tre oidc postgres postgres/tls runtime; do
  test "$(sudo stat -c '%U:%G:%a' "/run/secrets/$namespace")" = root:root:711
done

9. MinisForum and WSL2: issue the PostgreSQL server certificate

On the MinisForum, generate the PostgreSQL server private key and public CSR. Numeric ownership 999:999 is the PostgreSQL identity inside the released container; Ubuntu does not need a host user named 999.

sudo install -d -m 0700 /run/secrets/postgres/tls/private-key

sudo chown 999:999 /run/secrets/postgres/tls/private-key

sudo env OPENSSL_CONF=/dev/null openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:3072 -out /run/secrets/postgres/tls/private-key/value

sudo chown 999:999 /run/secrets/postgres/tls/private-key/value

sudo chmod 0400 /run/secrets/postgres/tls/private-key/value

sudo env OPENSSL_CONF=/dev/null openssl req -new -key /run/secrets/postgres/tls/private-key/value -subj '/CN=postgres.svrltreapcc02.home.arpa' -addext 'subjectAltName=DNS:postgres.svrltreapcc02.home.arpa' -out /tmp/postgres.svrltreapcc02.home.arpa.csr

sudo chown sysadmin:sysadmin /tmp/postgres.svrltreapcc02.home.arpa.csr

In WSL2, copy only the public CSR back and verify it:

install -d -m 0700 /home/kobus/ahri-tre-pki/requests

scp sysadmin@192.168.31.75:/tmp/postgres.svrltreapcc02.home.arpa.csr /home/kobus/ahri-tre-pki/requests/

openssl req -in /home/kobus/ahri-tre-pki/requests/postgres.svrltreapcc02.home.arpa.csr -verify -noout

Still in WSL2, create the public certificate extension file:

printf '%s\n' 'basicConstraints=critical,CA:FALSE' 'keyUsage=critical,digitalSignature,keyEncipherment' 'extendedKeyUsage=serverAuth' 'subjectAltName=DNS:postgres.svrltreapcc02.home.arpa' > /tmp/postgresql-server-certificate.ext

Sign the public CSR with the retained PostgreSQL CA:

sudo openssl x509 -req -sha256 -days 825 -in /home/kobus/ahri-tre-pki/requests/postgres.svrltreapcc02.home.arpa.csr -CA /home/kobus/ahri-tre-pki/public/postgresql-ca-chain.pem -CAkey /home/kobus/ahri-tre-pki/private/postgresql-ca-private-key.pem -CAcreateserial -extfile /tmp/postgresql-server-certificate.ext -out /home/kobus/ahri-tre-pki/public/postgres.svrltreapcc02.home.arpa.pem

Copy only the public leaf certificate to the MinisForum:

scp /home/kobus/ahri-tre-pki/public/postgres.svrltreapcc02.home.arpa.pem sysadmin@192.168.31.75:ahri-tre-transfer/v0.10.8/

On the MinisForum, install and verify it:

sudo install -D -m 0444 ~/ahri-tre-transfer/v0.10.8/postgres.svrltreapcc02.home.arpa.pem /etc/ahri-tre/postgresql/tls/certificate.pem

sudo chown 999:999 /etc/ahri-tre/postgresql/tls/certificate.pem

sudo env OPENSSL_CONF=/dev/null openssl verify -CAfile site/postgresql/public-ca-chain.pem /etc/ahri-tre/postgresql/tls/certificate.pem

sudo env OPENSSL_CONF=/dev/null openssl x509 -in /etc/ahri-tre/postgresql/tls/certificate.pem -noout -checkhost postgres.svrltreapcc02.home.arpa

sudo stat -c '%u:%g:%a %n' /etc/ahri-tre/postgresql/tls/certificate.pem /run/secrets/postgres/tls/private-key/value

The certificate check must succeed. The final output must show 999:999:444 for the certificate and 999:999:400 for the private key.

10. MinisForum: create the PostgreSQL password projections

Create the protected directories:

sudo install -d -o root -g root -m 0700 /run/secrets/postgres/bootstrap-password /run/secrets/postgres/administrator-passfile

sudo install -d -o ahri-tre-runtime -g ahri-tre-runtime -m 0700 /run/secrets/postgres/administrator-password

Start a temporary root shell:

sudo bash

The prompt changes from $ to #. Paste this block at the # prompt:

set -eu
umask 077
postgres_password="$(env OPENSSL_CONF=/dev/null openssl rand -hex 32)"
printf '%s' "$postgres_password" > /run/secrets/postgres/bootstrap-password/value
printf '%s' "$postgres_password" > /run/secrets/postgres/administrator-password/value
printf '%s\n' "postgres.svrltreapcc02.home.arpa:5432:*:ahri_tre_administrator:${postgres_password}" > /run/secrets/postgres/administrator-passfile/value
unset postgres_password
printf '%s' 'site-v3' > /run/secrets/postgres/administrator-password/version
chown root:root /run/secrets/postgres/bootstrap-password/value /run/secrets/postgres/administrator-passfile/value
chmod 0400 /run/secrets/postgres/bootstrap-password/value
chmod 0600 /run/secrets/postgres/administrator-passfile/value
chown ahri-tre-runtime:ahri-tre-runtime /run/secrets/postgres/administrator-password/value /run/secrets/postgres/administrator-password/version
chmod 0400 /run/secrets/postgres/administrator-password/value /run/secrets/postgres/administrator-password/version
exit

The prompt returns to $. Verify equality without displaying the password:

sudo cmp --silent /run/secrets/postgres/bootstrap-password/value /run/secrets/postgres/administrator-password/value && echo 'PostgreSQL password projections match'

The confirmation must be printed.

11. Browser and MinisForum: project the ORCID client secret

In a browser, sign in to ORCID Sandbox Developer Tools, open application APP-267KB7OA1UIVOI14, and copy its client secret. Confirm that its redirect URI is exactly:

https://runtime.svrltreapcc02.home.arpa/v1/runtime-login/callback

Back at the MinisForum SSH prompt, create the protected directory:

sudo install -d -o root -g ahri-tre-oidc -m 0750 /run/secrets/oidc/client-secret

Start a temporary root shell:

sudo bash

At the # prompt, run:

IFS= read -r -s -p 'Paste the ORCID Sandbox client secret, then press Enter: ' orcid_client_secret

Paste the secret and press Enter. No characters are displayed. Then run:

printf '\n'
test -n "$orcid_client_secret"
umask 027
printf '%s' "$orcid_client_secret" > /run/secrets/oidc/client-secret/value
unset orcid_client_secret
printf '%s' 'site-v3' > /run/secrets/oidc/client-secret/version
chown root:ahri-tre-oidc /run/secrets/oidc/client-secret/value /run/secrets/oidc/client-secret/version
chmod 0440 /run/secrets/oidc/client-secret/value /run/secrets/oidc/client-secret/version
exit

Verify only ownership and modes:

sudo stat -c '%U:%G %a %n' /run/secrets/oidc/client-secret/value /run/secrets/oidc/client-secret/version

Both lines must start with root:ahri-tre-oidc 440.

12. WSL2 and MinisForum: project the Runtime private key

In WSL2, copy the retained matching Runtime key to a temporary protected file on the MinisForum:

sudo install -o kobus -g kobus -m 0400 /home/kobus/ahri-tre-pki/private/runtime-private-key.pem /tmp/runtime-private-key.transfer

scp /tmp/runtime-private-key.transfer sysadmin@192.168.31.75:runtime-private-key.transfer

rm -- /tmp/runtime-private-key.transfer

On the MinisForum, project it and remove the transfer copy:

sudo install -d -o ahri-tre-runtime -g ahri-tre-runtime -m 0700 /run/secrets/runtime/private-key

sudo install -o ahri-tre-runtime -g ahri-tre-runtime -m 0400 /home/sysadmin/runtime-private-key.transfer /run/secrets/runtime/private-key/value

printf '%s' 'site-v3' | sudo tee /run/secrets/runtime/private-key/version >/dev/null

sudo chown ahri-tre-runtime:ahri-tre-runtime /run/secrets/runtime/private-key/version

sudo chmod 0400 /run/secrets/runtime/private-key/version

rm -- /home/sysadmin/runtime-private-key.transfer

sudo -u ahri-tre-runtime env OPENSSL_CONF=/dev/null openssl pkey -in /run/secrets/runtime/private-key/value -check -noout

The final command must report a valid key.

13. MinisForum and WSL2: create and back up the root identity

The root identity decrypts this Deployment’s Managed-secret store. It is not an ORCID identity or TLS key. Create it only once:

sudo install -d -o root -g root -m 0700 /root/ahri-tre-recovery

sudo test ! -e /root/ahri-tre-recovery/minisforum-root-identity.txt || { echo 'STOP: root identity already exists'; exit 1; }

sudo bash -c 'set -eu; umask 077; age-keygen | sed -n "/^AGE-SECRET-KEY-1/p" > /root/ahri-tre-recovery/minisforum-root-identity.txt; test -s /root/ahri-tre-recovery/minisforum-root-identity.txt'

Project it for the Runtime:

sudo install -d -o ahri-tre-runtime -g ahri-tre-runtime -m 0700 /run/secrets/ahri-tre/root-identity

sudo install -o ahri-tre-runtime -g ahri-tre-runtime -m 0400 /root/ahri-tre-recovery/minisforum-root-identity.txt /run/secrets/ahri-tre/root-identity/value

printf '%s' 'site-v3' | sudo tee /run/secrets/ahri-tre/root-identity/version >/dev/null

sudo chown ahri-tre-runtime:ahri-tre-runtime /run/secrets/ahri-tre/root-identity/version

sudo chmod 0400 /run/secrets/ahri-tre/root-identity/version

Create a temporary transfer copy:

sudo install -o sysadmin -g sysadmin -m 0400 /root/ahri-tre-recovery/minisforum-root-identity.txt /home/sysadmin/minisforum-root-identity.transfer

In WSL2, retrieve the separate recovery copy:

install -d -m 0700 /home/kobus/ahri-tre-recovery

scp sysadmin@192.168.31.75:/home/sysadmin/minisforum-root-identity.transfer /home/kobus/ahri-tre-recovery/minisforum-root-identity.txt

chmod 0400 /home/kobus/ahri-tre-recovery/minisforum-root-identity.txt

grep -q '^AGE-SECRET-KEY-1' /home/kobus/ahri-tre-recovery/minisforum-root-identity.txt && echo 'Separate root-identity backup is valid'

Back on the MinisForum, remove only the temporary transfer file:

rm -- /home/sysadmin/minisforum-root-identity.transfer

Keep the WSL2 recovery copy separate from Datastore backups.

14. MinisForum: verify every Secret projection

This check displays only paths, numeric identities, and modes:

sudo stat -c '%U(%u):%G(%g) %a %n' /run/secrets /run/secrets/ahri-tre /run/secrets/oidc /run/secrets/postgres /run/secrets/postgres/tls /run/secrets/runtime /run/secrets/postgres/bootstrap-password/value /run/secrets/postgres/administrator-password/value /run/secrets/postgres/administrator-password/version /run/secrets/postgres/administrator-passfile/value /run/secrets/postgres/tls/private-key/value /run/secrets/oidc/client-secret/value /run/secrets/oidc/client-secret/version /run/secrets/runtime/private-key/value /run/secrets/runtime/private-key/version /run/secrets/ahri-tre/root-identity/value /run/secrets/ahri-tre/root-identity/version

Compare the output with site/injected-secrets.json. Required results are:

• bootstrap password: root:root 400;
• administrator password and version: ahri-tre-runtime:ahri-tre-runtime 400;
• administrator passfile: root:root 600;
• PostgreSQL key: numeric 999:999 400;
• ORCID client secret and version: root:ahri-tre-oidc 440;
• Runtime key and version: ahri-tre-runtime:ahri-tre-runtime 400; and
• root identity and version: ahri-tre-runtime:ahri-tre-runtime 400.

The Ubuntu host may display UID/GID 999 with unrelated names. The numeric values are authoritative. Never use cat, less, head, or an editor on a Secret value file.

15. MinisForum: install the boot-time Secret projector

This explicit installation copies the current, verified projections into the root-only persistent authority at /var/lib/ahri-tre/injected-secret-authority. It then installs and starts the projector service. It does not print Secret contents.

sudo site/install-secret-projector.sh --confirm-host svrltreapcc02 --confirm-address 192.168.31.75

Expected:

installed and verified the boot-time Injected-secret projector

Verify both persistent sources and ephemeral projections:

sudo /usr/libexec/ahri-tre/secret-projector.sh verify

Expected: verified persistent and projected Injected secrets.

sudo systemctl is-active ahri-tre-secret-projector.service

Expected: active. Do not continue if capture or verification fails. Never open files beneath the persistent authority with cat, less, or an editor. The authority is Secret material, not a normal Datastore backup: exclude it from broad file backups and protect the MinisForum system disk and root account to the same standard as the original private keys and passwords.

16. MinisForum: install managed PostgreSQL

Review, but do not edit, the generated policy:

less site/postgresql/deployment-contract.json

less site/postgresql/pg_hba.conf

less site/postgresql/pg_ident.conf

Press q after each file. Then install PostgreSQL:

sudo site/postgresql/install.sh --confirm-host svrltreapcc02 --confirm-address 192.168.31.75

Wait for health:

sudo docker inspect --format '{{.Config.Image}} {{.State.Health.Status}}' ahri-tre-postgresql

Expected:

ahri-tre/postgresql-orcid-validator:0.10.8 healthy

Confirm PostgreSQL is loopback-only:

sudo ss -ltnp '( sport = :5432 )'

The local address must be 127.0.0.1:5432, never 0.0.0.0:5432 or 192.168.31.75:5432.

17. MinisForum: provision the Runtime and create the Datastore

Run the release-bound provisioner once:

sudo site/provision.sh --confirm-host svrltreapcc02 --confirm-address 192.168.31.75

The command installs Application configuration, initializes the Managed-secret store, starts the Runtime, and creates ahri-tre-test. It may print only the commands that perform visible work.

Verify everything explicitly:

sudo systemctl is-active ahri-tre-runtime.service

Expected: active.

sudo site/verify-readiness.sh

Expected: MinisForum v3 site is ready.

D=ahri-tre-test; R=/usr/libexec/ahri-tre/ahri-tre-runtime; sudo -u ahri-tre-runtime env LD_LIBRARY_PATH=/usr/lib/ahri-tre "$R" datastore reconcile "$D"

Expected: JSON containing "status":"ready". Record its public datastore_id; do not alter the binding.

18. MinisForum: prove reboot persistence

Reboot the MinisForum:

sudo reboot

The SSH connection closes. Wait about one minute, then reconnect from WSL2:

ssh sysadmin@192.168.31.75

On the MinisForum, verify the ordered services:

sudo systemctl is-active ahri-tre-secret-projector.service ahri-tre-postgresql.service ahri-tre-runtime.service

Expected: three lines, each saying active.

sudo /usr/libexec/ahri-tre/secret-projector.sh verify

cd ~/ahri-tre-transfer/v0.10.8/ahri-tre-test-datastore-deployment-kit-0.3.14
sudo site/verify-readiness.sh

Expected: projection verification succeeds and readiness prints MinisForum v3 site is ready. If any service or check fails, keep the persistent authority and all rollback material, and diagnose before continuing.

19. MinisForum: make the first Datastore backup

Create and verify a bounded backup:

sudo install -d -o root -g root -m 0700 /var/backups/ahri-tre/manual

sudo site/backup.sh /var/backups/ahri-tre/manual/initial-v0.10.8.dump

sudo find /var/backups/ahri-tre -maxdepth 2 -type f -printf '%TY-%Tm-%Td %TH:%TM %p\n' | sort

Keep the Deployment root identity backup separate from these data backups. Both are required for recovery.

20. Browser/client and MinisForum: qualify ORCID admission

On the MinisForum, admit one real ORCID Sandbox identity using its canonical public iD:

sudo site/admit-user.sh REPLACE_WITH_ADMITTED_ORCID_ID

Replace the final value with the real canonical hyphenated iD, in the form 0000-0000-0000-0000. Do not add the internal orcid_ role prefix.

From a configured WSL2 or MacBook client:

1. sign in with that Sandbox identity and open the ahri-tre-test Datastore Session; it must succeed;
2. sign out and use a second real, valid Sandbox identity that was not admitted; Session opening must be rejected; and
3. retain only the outcome, never a token.

The deterministic invalid-token evidence is already published in site/validator/qualification-result.json. Do not mint synthetic ORCID tokens on the MinisForum.

Only these public files may proceed to the client installation workflows:

• site/client.toml;
• site/public-ca-chain.pem; and
• site/name-resolution.md.

Do not publish Application configuration, PostgreSQL policy, Secret projections, private keys, credentials, or validator administration material.

Installing a future release

Do not replace version numbers mechanically. A future installation must use a site kit generated for the intended hostname, address, certificates, ORCID registration, storage paths, and release artifacts. Download its archive and checksum together, follow that kit’s release-specific requirements, and never mix files between kit versions.

Preparing Secrets for Clean-Host Installed-Package Conformance

This procedure prepares the external Secret inputs required before running the installed-package conformance harness on a clean MinisForum. It does not install the AHRI TRE server, PostgreSQL container, Secret projector, or site package. The conformance harness must remain the only package installer so that its evidence proves the behavior of the candidate archive.

The worked example uses:

• Ubuntu MinisForum svrltreapcc02 at 192.168.31.75;
• Datastore ahri-tre-test;
• candidate kit 0.3.14;
• fixed predecessor kit 0.3.12; and
• the site identity and public material embedded in the candidate archive.

There are three command locations. The controller may be the original WSL2 shell or an explicitly authorized Apple Silicon Mac that completed the guarded controller migration in the failed-recovery reset:

• Controller holds the candidate, predecessor, and protected authority or migrated candidate-bound leaf material.
• MinisForum is an SSH session whose prompt starts with sysadmin@svrltreapcc02.
• Browser is used only to retrieve the ORCID Sandbox application Secret.

Guided preparation wizard

The dedicated wizard performs Sections 1 through 12 in order, including the browser-assisted ORCID Secret step. Run it from the repository root on the recorded WSL2 controller or its authorized macOS replacement:

./scripts/minisforum-conformance-preparation-wizard.sh

The wizard is fixed to the candidate, predecessor, host, address, Deployment, Datastore, and public identities documented below. It verifies those values against the candidate rather than accepting overrides. It never writes a Secret to .env, the repository, a command-line argument, or the evidence directory. It stops after printing HOST READY; it does not create evidence or invoke the conformance harness. Continue with Section 13 only after that ready result.

If it is restarted after an interruption, the wizard validates and reuses a complete projection but refuses partial state. Reusing an already generated root identity also requires explicit confirmation that it came only from the interrupted preparation and has never been used by an installation; its separate backup must identify the same public recipient.

When the PostgreSQL CA signing key remains available, the wizard continues to issue a fresh PostgreSQL leaf certificate. When the unavailable controller was replaced by macOS, it instead accepts only the migrated PostgreSQL leaf key and certificate produced by the guarded reset. It verifies their key binding, hostname, and trust beneath the replacement candidate’s embedded PostgreSQL CA before reprojecting them. The Runtime migration follows the same public-key binding check.

The detailed procedure below remains the canonical explanation of every wizard action and can be used to inspect or recover an interrupted preparation.

Never paste a password, private key, client Secret, authorization code, or token into a command-line argument, terminal log, screenshot, evidence bundle, repository file, or release archive. Commands below inspect only public material or safe ownership and mode metadata.

What this preparation may create

Before the harness starts, this procedure creates only:

• the ahri-tre-runtime and ahri-tre-web service identities and their declared groups;
• temporary Secret projections beneath /run/secrets;
• the public PostgreSQL leaf certificate under /etc/ahri-tre/postgresql/tls;
• a new Deployment root identity under /root/ahri-tre-recovery; and
• a separately protected root-identity backup in WSL2.

It must not create an AHRI TRE Managed-secret store, PostgreSQL data directory, PostgreSQL container, installed server manifest, or installed Runtime binary. Creating service identities alone does not install package files. The harness will run the predecessor installer later; systemd-sysusers safely confirms the already prepared identities at that point.

Required inputs

Obtain all four public files through the controlled candidate handoff:

ahri-tre-test-datastore-deployment-kit-0.3.14.tar.gz
ahri-tre-test-datastore-deployment-kit-0.3.14.tar.gz.sha256
ahri-tre-test-datastore-deployment-kit-0.3.12.tar.gz
ahri-tre-test-datastore-deployment-kit-0.3.12.tar.gz.sha256

If you are also the candidate composer, build the server, site, and WSL2 publications in one fresh component root, compose the macOS publication from the pinned Apple release and the extracted checksummed WSL2 generation client, then run:

candidate_output="$PWD/dist/conformance-candidate-postgresql-ownership-output-0.3.14"
component_root="$PWD/dist/conformance-candidate-postgresql-ownership-0.3.14"
install -d -m 0700 "$candidate_output"
./scripts/compose-installed-conformance-candidate.sh \
  --component-root "$component_root" \
  --output-directory "$candidate_output"

The component root must contain exactly server/, site/, and both client publications under clients/. The composer independently verifies their identities and checksums, embeds the committed harness, creates the complete candidate checksum inventory, and refuses to overwrite an earlier candidate. It never installs a package or invokes the harness. The macOS publication’s native install and launch remain part of later Apple Silicon conformance.

The candidate checksum must have been produced outside the candidate archive and must identify the exact candidate bytes used on every conformance host. The candidate digest for this qualification is:

bdd0fe785ed333caf1f9c7de8b70479d5c37b5526d7ace6e28de74105c2cab48

The fixed predecessor digest is:

6f5555c25d96274d7772d1b46d4409bd36a42205f21c4c715de8df5179ab18ed

The WSL2 operator must also possess:

• the Runtime private key matching the Runtime certificate embedded in the candidate Application configuration;
• the PostgreSQL CA private key matching the candidate’s PostgreSQL public CA; and
• access to the ORCID Sandbox application whose public client ID is embedded in the candidate.

If either private key is unavailable or does not match the candidate, stop. Do not generate a replacement key and pair it with unrelated public material. A changed authority requires new public certificates and a newly built candidate archive. If only the Runtime leaf key is unavailable and no conformance phase has begun, follow Recovering a Conformance Candidate After Runtime Key Loss to issue a new leaf beneath the retained deployment CA, regenerate the site, and compose replacement candidate bytes before restarting this procedure.

1. MinisForum: reconfirm the clean host

Connect from WSL2:

ssh sysadmin@192.168.31.75

On the MinisForum, verify the retained host foundation:

hostname -s

Expected: svrltreapcc02.

ip -4 -o address show | grep -F ' 192.168.31.75/'

findmnt --mountpoint /data

sudo docker network inspect bridge --format '{{(index .IPAM.Config 0).Gateway}}'

The address must be assigned, /data must be the intended separate filesystem, and the Docker bridge gateway must be 172.17.0.1.

Verify the installation boundary is empty:

sudo test ! -e /usr/share/ahri-tre/server/component-versions.json
sudo test ! -e /usr/libexec/ahri-tre/ahri-tre-runtime
sudo test ! -e /var/lib/ahri-tre/secrets
sudo test ! -e /data/ahri-tre/postgresql
sudo docker inspect ahri-tre-postgresql >/dev/null 2>&1; test $? -ne 0

Every command must exit zero. Return to WSL2:

exit

2. WSL2: verify the private authority material

The example uses these protected paths outside the repository:

/home/kobus/ahri-tre-pki/private/runtime-private-key.pem
/home/kobus/ahri-tre-pki/private/postgresql-ca-private-key.pem
/home/kobus/ahri-tre-pki/public/postgresql-ca-chain.pem

Verify their presence without printing their contents:

sudo test -s /home/kobus/ahri-tre-pki/private/runtime-private-key.pem

sudo test -s /home/kobus/ahri-tre-pki/private/postgresql-ca-private-key.pem

test -s /home/kobus/ahri-tre-pki/public/postgresql-ca-chain.pem

Confirm that the PostgreSQL CA private key matches its public certificate:

sudo openssl pkey -in /home/kobus/ahri-tre-pki/private/postgresql-ca-private-key.pem -pubout -outform DER | sha256sum

openssl x509 -in /home/kobus/ahri-tre-pki/public/postgresql-ca-chain.pem -pubkey -noout | openssl pkey -pubin -outform DER | sha256sum

The two SHA-256 values must be identical.

3. WSL2: verify and extract the candidate inputs

Place the four public input files in a protected directory:

install -d -m 0700 /home/kobus/ahri-tre-conformance/input

cd /home/kobus/ahri-tre-conformance/input

The candidate checksum declaration must contain the exact admitted digest, two spaces, and the candidate archive basename:

grep -qxF 'bdd0fe785ed333caf1f9c7de8b70479d5c37b5526d7ace6e28de74105c2cab48  ahri-tre-test-datastore-deployment-kit-0.3.14.tar.gz' ahri-tre-test-datastore-deployment-kit-0.3.14.tar.gz.sha256

sha256sum --check --strict ahri-tre-test-datastore-deployment-kit-0.3.14.tar.gz.sha256

Expected: the candidate archive reports OK.

Verify the predecessor declaration and bytes against the fixed digest:

grep -qxF '6f5555c25d96274d7772d1b46d4409bd36a42205f21c4c715de8df5179ab18ed  ahri-tre-test-datastore-deployment-kit-0.3.12.tar.gz' ahri-tre-test-datastore-deployment-kit-0.3.12.tar.gz.sha256

sha256sum --check --strict ahri-tre-test-datastore-deployment-kit-0.3.12.tar.gz.sha256

Expected: the predecessor archive reports OK.

Extract review copies without applying archive ownership or permissions:

test ! -e review
install -d -m 0700 review

tar --extract --gzip --file ahri-tre-test-datastore-deployment-kit-0.3.14.tar.gz --directory review --no-same-owner --no-same-permissions

tar --extract --gzip --file ahri-tre-test-datastore-deployment-kit-0.3.12.tar.gz --directory review --no-same-owner --no-same-permissions

Define the reviewed roots for this WSL2 shell:

CANDIDATE_ROOT=/home/kobus/ahri-tre-conformance/input/review/ahri-tre-test-datastore-deployment-kit-0.3.14
PREDECESSOR_ROOT=/home/kobus/ahri-tre-conformance/input/review/ahri-tre-test-datastore-deployment-kit-0.3.12

test -x "$CANDIDATE_ROOT/conformance/run.sh"
test -f "$CANDIDATE_ROOT/site/site-inputs.json"
test -f "$PREDECESSOR_ROOT/server/sysusers/ahri-tre.conf"

4. WSL2: bind the private material to the candidate

Review the candidate’s non-secret site identity:

jq '{kit_version, hostname, ipv4_address, deployment_id, datastore_id, runtime_dns_name, runtime_https_port, postgresql: {logical_host: .postgresql.logical_host, routing_address: .postgresql.routing_address}, oidc_client_id: .oidc.client_id}' "$CANDIDATE_ROOT/site/site-inputs.json"

Stop unless it names the intended MinisForum, address, Deployment, Datastore, Runtime origin, PostgreSQL route, and ORCID public client ID. Record the Deployment UUID; it is public confirmation input for every later conformance phase.

Confirm that the candidate contains only the declared site-v3 Injected version:

jq -e '[.requirements[].expected_version] | unique == ["site-v3"]' "$CANDIDATE_ROOT/site/injected-secrets.json"

Confirm that the retained PostgreSQL CA is byte-for-byte identical to the candidate public CA:

cmp --silent /home/kobus/ahri-tre-pki/public/postgresql-ca-chain.pem "$CANDIDATE_ROOT/site/postgresql/public-ca-chain.pem" && echo 'PostgreSQL CA matches candidate'

Extract the public Runtime leaf certificate from the candidate Application configuration:

python3 -c 'import sys,tomllib; document=tomllib.load(open(sys.argv[1], "rb")); print(document["services"]["trusted_runtime"]["certificate_chain"][0], end="")' "$CANDIDATE_ROOT/site/application.toml" > /tmp/ahri-tre-candidate-runtime-certificate.pem

Verify its chain and DNS identity:

openssl verify -CAfile "$CANDIDATE_ROOT/site/public-ca-chain.pem" /tmp/ahri-tre-candidate-runtime-certificate.pem

openssl x509 -in /tmp/ahri-tre-candidate-runtime-certificate.pem -noout -checkhost runtime.svrltreapcc02.home.arpa

Compare the public key from that certificate with the retained Runtime private key:

openssl x509 -in /tmp/ahri-tre-candidate-runtime-certificate.pem -pubkey -noout | openssl pkey -pubin -outform DER | sha256sum

sudo openssl pkey -in /home/kobus/ahri-tre-pki/private/runtime-private-key.pem -pubout -outform DER | sha256sum

The two SHA-256 values must be identical. Remove the temporary public leaf certificate:

rm -- /tmp/ahri-tre-candidate-runtime-certificate.pem

5. WSL2 and MinisForum: transfer and verify the public inputs

Create a protected input directory on the MinisForum:

ssh sysadmin@192.168.31.75 'umask 077; mkdir -p ~/ahri-tre-conformance/input'

Transfer the same four verified files:

scp /home/kobus/ahri-tre-conformance/input/ahri-tre-test-datastore-deployment-kit-0.3.14.tar.gz /home/kobus/ahri-tre-conformance/input/ahri-tre-test-datastore-deployment-kit-0.3.14.tar.gz.sha256 /home/kobus/ahri-tre-conformance/input/ahri-tre-test-datastore-deployment-kit-0.3.12.tar.gz /home/kobus/ahri-tre-conformance/input/ahri-tre-test-datastore-deployment-kit-0.3.12.tar.gz.sha256 sysadmin@192.168.31.75:ahri-tre-conformance/input/

Connect and verify the transferred bytes again:

ssh sysadmin@192.168.31.75

cd ~/ahri-tre-conformance/input

grep -qxF 'bdd0fe785ed333caf1f9c7de8b70479d5c37b5526d7ace6e28de74105c2cab48  ahri-tre-test-datastore-deployment-kit-0.3.14.tar.gz' ahri-tre-test-datastore-deployment-kit-0.3.14.tar.gz.sha256
sha256sum --check --strict ahri-tre-test-datastore-deployment-kit-0.3.14.tar.gz.sha256

grep -qxF '6f5555c25d96274d7772d1b46d4409bd36a42205f21c4c715de8df5179ab18ed  ahri-tre-test-datastore-deployment-kit-0.3.12.tar.gz' ahri-tre-test-datastore-deployment-kit-0.3.12.tar.gz.sha256
sha256sum --check --strict ahri-tre-test-datastore-deployment-kit-0.3.12.tar.gz.sha256

Extract both public archives without installing them:

test ! -e extracted
install -d -m 0700 extracted

tar --extract --gzip --file ahri-tre-test-datastore-deployment-kit-0.3.14.tar.gz --directory extracted --no-same-owner --no-same-permissions
tar --extract --gzip --file ahri-tre-test-datastore-deployment-kit-0.3.12.tar.gz --directory extracted --no-same-owner --no-same-permissions

Define the MinisForum roots for the current SSH shell:

INPUT_ROOT=/home/sysadmin/ahri-tre-conformance/input
CANDIDATE_ROOT="$INPUT_ROOT/extracted/ahri-tre-test-datastore-deployment-kit-0.3.14"
PREDECESSOR_ROOT="$INPUT_ROOT/extracted/ahri-tre-test-datastore-deployment-kit-0.3.12"

6. MinisForum: create only the service identities

Use the fixed predecessor’s declared system identities. Do not run server/install.sh or systemd-tmpfiles:

sudo systemd-sysusers "$PREDECESSOR_ROOT/server/sysusers/ahri-tre.conf"

Verify the required identities:

id ahri-tre-runtime
id ahri-tre-web
getent group ahri-tre
getent group ahri-tre-oidc

Create the root-owned projection root and shared traversal namespaces. Mode 0711 permits traversal to the separately restricted leaf directories; it does not make any Secret value readable:

sudo install -d -o root -g root -m 0755 /run/secrets
sudo install -d -o root -g root -m 0711 \
  /run/secrets/ahri-tre \
  /run/secrets/oidc \
  /run/secrets/postgres \
  /run/secrets/postgres/tls \
  /run/secrets/runtime

Verify this boundary before creating any leaf:

test "$(sudo stat -c '%U:%G:%a' /run/secrets)" = root:root:755
for namespace in ahri-tre oidc postgres postgres/tls runtime; do
  test "$(sudo stat -c '%U:%G:%a' "/run/secrets/$namespace")" = root:root:711
done

Recheck that no package or Managed-store boundary was installed:

sudo test ! -e /usr/share/ahri-tre/server/component-versions.json
sudo test ! -e /usr/libexec/ahri-tre/ahri-tre-runtime
sudo test ! -e /var/lib/ahri-tre/secrets

7. MinisForum and WSL2: issue the PostgreSQL leaf certificate

On the MinisForum, create a new PostgreSQL private key and public CSR. UID and GID 999 are the declared container identity; no host account named 999 is required.

sudo install -d -m 0700 /run/secrets/postgres/tls/private-key
sudo chown 999:999 /run/secrets/postgres/tls/private-key

sudo env OPENSSL_CONF=/dev/null openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:3072 -out /run/secrets/postgres/tls/private-key/value
sudo chown 999:999 /run/secrets/postgres/tls/private-key/value
sudo chmod 0400 /run/secrets/postgres/tls/private-key/value

sudo env OPENSSL_CONF=/dev/null openssl req -new -key /run/secrets/postgres/tls/private-key/value -subj '/CN=postgres.svrltreapcc02.home.arpa' -addext 'subjectAltName=DNS:postgres.svrltreapcc02.home.arpa' -out /tmp/postgres.svrltreapcc02.home.arpa.csr
sudo chown sysadmin:sysadmin /tmp/postgres.svrltreapcc02.home.arpa.csr

In WSL2, copy and verify only the public CSR:

install -d -m 0700 /home/kobus/ahri-tre-pki/requests
scp sysadmin@192.168.31.75:/tmp/postgres.svrltreapcc02.home.arpa.csr /home/kobus/ahri-tre-pki/requests/
openssl req -in /home/kobus/ahri-tre-pki/requests/postgres.svrltreapcc02.home.arpa.csr -verify -noout

Create the public leaf-certificate extensions:

printf '%s\n' 'basicConstraints=critical,CA:FALSE' 'keyUsage=critical,digitalSignature,keyEncipherment' 'extendedKeyUsage=serverAuth' 'subjectAltName=DNS:postgres.svrltreapcc02.home.arpa' > /tmp/postgresql-server-certificate.ext

Sign the CSR with the candidate-matching PostgreSQL CA:

sudo openssl x509 -req -sha256 -days 825 -in /home/kobus/ahri-tre-pki/requests/postgres.svrltreapcc02.home.arpa.csr -CA /home/kobus/ahri-tre-pki/public/postgresql-ca-chain.pem -CAkey /home/kobus/ahri-tre-pki/private/postgresql-ca-private-key.pem -CAcreateserial -extfile /tmp/postgresql-server-certificate.ext -out /home/kobus/ahri-tre-pki/public/postgres.svrltreapcc02.home.arpa.pem

Copy only the public leaf certificate to the MinisForum:

scp /home/kobus/ahri-tre-pki/public/postgres.svrltreapcc02.home.arpa.pem sysadmin@192.168.31.75:ahri-tre-conformance/input/

On the MinisForum, install and verify the public certificate:

sudo install -D -m 0444 /home/sysadmin/ahri-tre-conformance/input/postgres.svrltreapcc02.home.arpa.pem /etc/ahri-tre/postgresql/tls/certificate.pem
sudo chown 999:999 /etc/ahri-tre/postgresql/tls/certificate.pem

sudo env OPENSSL_CONF=/dev/null openssl verify -CAfile "$CANDIDATE_ROOT/site/postgresql/public-ca-chain.pem" /etc/ahri-tre/postgresql/tls/certificate.pem
sudo env OPENSSL_CONF=/dev/null openssl x509 -in /etc/ahri-tre/postgresql/tls/certificate.pem -noout -checkhost postgres.svrltreapcc02.home.arpa

sudo stat -c '%u:%g:%a %n' /etc/ahri-tre/postgresql/tls/certificate.pem /run/secrets/postgres/tls/private-key/value

Expected modes are 999:999:444 for the public certificate and 999:999:400 for the private key. Remove the public CSR from the MinisForum:

rm -- /tmp/postgres.svrltreapcc02.home.arpa.csr

8. MinisForum: create fresh PostgreSQL password projections

Create the protected directories:

sudo install -d -o root -g root -m 0700 /run/secrets/postgres/bootstrap-password /run/secrets/postgres/administrator-passfile
sudo install -d -o ahri-tre-runtime -g ahri-tre-runtime -m 0700 /run/secrets/postgres/administrator-password

Start a temporary root shell:

sudo bash

At the # prompt, paste this complete block:

set -eu
umask 077
postgres_password="$(env OPENSSL_CONF=/dev/null openssl rand -hex 32)"
printf '%s' "$postgres_password" > /run/secrets/postgres/bootstrap-password/value
printf '%s' "$postgres_password" > /run/secrets/postgres/administrator-password/value
printf '%s\n' "postgres.svrltreapcc02.home.arpa:5432:*:ahri_tre_administrator:${postgres_password}" > /run/secrets/postgres/administrator-passfile/value
unset postgres_password
printf '%s' 'site-v3' > /run/secrets/postgres/administrator-password/version
chown root:root /run/secrets/postgres/bootstrap-password/value /run/secrets/postgres/administrator-passfile/value
chmod 0400 /run/secrets/postgres/bootstrap-password/value
chmod 0600 /run/secrets/postgres/administrator-passfile/value
chown ahri-tre-runtime:ahri-tre-runtime /run/secrets/postgres/administrator-password/value /run/secrets/postgres/administrator-password/version
chmod 0400 /run/secrets/postgres/administrator-password/value /run/secrets/postgres/administrator-password/version
exit

Verify equality without displaying the password:

sudo cmp --silent /run/secrets/postgres/bootstrap-password/value /run/secrets/postgres/administrator-password/value && echo 'PostgreSQL password projections match'

9. Browser and MinisForum: project the ORCID client Secret

Read the public client ID and callback from the candidate:

jq -r '.oidc.client_id, ("https://" + .runtime_dns_name + "/v1/runtime-login/callback")' "$CANDIDATE_ROOT/site/site-inputs.json"

In a browser, sign in to ORCID Sandbox Developer Tools. Open the application with that exact public client ID, confirm that the displayed callback URI is identical, and copy its client Secret.

On the MinisForum, create the protected directory:

sudo install -d -o root -g ahri-tre-oidc -m 0750 /run/secrets/oidc/client-secret

Start a temporary root shell:

sudo bash

Read the Secret without terminal echo:

IFS= read -r -s -p 'Paste the ORCID Sandbox client secret, then press Enter: ' orcid_client_secret

Then run:

printf '\n'
test -n "$orcid_client_secret"
umask 027
printf '%s' "$orcid_client_secret" > /run/secrets/oidc/client-secret/value
unset orcid_client_secret
printf '%s' 'site-v3' > /run/secrets/oidc/client-secret/version
chown root:ahri-tre-oidc /run/secrets/oidc/client-secret/value /run/secrets/oidc/client-secret/version
chmod 0440 /run/secrets/oidc/client-secret/value /run/secrets/oidc/client-secret/version
exit

Verify only ownership and modes:

sudo stat -c '%U:%G %a %n' /run/secrets/oidc/client-secret/value /run/secrets/oidc/client-secret/version

Both lines must begin root:ahri-tre-oidc 440.

10. WSL2 and MinisForum: project the Runtime private key

In WSL2, make a temporary protected transfer copy:

sudo install -o kobus -g kobus -m 0400 /home/kobus/ahri-tre-pki/private/runtime-private-key.pem /tmp/runtime-private-key.transfer
scp /tmp/runtime-private-key.transfer sysadmin@192.168.31.75:runtime-private-key.transfer
rm -- /tmp/runtime-private-key.transfer

On the MinisForum, project it and remove the transfer copy:

sudo install -d -o ahri-tre-runtime -g ahri-tre-runtime -m 0700 /run/secrets/runtime/private-key
sudo install -o ahri-tre-runtime -g ahri-tre-runtime -m 0400 /home/sysadmin/runtime-private-key.transfer /run/secrets/runtime/private-key/value
printf '%s' 'site-v3' | sudo tee /run/secrets/runtime/private-key/version >/dev/null
sudo chown ahri-tre-runtime:ahri-tre-runtime /run/secrets/runtime/private-key/version
sudo chmod 0400 /run/secrets/runtime/private-key/version
rm -- /home/sysadmin/runtime-private-key.transfer

Validate the projected key without printing it:

sudo -u ahri-tre-runtime env OPENSSL_CONF=/dev/null openssl pkey -in /run/secrets/runtime/private-key/value -check -noout

11. MinisForum and WSL2: create a new Deployment root identity

Never restore or reuse the identity from a deleted installation. On the MinisForum, generate a new X25519 identity:

sudo install -d -o root -g root -m 0700 /root/ahri-tre-recovery
sudo test ! -e /root/ahri-tre-recovery/conformance-root-identity.txt

sudo bash -c 'set -eu; umask 077; age-keygen | sed -n "/^AGE-SECRET-KEY-1/p" > /root/ahri-tre-recovery/conformance-root-identity.txt; test -s /root/ahri-tre-recovery/conformance-root-identity.txt'

Project it for the Runtime:

sudo install -d -o ahri-tre-runtime -g ahri-tre-runtime -m 0700 /run/secrets/ahri-tre/root-identity
sudo install -o ahri-tre-runtime -g ahri-tre-runtime -m 0400 /root/ahri-tre-recovery/conformance-root-identity.txt /run/secrets/ahri-tre/root-identity/value
printf '%s' 'site-v3' | sudo tee /run/secrets/ahri-tre/root-identity/version >/dev/null
sudo chown ahri-tre-runtime:ahri-tre-runtime /run/secrets/ahri-tre/root-identity/version
sudo chmod 0400 /run/secrets/ahri-tre/root-identity/version

Read the public Deployment UUID and create a temporary transfer copy:

DEPLOYMENT_ID="$(jq -r '.deployment_id' "$CANDIDATE_ROOT/site/site-inputs.json")"
sudo install -o sysadmin -g sysadmin -m 0400 /root/ahri-tre-recovery/conformance-root-identity.txt /home/sysadmin/conformance-root-identity.transfer
printf 'Deployment identity: %s\n' "$DEPLOYMENT_ID"

In WSL2, set the same public Deployment UUID from the reviewed candidate:

DEPLOYMENT_ID="$(jq -r '.deployment_id' /home/kobus/ahri-tre-conformance/input/review/ahri-tre-test-datastore-deployment-kit-0.3.14/site/site-inputs.json)"
RECOVERY_ROOT="/home/kobus/ahri-tre-recovery/conformance-${DEPLOYMENT_ID}"
install -d -m 0700 "$RECOVERY_ROOT"
test ! -e "$RECOVERY_ROOT/root-identity.txt"

If the last command fails, stop. Resolve the previous backup explicitly; do not overwrite it or silently reuse it. Retrieve and verify the new identity:

scp sysadmin@192.168.31.75:/home/sysadmin/conformance-root-identity.transfer "$RECOVERY_ROOT/root-identity.txt"
chmod 0400 "$RECOVERY_ROOT/root-identity.txt"
grep -q '^AGE-SECRET-KEY-1' "$RECOVERY_ROOT/root-identity.txt" && echo 'Separate root-identity backup is valid'

Back on the MinisForum, remove the temporary transfer copy:

rm -- /home/sysadmin/conformance-root-identity.transfer

Keep this identity separate from the encrypted Datastore backup and from the conformance evidence bundle.

12. MinisForum: verify the prepared boundary

Display only ownership and modes:

sudo stat -c '%U(%u):%G(%g) %a %n' /run/secrets /run/secrets/ahri-tre /run/secrets/oidc /run/secrets/postgres /run/secrets/postgres/tls /run/secrets/runtime /run/secrets/postgres/bootstrap-password/value /run/secrets/postgres/administrator-password/value /run/secrets/postgres/administrator-password/version /run/secrets/postgres/administrator-passfile/value /run/secrets/postgres/tls/private-key/value /run/secrets/oidc/client-secret/value /run/secrets/oidc/client-secret/version /run/secrets/runtime/private-key/value /run/secrets/runtime/private-key/version /run/secrets/ahri-tre/root-identity/value /run/secrets/ahri-tre/root-identity/version

Required results are:

• bootstrap password: root:root 400;
• administrator password and version: ahri-tre-runtime:ahri-tre-runtime 400;
• administrator passfile: root:root 600;
• PostgreSQL private key: numeric 999:999 400;
• ORCID client Secret and version: root:ahri-tre-oidc 440;
• Runtime private key and version: ahri-tre-runtime:ahri-tre-runtime 400; and
• root identity and version: ahri-tre-runtime:ahri-tre-runtime 400.

The host may display unrelated names for UID or GID 999; the numeric values are authoritative. Never use cat, less, head, or an editor on a Secret value file.

Verify every declared candidate projection exists, is a regular file rather than a symbolic link, and has the declared numeric owner, group, and mode:

while IFS=$'\t' read -r path owner group mode; do
  sudo test -f "$path" || exit 1
  sudo test ! -L "$path" || exit 1
  test "$(sudo stat -c '%U' "$path")" = "$owner" ||
    test "$(sudo stat -c '%u' "$path")" = "$owner" || exit 1
  test "$(sudo stat -c '%G' "$path")" = "$group" ||
    test "$(sudo stat -c '%g' "$path")" = "$group" || exit 1
  test "$(sudo stat -c '%a' "$path")" = "${mode#0}" || exit 1
done < <(sudo jq -r '.projections[] | [.projection_path, .owner, .group, .mode] | @tsv' "$CANDIDATE_ROOT/site/secret-projection-plan.json")

Reconfirm the clean-host installation boundary:

sudo test ! -e /usr/share/ahri-tre/server/component-versions.json
sudo test ! -e /usr/libexec/ahri-tre/ahri-tre-runtime
sudo test ! -e /var/lib/ahri-tre/secrets
sudo test ! -e /data/ahri-tre/postgresql
sudo docker inspect ahri-tre-postgresql >/dev/null 2>&1; test $? -ne 0

Preparation is complete only when every check exits zero. Do not reboot: the temporary inputs beneath /run/secrets are intentionally ephemeral.

13. MinisForum: hand off to the conformance harness

Create an evidence directory that contains no other files:

sudo install -d -o root -g root -m 0700 /var/backups/ahri-tre/conformance-evidence
sudo test -z "$(sudo find /var/backups/ahri-tre/conformance-evidence -mindepth 1 -maxdepth 1 -print -quit)"

Set the public Deployment confirmation and replace the safe identity placeholder with the canonical admitted ORCID Sandbox iD that will be used throughout the qualification:

DEPLOYMENT_ID="$(jq -r '.deployment_id' "$CANDIDATE_ROOT/site/site-inputs.json")"
SAFE_IDENTITY=REPLACE_WITH_ADMITTED_ORCID_ID

Invoke the harness from the extracted candidate while passing the original archives and their exact checksum declarations:

sudo "$CANDIDATE_ROOT/conformance/run.sh" server \
  --kit-archive "$INPUT_ROOT/ahri-tre-test-datastore-deployment-kit-0.3.14.tar.gz" \
  --kit-sha256 "$INPUT_ROOT/ahri-tre-test-datastore-deployment-kit-0.3.14.tar.gz.sha256" \
  --predecessor-archive "$INPUT_ROOT/ahri-tre-test-datastore-deployment-kit-0.3.12.tar.gz" \
  --predecessor-sha256 "$INPUT_ROOT/ahri-tre-test-datastore-deployment-kit-0.3.12.tar.gz.sha256" \
  --evidence-dir /var/backups/ahri-tre/conformance-evidence \
  --confirm-clean-host svrltreapcc02 \
  --confirm-host svrltreapcc02 \
  --confirm-address 192.168.31.75 \
  --confirm-deployment "$DEPLOYMENT_ID" \
  --confirm-datastore ahri-tre-test \
  --safe-identity "$SAFE_IDENTITY" \
  --backup-name conformance-0.3.14

The command must print Linux server installed-package evidence outcome=go. From this point onward, use the same candidate archive bytes and checksum for every workstation, recovery, cleanup, and finalization phase. Never copy a Secret projection or root identity into conformance evidence.

Continue with Running Installed-Package Conformance, which gives the complete backup, restore, WSL2 recovery, rollback, macOS, cleanup, and finalization sequence without requiring an issue or ticket.

Resetting a Failed Installed-Conformance Attempt

This procedure resets the known Linux server no-go caused by a stale 0.3.14 server lifecycle publication. It preserves the failed candidate and sanitized evidence, removes the partial AHRI-TRE installation, activates the corrected candidate, and leaves Secret preparation to the existing preparation wizard. Neither wizard installs package files or invokes the conformance harness.

Use this procedure only for this exact failed attempt:

• failed candidate SHA-256: b39afb3b5457258e69dd571bb1514543c1a5f3af09303930ce7a1a2834041f1e;
• corrected candidate SHA-256: 2caf994b9cf3d72d4eacd4a6fdeb859440481d2a6fb4cd3f02a3437a84e6a1cd;
• fixed 0.3.12 predecessor SHA-256: 6f5555c25d96274d7772d1b46d4409bd36a42205f21c4c715de8df5179ab18ed;
• MinisForum host svrltreapcc02 at 192.168.31.75; and
• Deployment f2ef37c5-7430-468a-a439-b3ba1b0527c1, Datastore ahri-tre-test.

The failed and corrected archives have the same filename and kit version. The SHA-256 value, not the filename, identifies their immutable contents.

What is kept and what is reset

The incomplete predecessor directory is moved into the protected failed-attempt record rather than deleted. The failed public archive, checksum, and extracted public packages are also retired. Secret values are not copied into evidence.

Prerequisites

Run from the repository root in the ordinary WSL2 Ubuntu shell, not inside the development container. The repository must contain the executable reset and preparation wizards.

The corrected candidate pair must exist at:

dist/conformance-candidate-retry-output-0.3.14/
├── ahri-tre-test-datastore-deployment-kit-0.3.14.tar.gz
└── ahri-tre-test-datastore-deployment-kit-0.3.14.tar.gz.sha256

Verify it before connecting to the server:

cd /home/kobus/repos/ahri-tre-rs

cd dist/conformance-candidate-retry-output-0.3.14
sha256sum --check --strict \
  ahri-tre-test-datastore-deployment-kit-0.3.14.tar.gz.sha256
cd /home/kobus/repos/ahri-tre-rs

The result must be OK, and the archive digest must be the corrected digest shown above. Stop if the corrected pair is missing or differs.

The MinisForum must be reachable as sysadmin@192.168.31.75. Password prompts from ssh and remote sudo belong to sysadmin on the MinisForum. Local sudo prompts belong to the WSL2 account.

Step 1: preserve and reset the known no-go

Start the dedicated wizard:

cd /home/kobus/repos/ahri-tre-rs
./scripts/reset-failed-installed-conformance-wizard.sh

The wizard performs seven guarded stages:

1. It verifies WSL2, the required tools, and the fixed corrected archive and checksum.
2. It extracts the corrected archive into a protected temporary directory, verifies every internal checksum, checks the exact current server lifecycle, and confirms that the embedded harness and lifecycle share committed source.
3. It obtains the sanitized linux-server.json, verifies outcome=no-go and failed_stage=server.upgrade, and preserves it locally before cleanup.
4. After an explicit confirmation, it verifies the installed uninstaller against the failed archive, runs it, retires the incomplete predecessor and failed public inputs, and removes only the generated AHRI-TRE attempt state.
5. It moves the failed local archive and checksum into the protected retirement directory, activates the corrected pair, and destroys the failed root identity backup so that it cannot be reused.
6. It proves the host is again a clean foundation and displays UFW state for human review.
7. It stops at RESET READY and names the Secret-preparation command. It does not continue into conformance.

At the firewall prompt, approve only this retained policy:

• active UFW;
• default deny incoming;
• default allow outgoing;
• SSH allowed;
• TCP 443 allowed to 192.168.31.75 from 192.168.31.0/24; and
• TCP 5432 denied.

The failed local record is stored under:

/home/kobus/ahri-tre-conformance/retired/no-go-b39afb3b5457258e69dd571bb1514543c1a5f3af09303930ce7a1a2834041f1e/

The protected server-side record is stored under:

/var/backups/ahri-tre/failed-installed-conformance/b39afb3b5457258e69dd571bb1514543c1a5f3af09303930ce7a1a2834041f1e/

If the wizard is interrupted, run the same command again. Its guards recognize already-retired evidence, public inputs, and candidate files. Do not manually invent package-SHA256SUMS, delete evidence, or copy the corrected archive over an unverified active archive.

Step 2: create entirely fresh candidate-matching Secrets

After the reset wizard reports RESET READY, run:

cd /home/kobus/repos/ahri-tre-rs
./scripts/minisforum-conformance-preparation-wizard.sh

The preparation wizard re-verifies the corrected active archive and the fixed predecessor; transfers those exact public inputs; creates fresh PostgreSQL TLS, password, Runtime, OIDC, and Deployment root-identity projections; writes a new protected WSL2 root-identity backup; and proves the resulting permissions and candidate bindings.

The corrected candidate is already active at:

/home/kobus/ahri-tre-conformance/input/ahri-tre-test-datastore-deployment-kit-0.3.14.tar.gz
/home/kobus/ahri-tre-conformance/input/ahri-tre-test-datastore-deployment-kit-0.3.14.tar.gz.sha256

The fixed predecessor remains at the same input location. Do not substitute a different predecessor or regenerate it.

Enter the ORCID client Secret only when the preparation wizard requests it. The value is read without echo and transferred directly to a protected remote file; it is not stored in the repository or local .env file.

Required stopping boundary

Stop when the preparation wizard prints:

HOST READY
The conformance harness has not been invoked.
Press Enter to finish at the host-ready boundary.

Do not install package files by hand, do not run an individual package installer, and do not invoke the conformance harness as part of this reset and preparation procedure. The verified /run/secrets projections are ephemeral; do not reboot after HOST READY and before the later conformance run.

Resetting the Secret-Projection Conformance No-Go

This procedure recovers the exact Linux server conformance attempt that stopped at site.provision because /run/secrets/ahri-tre had the wrong owner and mode. It preserves the sanitized no-go evidence and the complete checksummed predecessor rollback unit, removes only the partial AHRI TRE attempt, activates the corrected candidate, and then prepares entirely fresh external Secrets.

Neither wizard installs package files or invokes the conformance harness. Stop after the preparation wizard declares HOST READY.

Use this procedure only for these immutable inputs:

• failed candidate SHA-256: 2caf994b9cf3d72d4eacd4a6fdeb859440481d2a6fb4cd3f02a3437a84e6a1cd;
• corrected candidate SHA-256: 21a99920541e044000bba7134f685f8780ba1de44ab9637408d8ffd5c470b125;
• fixed 0.3.12 predecessor SHA-256: 6f5555c25d96274d7772d1b46d4409bd36a42205f21c4c715de8df5179ab18ed;
• MinisForum svrltreapcc02 at 192.168.31.75;
• Deployment f2ef37c5-7430-468a-a439-b3ba1b0527c1; and
• Datastore ahri-tre-test.

The failed and corrected archives deliberately have the same filename and kit version. Their SHA-256 values identify their different immutable contents.

What remains and what is reset

The complete /var/backups/ahri-tre/server-previous rollback unit is verified with its own package-SHA256SUMS, then moved into this attempt’s protected record. It is not reused for the next run.

1. Verify the corrected candidate in WSL2

Run from the ordinary WSL2 Ubuntu shell, not the development container:

cd /home/kobus/repos/ahri-tre-rs
cd dist/conformance-candidate-projection-retry-output-0.3.14
sha256sum --check --strict \
  ahri-tre-test-datastore-deployment-kit-0.3.14.tar.gz.sha256
cd /home/kobus/repos/ahri-tre-rs

The check must print OK. The archive digest must be the corrected digest shown above. Do not copy it over the active failed archive by hand.

The fixed predecessor archive and checksum must remain in:

/home/kobus/ahri-tre-conformance/input/

Do not regenerate or replace the predecessor.

2. Run the exact-state reset wizard

Keep the MinisForum running in its failed state. Do not reboot or manually change /run/secrets first. Start:

cd /home/kobus/repos/ahri-tre-rs
./scripts/reset-failed-secret-projection-conformance-wizard.sh

The wizard performs seven guarded stages:

1. It verifies WSL2, required tools, and the exact corrected archive pair.
2. It verifies all internal checksums, committed-source server lifecycle, the corrected projector, and the corrected root-owned namespace declaration.
3. It accepts only sanitized Linux evidence with outcome=no-go and failed_stage=site.provision, then preserves it locally.
4. It verifies the host, failed candidate digest, installed package and partial site ownership, installed projector, observed 0700 namespace, captured authority, and complete predecessor rollback unit. After confirmation, it verifies the installed uninstall plan, stops the units present at this partial-install boundary, removes only regular files declared by the server and site ownership manifests, and clears generated attempt state.
5. It retires the exact failed local pair, activates the corrected pair, and removes the failed root-identity backup.
6. It proves the retained host foundation and displays UFW for human review.
7. It stops at RESET READY without installing packages or invoking conformance.

Approve the firewall prompt only when UFW remains active with default-deny incoming, default-allow outgoing, SSH allowed, LAN HTTPS on TCP 443 allowed, and PostgreSQL TCP 5432 denied.

The local failed record is retained under:

/home/kobus/ahri-tre-conformance/retired/no-go-2caf994b9cf3d72d4eacd4a6fdeb859440481d2a6fb4cd3f02a3437a84e6a1cd/

The protected server record is retained under:

/var/backups/ahri-tre/failed-installed-conformance/2caf994b9cf3d72d4eacd4a6fdeb859440481d2a6fb4cd3f02a3437a84e6a1cd/

If the wizard refuses, stop. Do not delete evidence, manufacture a checksum manifest, manually invoke an installer, or alter ownership to bypass a guard. An interrupted wizard can be run again; it recognizes its exact retired state.

3. Prepare fresh candidate-matching Secrets

After the reset wizard prints RESET READY, run:

cd /home/kobus/repos/ahri-tre-rs
./scripts/minisforum-conformance-preparation-wizard.sh

The preparation wizard re-verifies the corrected candidate and fixed predecessor, transfers those exact inputs, creates fresh PostgreSQL TLS and password projections, retrieves the ORCID Sandbox client Secret, projects the candidate Runtime key, creates and separately backs up a fresh Deployment root identity, and proves the shared namespace and leaf permissions.

The shared projection root must be root:root:0755. These traversal-only namespaces must each be root:root:0711:

/run/secrets/ahri-tre
/run/secrets/oidc
/run/secrets/postgres
/run/secrets/postgres/tls
/run/secrets/runtime

The restricted leaf directories and files retain the owners and modes declared by the candidate’s projection plan. Mode 0711 on a shared namespace does not make Secret values readable.

Required stopping boundary

Stop when the preparation wizard prints:

HOST READY
The conformance harness has not been invoked.
Press Enter to finish at the host-ready boundary.

Do not reboot after HOST READY, because /run/secrets is intentionally ephemeral. Do not install packages or invoke conformance in this reset and preparation phase.

Resetting the Recovery-Backup Conformance No-Go

This procedure recovers the Linux installed-conformance attempt that passed the server phase and then stopped at recovery backup.verify. The failed adapter changed /var/lib/ahri-tre from root:ahri-tre:0750 to 0700 while recording its candidate binding. That prevented ahri-tre-runtime from traversing the directory, so Managed-secret verification reported NotInitialized.

Do not repair that installation with chmod and continue qualifying it. It is a valid no-go result for immutable candidate bytes. Use the guarded wizard to preserve the result, remove the disposable installation, activate the corrected candidate, and then prepare fresh Secrets. Neither wizard installs packages or invokes conformance.

Exact scope

Use this procedure only with:

• failed candidate SHA-256 21a99920541e044000bba7134f685f8780ba1de44ab9637408d8ffd5c470b125;
• corrected candidate SHA-256 de2400a3d050b71a6d97165b1785df2c512d830b063d261d7ec43dc05d4e9721;
• corrected source revision f3d902775477564edd1fcc873e0e2e60565d2cb3;
• fixed 0.3.12 predecessor SHA-256 6f5555c25d96274d7772d1b46d4409bd36a42205f21c4c715de8df5179ab18ed;
• host svrltreapcc02 at 192.168.31.75;
• Deployment f2ef37c5-7430-468a-a439-b3ba1b0527c1; and
• Datastore ahri-tre-test.

The candidate filename remains ahri-tre-test-datastore-deployment-kit-0.3.14.tar.gz; the checksum identifies which immutable candidate it contains.

What remains and what is removed

The wizard retains the hostname, reserved address, /data mount, Docker installation and default bridge, hosts mappings, intended UFW policy, and declared service identities. It preserves the successful Linux-server record, the recovery no-go record, the candidate binding, and the checksummed predecessor rollback unit under the failed candidate’s protected record.

It removes the failed AHRI TRE package files, managed PostgreSQL container, disposable PostgreSQL data, Lake and scratch directories, configuration, Managed-secret store, Injected-secret authority, ephemeral projections, Runtime state, and the failed attempt’s separate root-identity backup. No previous AHRI TRE installation is reused.

1. Verify the corrected candidate

In the ordinary WSL2 Ubuntu shell:

cd /home/kobus/repos/ahri-tre-rs/dist/conformance-candidate-recovery-permissions-output-0.3.14
sha256sum --check --strict \
  ahri-tre-test-datastore-deployment-kit-0.3.14.tar.gz.sha256

The result must be OK and the checksum file must contain exactly:

de2400a3d050b71a6d97165b1785df2c512d830b063d261d7ec43dc05d4e9721  ahri-tre-test-datastore-deployment-kit-0.3.14.tar.gz

Keep the fixed predecessor archive and checksum in /home/kobus/ahri-tre-conformance/input. Do not regenerate it.

2. Run the guarded reset

Leave the MinisForum in the failed state. Do not reboot, change permissions, delete the container, or edit evidence first. Run:

cd /home/kobus/repos/ahri-tre-rs
./scripts/reset-failed-recovery-backup-conformance-wizard.sh

The five stages:

1. verify WSL2 and the exact replacement inputs;
2. verify all candidate checksums, source revision, and embedded fix;
3. accept only the exact host, failed archive, successful server evidence, backup.verify no-go, 0700 failed state, candidate binding, managed PostgreSQL container, empty recovery output, and checksummed predecessor; then preserve evidence and remove only the verified failed installation;
4. retire the failed local pair, activate the corrected pair, and remove the failed root-identity backup; and
5. prove the retained clean foundation, display UFW for review, and stop at RESET READY.

Approve the firewall prompt only if UFW is active with default-deny incoming, default-allow outgoing, SSH allowed, LAN HTTPS on TCP 443 allowed, and PostgreSQL TCP 5432 denied.

If any guard refuses, stop. Do not bypass it by changing state manually.

3. Prepare fresh candidate-matching Secrets

After RESET READY, run:

cd /home/kobus/repos/ahri-tre-rs
./scripts/minisforum-conformance-preparation-wizard.sh

The preparation wizard must verify and transfer the corrected archive and checksum above plus the fixed predecessor inputs. It generates a fresh Deployment root identity and fresh candidate-matching Secret projections.

Stop when it prints:

HOST READY
The conformance harness has not been invoked.
Press Enter to finish at the host-ready boundary.

Do not reboot after HOST READY; the verified /run/secrets projections are ephemeral. Do not install packages or invoke conformance until the later HITL step explicitly begins.

Resetting Failed Recovery Qualification

Runtime-restart failure after a successful restore

Use this procedure when candidate 7c95cf54622c7e3a63d35b14db76b5e1d3dcf66c8c3af8518e3a2e9905942867 passed the Linux server, recovery backup, and recovery restore phases, then its fresh WSL2 recovery qualification failed at runtime.process_restart with:

Runtime client credential is unavailable or invalid

The v0.10.7 Managed runtime retired its still-valid bounded credential during ordinary daemon shutdown. The installed qualifier correctly stopped and restarted the daemon without logging out, but the restarted process could no longer use the same Runtime login. Release v0.10.8 preserves that credential generation across process restart and makes daemon stop wait for the old process, not only disappearance of its socket. Explicit logout, reboot, and uninstall semantics are unchanged.

Do not rerun the v0.10.7 WSL2 package and do not manually alter the restored server. The failed candidate cannot produce valid workstation or recovery evidence.

The dedicated reset accepts only the exact v0.10.7 pending-wsl2 restore:

• candidate SHA-256 7c95cf54622c7e3a63d35b14db76b5e1d3dcf66c8c3af8518e3a2e9905942867;
• successful linux-server.json and recovery-backup.json, with no recovery acceptance record;
• the matching candidate binding and recovery-pending.json for restore;
• the checksummed recovery backup and fixed 0.3.12 predecessor;
• the cryptographically verified post-qualification Managed-secret store with its exact Deployment, recipient fingerprint, nine active entries, ten audit records, and checksum-bound safe metadata inventory; this intentionally differs from the pre-qualification backup after two failed-login cleanup histories were appended;
• the installed v0.10.7 server, healthy v0.10.7 PostgreSQL container, active PostgreSQL and Trusted-runtime services, and declared filesystem traversal;
• the immutable ready Datastore identity def55da2-0cf5-4fdb-8934-ec517ccaf78a, with exactly 16 sequences and 45 tables owned by its derived PostgreSQL role tre_store_def55da20cf54fdb8934ec517ccaf78a;
• host svrltreapcc02 at 192.168.31.75, Deployment f2ef37c5-7430-468a-a439-b3ba1b0527c1, and Datastore ahri-tre-test.

It preserves the exact failed boundary beneath its candidate digest, removes only the disposable AHRI TRE installation and attempt-specific recovery state, and retains the hostname, reserved address, /data mount, Docker foundation, hosts mappings, firewall, and service identities. A mismatch is refused before mutation.

The exact replacement is candidate SHA-256 bdd0fe785ed333caf1f9c7de8b70479d5c37b5526d7ace6e28de74105c2cab48 at candidate source revision 8967fa05ec30eb13f0291f7c7a2714649a0db302. Its release components are v0.10.8 source revision a292eab3d21a02060b115927da3266f62ad982a0.

Verify it from the repository shell on the recorded WSL2 controller or an explicitly authorized Apple Silicon macOS replacement controller:

cd dist/conformance-candidate-runtime-restart-output-0.3.14
sha256sum --check --strict \
  ahri-tre-test-datastore-deployment-kit-0.3.14.tar.gz.sha256
cd ../..

Then return to the repository root and run:

./scripts/reset-failed-wsl2-recovery-conformance-wizard.sh

On macOS, the wizard first recovers the exact failed public candidate and fixed predecessor from the accepted server boundary when they are not already local. Before the destructive reset, it copies only the Runtime private leaf key and PostgreSQL server leaf key/certificate to protected paths below ~/ahri-tre-pki. It proves that the Runtime certificate and PostgreSQL public authority are unchanged between the failed and replacement candidates, then proves that both recovered keys match their replacement certificates. A mismatch stops before the server reset. These controller-recovery inputs are never placed in the repository, release archive, or evidence directory.

Approve the firewall prompt only when UFW is active with default-deny incoming, default-allow outgoing, SSH allowed, LAN HTTPS on TCP 443 allowed, and PostgreSQL TCP 5432 denied. Stop at RESET READY. Then run:

./scripts/minisforum-conformance-preparation-wizard.sh

The preparation wizard must verify the replacement checksum and stop at HOST READY. Do not reboot because the projected Secrets are ephemeral. Begin the conformance sequence again from the Linux server phase; evidence from the failed candidate is not reusable.

PostgreSQL-ownership failure during restore

Use this procedure only when installed conformance passed the Linux server and recovery-backup phases, then recovery-restore failed after committing the database because its metadata objects had the wrong owner:

ahri_tre_administrator|S|16
ahri_tre_administrator|r|45

Candidate c6e84103ca262606740d82407a79a15d63184a38013caabb86854b371623eec9 included a redundant --no-owner on its custom-format dump and actively suppressed ownership restoration with pg_restore --no-owner. The restore therefore created all 61 non-system relations—16 sequences and 45 tables— under the connecting ahri_tre_administrator role instead of retaining the immutable Datastore owner role. Installed readiness reported that the Datastore binding was not ready. Do not alter owners manually or resume recovery with that candidate. The candidate did not produce a valid restore result.

The guarded reset retires the exact failed attempt, removes its disposable installation, activates the replacement candidate, and stops before Secret preparation. It never installs a package or invokes the conformance harness.

Exact scope

The reset accepts only:

• failed candidate SHA-256 c6e84103ca262606740d82407a79a15d63184a38013caabb86854b371623eec9;
• replacement candidate SHA-256 7c95cf54622c7e3a63d35b14db76b5e1d3dcf66c8c3af8518e3a2e9905942867;
• replacement source revision 1fc09bb67cdb815832ab1fb356f4a49ebc40e63c;
• fixed 0.3.12 predecessor SHA-256 6f5555c25d96274d7772d1b46d4409bd36a42205f21c4c715de8df5179ab18ed;
• host svrltreapcc02 at 192.168.31.75;
• Deployment f2ef37c5-7430-468a-a439-b3ba1b0527c1; and
• Datastore ahri-tre-test.

The filename remains ahri-tre-test-datastore-deployment-kit-0.3.14.tar.gz; its external checksum identifies the immutable candidate bytes.

The host-side guard requires the exact successful linux-server.json and recovery-backup.json records, the recovery.json no-go at restore.verify, the checksummed predecessor and backup, the restored Managed-secret and Lake contents, no recovery-qualification state, and the running managed PostgreSQL container with its durable deployment-contract bind and committed database. It also requires root:ahri-tre:0750 on /var/lib/ahri-tre, the exact declared root:ahri-tre:0750 configuration parent, root:root:0755 on /data/ahri-tre, runtime-owned 0700 Lake and scratch leaves, an active PostgreSQL service, inactive Runtime and Web services, and exactly 16 sequences plus 45 tables owned by ahri_tre_administrator. The number 142 previously reported during diagnosis was the administrator role’s PostgreSQL object ID, not an object count. A different state is refused without mutation.

What the reset preserves

The wizard preserves the three conformance records, verified recovery backup, candidate binding, and checksummed predecessor beneath the failed candidate digest. It retains the hostname, reserved address, /data mount, Docker foundation and bridge, hosts mappings, firewall configuration, and declared service identities.

It removes the active AHRI TRE package files, managed PostgreSQL container, disposable PostgreSQL data, Lake and scratch state, configuration, Managed Secrets, Injected-secret projections, Runtime state, and the attempt’s separate root-identity copy. None of the retired installation is reused.

1. Verify the replacement archive

In the ordinary WSL2 Ubuntu shell:

cd /home/kobus/repos/ahri-tre-rs/dist/conformance-candidate-postgresql-ownership-output-0.3.14
sha256sum --check --strict \
  ahri-tre-test-datastore-deployment-kit-0.3.14.tar.gz.sha256

The result must be OK, and the checksum file must contain exactly:

7c95cf54622c7e3a63d35b14db76b5e1d3dcf66c8c3af8518e3a2e9905942867  ahri-tre-test-datastore-deployment-kit-0.3.14.tar.gz

Keep the fixed predecessor archive and checksum in /home/kobus/ahri-tre-conformance/input.

2. Run the guarded reset

Leave the MinisForum unchanged and run from the repository root:

./scripts/reset-incomplete-recovery-conformance-wizard.sh

The wizard:

1. verifies WSL2 and the exact replacement inputs;
2. verifies the replacement checksums, source revision, recovery workflow, durable PostgreSQL deployment-contract bind, PostgreSQL ownership preservation, and shared data-parent preservation;
3. accepts only the exact failed-restore state, verifies the backup and predecessor checksums, retires that state, and removes the bounded active installation;
4. retires the local failed candidate pair and activates the replacement pair;
5. proves the clean retained foundation, displays UFW, and stops at RESET READY.

Approve the firewall prompt only when UFW is active with default-deny incoming, default-allow outgoing, SSH allowed, LAN HTTPS on TCP 443 allowed, and PostgreSQL TCP 5432 denied. If any guard refuses, stop without modifying the state manually.

3. Prepare fresh Secrets

After RESET READY, run:

./scripts/minisforum-conformance-preparation-wizard.sh

The preparation wizard must use replacement SHA-256 7c95cf54622c7e3a63d35b14db76b5e1d3dcf66c8c3af8518e3a2e9905942867 and the fixed predecessor SHA-256 above. Stop when it declares HOST READY. Do not reboot because the verified /run/secrets projections are ephemeral.

Running Installed-Package Conformance

This runbook starts after Secret preparation declares HOST READY. It covers the Linux server, backup, restore, rollback, fresh installed-client recovery checks, final WSL2 and macOS checks, cleanup, and finalization. Package files are installed only by the conformance harness.

Use this exact candidate for every phase:

archive: ahri-tre-test-datastore-deployment-kit-0.3.23.tar.gz
SHA-256: 8c4664cd7fb48c040869c09172cf15cba2bb324c2958b0d97407c424ce20fcea

The fixed predecessor is:

archive: ahri-tre-test-datastore-deployment-kit-0.3.12.tar.gz
SHA-256: 6f5555c25d96274d7772d1b46d4409bd36a42205f21c4c715de8df5179ab18ed

The admitted ORCID Sandbox iD is 0009-0005-0445-6675. The intentionally unadmitted iD is 0009-0007-0768-5937. Never admit the second iD.

Do not reboot until recovery is complete. The root identity and other verified inputs beneath /run/secrets are ephemeral.

The accepted v0.3.22 recovery followed by its dataset.metadata.assertion no-go is retained evidence and must be retired before this runbook starts. None of the earlier reset wizards accepts that boundary. Issue 19 — Preserve numeric Dataset Variable order over OAuth owns its checksum-bound exact-state reset; do not substitute a broader manual cleanup.

After that reset stops at RESET READY, run ./scripts/minisforum-conformance-preparation-wizard.sh, stop at HOST READY, and then begin section 1.

1. Set the server paths

On the MinisForum as sysadmin:

INPUT_ROOT=/home/sysadmin/ahri-tre-conformance/input
CANDIDATE_ROOT="$INPUT_ROOT/extracted/ahri-tre-test-datastore-deployment-kit-0.3.23"
EVIDENCE_ROOT=/var/backups/ahri-tre/conformance-evidence
DEPLOYMENT_ID=f2ef37c5-7430-468a-a439-b3ba1b0527c1
SAFE_IDENTITY=0009-0005-0445-6675

Verify the exact inputs without splitting grep from its filename:

cd "$INPUT_ROOT"
grep -qxF '8c4664cd7fb48c040869c09172cf15cba2bb324c2958b0d97407c424ce20fcea  ahri-tre-test-datastore-deployment-kit-0.3.23.tar.gz' \
  ahri-tre-test-datastore-deployment-kit-0.3.23.tar.gz.sha256
sha256sum --check --strict \
  ahri-tre-test-datastore-deployment-kit-0.3.23.tar.gz.sha256
grep -qxF '6f5555c25d96274d7772d1b46d4409bd36a42205f21c4c715de8df5179ab18ed  ahri-tre-test-datastore-deployment-kit-0.3.12.tar.gz' \
  ahri-tre-test-datastore-deployment-kit-0.3.12.tar.gz.sha256
sha256sum --check --strict \
  ahri-tre-test-datastore-deployment-kit-0.3.12.tar.gz.sha256

Both checks must report OK.

2. Run the Linux server phase

sudo install -d -o root -g root -m 0700 "$EVIDENCE_ROOT"
sudo test -z "$(sudo find "$EVIDENCE_ROOT" -mindepth 1 -maxdepth 1 -print -quit)"

sudo "$CANDIDATE_ROOT/conformance/run.sh" server \
  --kit-archive "$INPUT_ROOT/ahri-tre-test-datastore-deployment-kit-0.3.23.tar.gz" \
  --kit-sha256 "$INPUT_ROOT/ahri-tre-test-datastore-deployment-kit-0.3.23.tar.gz.sha256" \
  --predecessor-archive "$INPUT_ROOT/ahri-tre-test-datastore-deployment-kit-0.3.12.tar.gz" \
  --predecessor-sha256 "$INPUT_ROOT/ahri-tre-test-datastore-deployment-kit-0.3.12.tar.gz.sha256" \
  --evidence-dir "$EVIDENCE_ROOT" \
  --confirm-clean-host svrltreapcc02 \
  --confirm-host svrltreapcc02 \
  --confirm-address 192.168.31.75 \
  --confirm-deployment "$DEPLOYMENT_ID" \
  --confirm-datastore ahri-tre-test \
  --safe-identity "$SAFE_IDENTITY" \
  --backup-name conformance-0.3.23

Positive output is Linux server installed-package evidence outcome=go. This phase admits SAFE_IDENTITY through the packaged site operation before any backup can be taken.

3. Back up the installed deployment

sudo "$CANDIDATE_ROOT/conformance/run.sh" recovery-backup \
  --kit-archive "$INPUT_ROOT/ahri-tre-test-datastore-deployment-kit-0.3.23.tar.gz" \
  --kit-sha256 "$INPUT_ROOT/ahri-tre-test-datastore-deployment-kit-0.3.23.tar.gz.sha256" \
  --evidence-dir "$EVIDENCE_ROOT" \
  --confirm-host svrltreapcc02 \
  --confirm-address 192.168.31.75 \
  --confirm-deployment "$DEPLOYMENT_ID" \
  --confirm-datastore ahri-tre-test \
  --safe-identity "$SAFE_IDENTITY" \
  --backup-name conformance-0.3.23

Positive output is Linux server conformance phase=recovery-backup outcome=go. Verify the backup:

sudo bash -c 'cd /var/backups/ahri-tre/recovery/conformance-0.3.23 && sha256sum --check --strict SHA256SUMS'

Select one genuine recovery client

Before section 4, select either macos or wsl2 for both fresh recovery qualifications. The examples below select native Apple Silicon macOS. The first restore and rollback invocation persists this choice with the exact target and candidate digest; every resume must repeat the same choice. Never rename or edit one platform’s output to represent the other.

If neither client is currently available, stop here. The installed server, exact candidate, checksummed recovery backup, and server evidence form the safe pre-recovery checkpoint. Do not empty the restore targets until the selected client can complete both fresh qualifications.

4. Prepare the exact empty restore targets

This is the destructive preparation step. It deletes only the current ahri-tre-test database objects, active Managed-secret ciphertext, and active Lake content. The verified backup remains under /var/backups/ahri-tre/recovery/conformance-0.3.23.

First verify the exact directories and stop writers:

sudo test "$(sudo stat -c '%U:%G:%a' /var/lib/ahri-tre/secrets)" = 'ahri-tre-runtime:ahri-tre-runtime:700'
sudo test "$(sudo stat -c '%U:%G:%a' /data/ahri-tre/lake)" = 'ahri-tre-runtime:ahri-tre-runtime:700'
sudo test -f /run/secrets/ahri-tre/root-identity/value
sudo test ! -L /run/secrets/ahri-tre/root-identity/value
sudo systemctl stop ahri-tre-web.service ahri-tre-runtime.service

Render the packaged administrator connection and empty the public schema:

CONTRACT="$CANDIDATE_ROOT/site/postgresql/deployment-contract.json"
OPERATOR="$CANDIDATE_ROOT/site/postgresql/contract/operator.sh"
PSQL="$CANDIDATE_ROOT/site/postgresql/bin/psql"
DB_CONNINFO="$(sudo env PATH="$CANDIDATE_ROOT/site/postgresql/bin:/usr/bin:/bin" \
  "$OPERATOR" "$CONTRACT" render-conninfo ahri_tre_test)"
sudo "$PSQL" -X --no-psqlrc --set=ON_ERROR_STOP=1 \
  --dbname="$DB_CONNINFO" \
  --command='DROP SCHEMA public CASCADE; CREATE SCHEMA public AUTHORIZATION pg_database_owner;'

Empty the two exact filesystem targets and verify all three targets:

sudo find /var/lib/ahri-tre/secrets -mindepth 1 -maxdepth 1 -exec rm -rf -- {} +
sudo find /data/ahri-tre/lake -mindepth 1 -maxdepth 1 -exec rm -rf -- {} +
sudo test -z "$(sudo find /var/lib/ahri-tre/secrets -mindepth 1 -maxdepth 1 -print -quit)"
sudo test -z "$(sudo find /data/ahri-tre/lake -mindepth 1 -maxdepth 1 -print -quit)"
sudo test ! -e /var/backups/ahri-tre/restore-staging
sudo "$PSQL" -X --no-psqlrc --tuples-only --no-align \
  --dbname="$DB_CONNINFO" \
  --command="SELECT EXISTS (
    SELECT 1 FROM pg_class c JOIN pg_namespace n ON n.oid = c.relnamespace
    WHERE n.nspname NOT IN ('pg_catalog','information_schema')
      AND c.relkind IN ('r','p','v','m','S','f'));"

The last command must print f. If it does not, stop; do not broaden the SQL or filesystem deletion commands.

5. Restore and stop at the selected-client boundary

sudo "$CANDIDATE_ROOT/conformance/run.sh" recovery-restore \
  --kit-archive "$INPUT_ROOT/ahri-tre-test-datastore-deployment-kit-0.3.23.tar.gz" \
  --kit-sha256 "$INPUT_ROOT/ahri-tre-test-datastore-deployment-kit-0.3.23.tar.gz.sha256" \
  --evidence-dir "$EVIDENCE_ROOT" \
  --confirm-host svrltreapcc02 \
  --confirm-address 192.168.31.75 \
  --confirm-deployment "$DEPLOYMENT_ID" \
  --confirm-datastore ahri-tre-test \
  --safe-identity "$SAFE_IDENTITY" \
  --backup-name conformance-0.3.23 \
  --recovery-platform macos

Positive output is Linux server conformance phase=recovery-restore outcome=pending-macos. The restore has completed; acceptance is intentionally waiting for a fresh native macOS qualification. This exact site is the closed CLI-only deployment: its site contract disables Web ingress and its Application configuration does not select services.web. Leave ahri-tre-web.service stopped.

6. Prepare the exact candidate on native macOS

On native Apple Silicon macOS 26:

MAC_INPUT_ROOT="$HOME/ahri-tre-conformance/input"
MAC_CANDIDATE_PARENT="$HOME/ahri-tre-conformance/macos-candidate-0.3.23"
MAC_CANDIDATE_ROOT="$MAC_CANDIDATE_PARENT/ahri-tre-test-datastore-deployment-kit-0.3.23"
MAC_RECOVERY_OUTPUT="$HOME/ahri-tre-conformance/recovery-qualification"
DEPLOYMENT_ID=f2ef37c5-7430-468a-a439-b3ba1b0527c1

cd "$MAC_INPUT_ROOT"
grep -qxF '8c4664cd7fb48c040869c09172cf15cba2bb324c2958b0d97407c424ce20fcea  ahri-tre-test-datastore-deployment-kit-0.3.23.tar.gz' \
  ahri-tre-test-datastore-deployment-kit-0.3.23.tar.gz.sha256
shasum --algorithm 256 --check \
  ahri-tre-test-datastore-deployment-kit-0.3.23.tar.gz.sha256
test ! -e "$MAC_CANDIDATE_PARENT"
test ! -e "$MAC_RECOVERY_OUTPUT"
install -d -m 0700 "$MAC_CANDIDATE_PARENT" "$MAC_RECOVERY_OUTPUT"
tar --extract --gzip \
  --file ahri-tre-test-datastore-deployment-kit-0.3.23.tar.gz \
  --directory "$MAC_CANDIDATE_PARENT" --no-same-owner --no-same-permissions
(cd "$MAC_CANDIDATE_ROOT" && shasum --algorithm 256 --check SHA256SUMS)

The macOS client must be absent before each recovery qualification. The harness installs it, runs the installed qualifier, uninstalls it, then publishes only the validated redacted result.

Each qualification output is immutable. A completed invocation may return the shell prompt after writing its go result, so never rerun merely because the final console line was missed. Before reusing an existing restore or rollback output, verify its exact candidate binding:

verify_macos_recovery_output() {
  jq -e \
    --arg deployment "$DEPLOYMENT_ID" \
    --arg digest '8c4664cd7fb48c040869c09172cf15cba2bb324c2958b0d97407c424ce20fcea' '
    .ok == true and .outcome == "go" and
    .gate == "installed_remote_cli_capability.v1" and
    .platform == "macos" and .target == "aarch64-apple-darwin" and
    .kit_version == "0.3.23" and .client_version == "0.10.12" and
    .protocol_version == "1.0.0" and
    .source_revision == "1ae413547dae23b7fcbb659d064c704cce34cfa7" and
    .deployment_id == $deployment and .datastore == "ahri-tre-test" and
    .candidate_archive_sha256 == $digest and
    (.stages | length == 34 and all(.status == "pass"))
  ' "$1" >/dev/null
}

If --qualification-output must name an absent absolute path appears, do not delete the existing file. Run this verifier against it. A zero exit means the earlier invocation completed and the procedure resumes at the corresponding scp; any other result is preserved as a no-go and requires diagnosis.

For a genuine WSL2 recovery path, use the same sequence with --recovery-platform wsl2, the wsl2-recovery action, --confirm-clean-host wsl2, sha256sum --check --strict, and distinct restore-wsl2.json and rollback-wsl2.json paths beneath a WSL2-owned mode 0700 directory. Resume the server with --client-qualification naming the matching transferred file. Do not mix the two platform choices within a pending operation.

7. Qualify and accept the restore

On the Mac, sign out of other ORCID Sandbox browser sessions, then run:

RESTORE_QUALIFICATION="$MAC_RECOVERY_OUTPUT/restore-macos.json"
test ! -e "$RESTORE_QUALIFICATION"

If that test exits nonzero, do not execute the qualification command below; verify the existing file and resume at scp when it passes. If it exits zero, run:

"$MAC_CANDIDATE_ROOT/conformance/run.sh" macos-recovery \
  --kit-archive "$MAC_INPUT_ROOT/ahri-tre-test-datastore-deployment-kit-0.3.23.tar.gz" \
  --kit-sha256 "$MAC_INPUT_ROOT/ahri-tre-test-datastore-deployment-kit-0.3.23.tar.gz.sha256" \
  --qualification-output "$RESTORE_QUALIFICATION" \
  --user "$(id -un)" \
  --confirm-clean-host macos \
  --confirm-deployment "$DEPLOYMENT_ID" \
  --confirm-datastore ahri-tre-test \
  --admitted-identity 0009-0005-0445-6675

verify_macos_recovery_output "$RESTORE_QUALIFICATION"

Authenticate as 0009-0005-0445-6675. Positive output ends with macos recovery installed-package qualification outcome=go. Transfer that single file:

scp "$RESTORE_QUALIFICATION" \
  sysadmin@192.168.31.75:/home/sysadmin/ahri-tre-conformance/input/restore-macos.json

On the MinisForum, resume the same server phase:

sudo "$CANDIDATE_ROOT/conformance/run.sh" recovery-restore \
  --kit-archive "$INPUT_ROOT/ahri-tre-test-datastore-deployment-kit-0.3.23.tar.gz" \
  --kit-sha256 "$INPUT_ROOT/ahri-tre-test-datastore-deployment-kit-0.3.23.tar.gz.sha256" \
  --evidence-dir "$EVIDENCE_ROOT" \
  --confirm-host svrltreapcc02 \
  --confirm-address 192.168.31.75 \
  --confirm-deployment "$DEPLOYMENT_ID" \
  --confirm-datastore ahri-tre-test \
  --safe-identity "$SAFE_IDENTITY" \
  --backup-name conformance-0.3.23 \
  --recovery-platform macos \
  --client-qualification /home/sysadmin/ahri-tre-conformance/input/restore-macos.json

It must print Linux server conformance phase=recovery-restore outcome=go.

8. Roll back and stop at the selected-client boundary

On the MinisForum:

sudo "$CANDIDATE_ROOT/conformance/run.sh" recovery-rollback \
  --kit-archive "$INPUT_ROOT/ahri-tre-test-datastore-deployment-kit-0.3.23.tar.gz" \
  --kit-sha256 "$INPUT_ROOT/ahri-tre-test-datastore-deployment-kit-0.3.23.tar.gz.sha256" \
  --evidence-dir "$EVIDENCE_ROOT" \
  --confirm-host svrltreapcc02 \
  --confirm-address 192.168.31.75 \
  --confirm-deployment "$DEPLOYMENT_ID" \
  --confirm-datastore ahri-tre-test \
  --safe-identity "$SAFE_IDENTITY" \
  --backup-name conformance-0.3.23 \
  --recovery-platform macos

It must print Linux server conformance phase=recovery-rollback outcome=pending-macos.

9. Qualify and accept the rollback

On the Mac, run a fresh harness-owned qualification with a different absent output path:

ROLLBACK_QUALIFICATION="$MAC_RECOVERY_OUTPUT/rollback-macos.json"
test ! -e "$ROLLBACK_QUALIFICATION"

Again, a nonzero result means verify and reuse an exact existing go result; it does not authorize overwriting it. Only when the path is absent, run:

"$MAC_CANDIDATE_ROOT/conformance/run.sh" macos-recovery \
  --kit-archive "$MAC_INPUT_ROOT/ahri-tre-test-datastore-deployment-kit-0.3.23.tar.gz" \
  --kit-sha256 "$MAC_INPUT_ROOT/ahri-tre-test-datastore-deployment-kit-0.3.23.tar.gz.sha256" \
  --qualification-output "$ROLLBACK_QUALIFICATION" \
  --user "$(id -un)" \
  --confirm-clean-host macos \
  --confirm-deployment "$DEPLOYMENT_ID" \
  --confirm-datastore ahri-tre-test \
  --admitted-identity 0009-0005-0445-6675

verify_macos_recovery_output "$ROLLBACK_QUALIFICATION"

scp "$ROLLBACK_QUALIFICATION" \
  sysadmin@192.168.31.75:/home/sysadmin/ahri-tre-conformance/input/rollback-macos.json

On the MinisForum, accept the result, dispose only the accepted 0.3.12 rollback unit, and let the harness return the server to 0.3.23:

sudo "$CANDIDATE_ROOT/conformance/run.sh" recovery-rollback \
  --kit-archive "$INPUT_ROOT/ahri-tre-test-datastore-deployment-kit-0.3.23.tar.gz" \
  --kit-sha256 "$INPUT_ROOT/ahri-tre-test-datastore-deployment-kit-0.3.23.tar.gz.sha256" \
  --evidence-dir "$EVIDENCE_ROOT" \
  --confirm-host svrltreapcc02 \
  --confirm-address 192.168.31.75 \
  --confirm-deployment "$DEPLOYMENT_ID" \
  --confirm-datastore ahri-tre-test \
  --safe-identity "$SAFE_IDENTITY" \
  --backup-name conformance-0.3.23 \
  --recovery-platform macos \
  --client-qualification /home/sysadmin/ahri-tre-conformance/input/rollback-macos.json \
  --confirm-dispose-rollback 0.3.12

It must print Linux server conformance phase=recovery-rollback outcome=go.

10. Aggregate recovery evidence

On the MinisForum:

sudo "$CANDIDATE_ROOT/conformance/run.sh" recovery \
  --kit-archive "$INPUT_ROOT/ahri-tre-test-datastore-deployment-kit-0.3.23.tar.gz" \
  --kit-sha256 "$INPUT_ROOT/ahri-tre-test-datastore-deployment-kit-0.3.23.tar.gz.sha256" \
  --evidence-dir "$EVIDENCE_ROOT" \
  --confirm-host svrltreapcc02 \
  --confirm-address 192.168.31.75 \
  --confirm-deployment "$DEPLOYMENT_ID" \
  --confirm-datastore ahri-tre-test \
  --safe-identity "$SAFE_IDENTITY" \
  --backup-name conformance-0.3.23

Positive output is Linux server conformance phase=recovery outcome=go.

11. Run the final WSL2 platform phase

In WSL2, transfer the exact candidate archive and checksum into the private input directory, verify or extract it, create an empty evidence directory, and run the full platform action:

If WSL2 is temporarily unavailable, skip this section, complete section 12, run the deferred checkpoint immediately after it, and stop before cleanup. Return to this section later with the same immutable candidate.

WSL_INPUT_ROOT="$HOME/ahri-tre-conformance/input"
WSL_CANDIDATE_PARENT="$HOME/ahri-tre-conformance/wsl2-candidate-0.3.23"
WSL_CANDIDATE_ROOT="$WSL_CANDIDATE_PARENT/ahri-tre-test-datastore-deployment-kit-0.3.23"
WSL_EVIDENCE="$HOME/ahri-tre-conformance/wsl2-final-evidence"
DEPLOYMENT_ID=f2ef37c5-7430-468a-a439-b3ba1b0527c1

cd "$WSL_INPUT_ROOT"
grep -qxF '8c4664cd7fb48c040869c09172cf15cba2bb324c2958b0d97407c424ce20fcea  ahri-tre-test-datastore-deployment-kit-0.3.23.tar.gz' \
  ahri-tre-test-datastore-deployment-kit-0.3.23.tar.gz.sha256
sha256sum --check --strict \
  ahri-tre-test-datastore-deployment-kit-0.3.23.tar.gz.sha256
if [[ ! -e "$WSL_CANDIDATE_ROOT" ]]; then
  install -d -m 0700 "$WSL_CANDIDATE_PARENT"
  tar --extract --gzip \
    --file ahri-tre-test-datastore-deployment-kit-0.3.23.tar.gz \
    --directory "$WSL_CANDIDATE_PARENT" --no-same-owner --no-same-permissions
fi
test -d "$WSL_CANDIDATE_ROOT"
test ! -L "$WSL_CANDIDATE_ROOT"
(cd "$WSL_CANDIDATE_ROOT" && sha256sum --check --strict SHA256SUMS)
test ! -e "$WSL_EVIDENCE"
install -d -m 0700 "$WSL_EVIDENCE"

"$WSL_CANDIDATE_ROOT/conformance/run.sh" wsl2 \
  --kit-archive "$WSL_INPUT_ROOT/ahri-tre-test-datastore-deployment-kit-0.3.23.tar.gz" \
  --kit-sha256 "$WSL_INPUT_ROOT/ahri-tre-test-datastore-deployment-kit-0.3.23.tar.gz.sha256" \
  --evidence-dir "$WSL_EVIDENCE" \
  --user "$USER" \
  --confirm-clean-host wsl2 \
  --confirm-deployment "$DEPLOYMENT_ID" \
  --confirm-datastore ahri-tre-test \
  --admitted-identity 0009-0005-0445-6675 \
  --unadmitted-identity 0009-0007-0768-5937

Use the admitted account for the first browser step. Sign it out at https://sandbox.orcid.org, then use the unadmitted account only when prompted. Positive output is wsl2 installed-package evidence outcome=go.

Transfer the result:

scp "$WSL_EVIDENCE/wsl2-client.json" \
  sysadmin@192.168.31.75:/home/sysadmin/ahri-tre-conformance/input/wsl2-client.json

On the MinisForum:

sudo install -o root -g root -m 0600 \
  /home/sysadmin/ahri-tre-conformance/input/wsl2-client.json \
  "$EVIDENCE_ROOT/wsl2-client.json"

12. Run the native Apple Silicon macOS phase

Transfer the same candidate archive and checksum to a private macOS directory. Do not transfer an unpacked client package. On the Mac:

MAC_INPUT_ROOT="$HOME/ahri-tre-conformance/input"
MAC_CANDIDATE_PARENT="$HOME/ahri-tre-conformance/macos-candidate-0.3.23"
MAC_CANDIDATE_ROOT="$MAC_CANDIDATE_PARENT/ahri-tre-test-datastore-deployment-kit-0.3.23"
MAC_EVIDENCE="$HOME/ahri-tre-conformance/macos-evidence"
DEPLOYMENT_ID=f2ef37c5-7430-468a-a439-b3ba1b0527c1

cd "$MAC_INPUT_ROOT"
grep -qxF '8c4664cd7fb48c040869c09172cf15cba2bb324c2958b0d97407c424ce20fcea  ahri-tre-test-datastore-deployment-kit-0.3.23.tar.gz' \
  ahri-tre-test-datastore-deployment-kit-0.3.23.tar.gz.sha256
shasum --algorithm 256 --check \
  ahri-tre-test-datastore-deployment-kit-0.3.23.tar.gz.sha256
test -d "$MAC_CANDIDATE_ROOT"
test ! -L "$MAC_CANDIDATE_ROOT"
test ! -e "$MAC_EVIDENCE"
install -d -m 0700 "$MAC_EVIDENCE"
(cd "$MAC_CANDIDATE_ROOT" && shasum --algorithm 256 --check SHA256SUMS)

"$MAC_CANDIDATE_ROOT/conformance/run.sh" macos \
  --kit-archive "$MAC_INPUT_ROOT/ahri-tre-test-datastore-deployment-kit-0.3.23.tar.gz" \
  --kit-sha256 "$MAC_INPUT_ROOT/ahri-tre-test-datastore-deployment-kit-0.3.23.tar.gz.sha256" \
  --evidence-dir "$MAC_EVIDENCE" \
  --user "$(id -un)" \
  --confirm-clean-host macos \
  --confirm-deployment "$DEPLOYMENT_ID" \
  --confirm-datastore ahri-tre-test \
  --admitted-identity 0009-0005-0445-6675 \
  --unadmitted-identity 0009-0007-0768-5937

If the first installation requests the one-time credential-adapter activation reboot, the harness exits safely with a platform.install no-go and uninstalls the client. Reboot the Mac, restore the shell variables above, and preserve the exact resumable record before retrying:

PLATFORM_REBOOT_RECORD="$HOME/ahri-tre-conformance/macos-platform-install-reboot-0.3.23.json"
test ! -e "$PLATFORM_REBOOT_RECORD"
jq -e '
  .schema_version == "ahri-tre.test-datastore-kit.evidence.v1" and
  .role == "macos-client" and .target == "aarch64-apple-darwin" and
  .outcome == "no-go" and .platform == {name:"macos",clean_host:true} and
  .failed_stage == "platform.install" and
  .failure_code == "installed_stage_failed"
' "$MAC_EVIDENCE/macos-client.json" >/dev/null
mv "$MAC_EVIDENCE/macos-client.json" "$PLATFORM_REBOOT_RECORD"
test -z "$(find "$MAC_EVIDENCE" -mindepth 1 -maxdepth 1 -print -quit)"

Then rerun the same macos command once against the now-empty evidence directory. Do not recreate the directory and do not reboot the MinisForum.

This requires native Apple Silicon macOS 26. Follow the same admitted then unadmitted browser-account sequence. Positive output is macos installed-package evidence outcome=go.

Transfer macos-client.json to the MinisForum and install it into the evidence directory:

scp "$MAC_EVIDENCE/macos-client.json" \
  sysadmin@192.168.31.75:/home/sysadmin/ahri-tre-conformance/input/macos-client.json

On the MinisForum:

sudo install -o root -g root -m 0600 \
  /home/sysadmin/ahri-tre-conformance/input/macos-client.json \
  "$EVIDENCE_ROOT/macos-client.json"

Deferred-finalization checkpoint

When the final WSL2 record is not yet available, verify the complete recovery and native macOS state on the still-installed server:

sudo "$CANDIDATE_ROOT/conformance/run.sh" recovery-checkpoint \
  --kit-archive "$INPUT_ROOT/ahri-tre-test-datastore-deployment-kit-0.3.23.tar.gz" \
  --kit-sha256 "$INPUT_ROOT/ahri-tre-test-datastore-deployment-kit-0.3.23.tar.gz.sha256" \
  --evidence-dir "$EVIDENCE_ROOT" \
  --confirm-host svrltreapcc02 \
  --confirm-address 192.168.31.75 \
  --confirm-deployment "$DEPLOYMENT_ID" \
  --confirm-datastore ahri-tre-test \
  --safe-identity "$SAFE_IDENTITY" \
  --backup-name conformance-0.3.23

Positive output is Linux server conformance phase=recovery-checkpoint outcome=go. Stop here and preserve the installed server, candidate archive and checksum, backup, and evidence directory. When WSL2 becomes available, rerun this checkpoint first, complete section 11, install wsl2-client.json into the evidence directory, then continue to cleanup. Finalization still fails closed without that genuine WSL2 record.

13. Run final cleanup

Reverify the candidate checksum on the MinisForum, then invoke cleanup through the harness embedded in those exact bytes:

cd "$INPUT_ROOT"
sha256sum --check --strict \
  ahri-tre-test-datastore-deployment-kit-0.3.23.tar.gz.sha256

sudo "$CANDIDATE_ROOT/conformance/run.sh" cleanup \
  --platform server \
  --evidence-dir "$EVIDENCE_ROOT" \
  --confirm-host svrltreapcc02 \
  --confirm-deployment "$DEPLOYMENT_ID" \
  --confirm-datastore ahri-tre-test

Cleanup removes package-owned server files and retains the recoverable Datastore, Lake, PostgreSQL container, logs, configuration, and backups. Positive output is server cleanup outcome=go.

14. Finalize and verify the evidence bundle

cd "$INPUT_ROOT"
sha256sum --check --strict \
  ahri-tre-test-datastore-deployment-kit-0.3.23.tar.gz.sha256

sudo "$CANDIDATE_ROOT/conformance/run.sh" finalize \
  --evidence-dir "$EVIDENCE_ROOT" \
  --confirm-deployment "$DEPLOYMENT_ID" \
  --confirm-datastore ahri-tre-test

sudo bash -c 'cd /var/backups/ahri-tre/conformance-evidence && sha256sum --check --strict SHA256SUMS'
sudo jq -e '.outcome == "go"' \
  /var/backups/ahri-tre/conformance-evidence/final-acceptance.json >/dev/null

Finalization succeeds only when the evidence directory contains exactly the declared server, WSL2, macOS, recovery, and recovery-operation records. Never copy raw browser output, credentials, Secret material, root identities, or any transient recovery qualification file into that directory.

Recovering a Conformance Candidate After Runtime Key Loss

Use this procedure when Secret preparation stops because the Runtime private key is missing or does not match the Runtime certificate in the 0.3.14 installed-conformance candidate. It creates a fresh Runtime key, issues a new Runtime certificate beneath the retained deployment CA, regenerates the site bundle, and composes a replacement candidate archive and checksum.

This is a pre-conformance recovery. Do not use it after any server, WSL2, macOS, recovery, cleanup, or finalization phase has consumed the candidate. Once any phase has started, preserve that candidate and its evidence as one immutable attempt instead of replacing its bytes.

The procedure retains all identities and public authorities except the Runtime leaf key pair:

• kit 0.3.14 and release 0.10.8;
• host svrltreapcc02 at 192.168.31.75;
• Runtime DNS name runtime.svrltreapcc02.home.arpa;
• Deployment f2ef37c5-7430-468a-a439-b3ba1b0527c1;
• Datastore ahri-tre-test;
• the existing deployment CA and PostgreSQL CA;
• the existing server and client package bytes; and
• the fixed 0.3.12 predecessor and its checksum.

The existing clients can be retained because they authenticate the Runtime through the deployment CA rather than pinning the Runtime leaf certificate. This procedure proves that their complete generated public input surface is unchanged before reusing them.

Run WSL2 commands in the ordinary Ubuntu shell. Run Cargo commands in the persistent VS Code development container. Never display a private key, put a CA passphrase on a command line, or copy private material into the repository, candidate archive, terminal log, screenshot, or evidence directory. Keep the WSL2 shell open while using the separate development-container terminal so that its reviewed path variables remain in scope.

Guided recovery wizard

The dedicated wizard performs Sections 1 through 12 in order. Run it from the repository root in the ordinary WSL2 Ubuntu shell:

./scripts/recover-installed-conformance-candidate-wizard.sh

The wizard fixes the candidate, predecessor, host, address, Deployment, Datastore, and public identities to the values documented below. It asks for confirmation before retiring the old public leaf inputs, advancing the deployment CA serial, and activating the replacement candidate. The CA-key passphrase is read only by OpenSSL and is never stored. The wizard pauses while the operator runs the displayed package-site command in the persistent development container, then independently verifies its output.

Successful completion prints CANDIDATE READY. It does not install a package, invoke the conformance harness, or prepare the MinisForum host. Continue with the Secret-preparation wizard only after that result. The detailed procedure below remains the canonical explanation of every wizard action and the recovery reference if an interrupted run leaves guarded output paths behind. If an interruption leaves only the verified recovery source plus a protected Runtime key and matching public CSR, rerunning the wizard verifies and reuses those exact inputs instead of generating another key. If an interruption occurs after package-site but before the clients are copied, rerunning also recognizes the complete issued identity and a component root containing exactly server and site. It reverifies those artifacts, does not reopen the CA signing key, does not advance the CA serial, and resumes at the Stage 9 verification boundary.

1. Confirm the recovery boundary

In WSL2, confirm that the preparation wizard stopped before declaring the host ready and that no conformance phase used the current candidate. Do not proceed unless both statements are true.

Set the fixed paths:

cd /home/kobus/repos/ahri-tre-rs
repository_root="$PWD"
input_root="$HOME/ahri-tre-conformance/input"
archive="ahri-tre-test-datastore-deployment-kit-0.3.14.tar.gz"
checksum="$archive.sha256"
predecessor="ahri-tre-test-datastore-deployment-kit-0.3.12.tar.gz"
predecessor_checksum="$predecessor.sha256"
runtime_name="runtime.svrltreapcc02.home.arpa"

Verify the current candidate and the fixed predecessor before using either as a recovery input:

test -s "$input_root/$archive"
test -s "$input_root/$checksum"
test -s "$input_root/$predecessor"
test -s "$input_root/$predecessor_checksum"

cd "$input_root"
sha256sum --check --strict "$checksum"
grep -qxF \
  "6f5555c25d96274d7772d1b46d4409bd36a42205f21c4c715de8df5179ab18ed  $predecessor" \
  "$predecessor_checksum"
sha256sum --check --strict "$predecessor_checksum"

All checks must report OK. Return to the repository and extract the current candidate into a new private recovery directory:

cd "$repository_root"
source_parent="$repository_root/dist/conformance-candidate-recovery-source-0.3.14"
test ! -e "$source_parent"
install -d -m 0700 "$source_parent"
tar --extract --gzip \
  --file "$input_root/$archive" \
  --directory "$source_parent" \
  --no-same-owner \
  --no-same-permissions
source_root="$source_parent/ahri-tre-test-datastore-deployment-kit-0.3.14"

test -f "$source_root/SHA256SUMS"
(cd "$source_root" && sha256sum --check --strict SHA256SUMS)

Do not alter this extracted source. It supplies the already verified server and clients and the old site inputs used for byte comparisons.

2. Verify the retained deployment CA

The recovery reuses the existing deployment CA that was previously generated to issue certificates for this MinisForum deployment. It does not create or replace a CA. The CA private key remains protected in WSL2; the MinisForum uses certificates and public trust issued beneath it, not this signing key. Confirm the protected and public inputs:

umask 077
pki_root="$HOME/ahri-tre-pki"
ca_key="$pki_root/ca/ca-private-key.pem"
ca_certificate="$pki_root/ca/deployment-ca-certificate.pem"
ca_serial="$pki_root/ca/deployment-ca-certificate.srl"
ca_chain="$pki_root/public/deployment-ca-chain.pem"

test -s "$ca_key"
test -s "$ca_certificate"
test -s "$ca_serial"
test -s "$ca_chain"
test "$(grep -c 'BEGIN CERTIFICATE' "$ca_chain")" -eq 1
cmp --silent "$ca_certificate" "$ca_chain"
openssl verify -CAfile "$ca_chain" "$ca_certificate"

Confirm that the private signing key belongs to that public CA. OpenSSL will prompt for the passphrase chosen when this existing deployment CA was originally generated; it is not asking you to create a new password or enter a Runtime password. Enter the existing CA-key passphrase interactively:

ca_key_sha256="$({
  openssl pkey -in "$ca_key" -pubout -outform DER
} | sha256sum | cut -d ' ' -f 1)"
ca_certificate_sha256="$({
  openssl x509 -in "$ca_certificate" -pubkey -noout |
    openssl pkey -pubin -outform DER
} | sha256sum | cut -d ' ' -f 1)"
test "$ca_key_sha256" = "$ca_certificate_sha256"
unset ca_key_sha256 ca_certificate_sha256

The final test must return silently. A failure means this is not the signing authority embedded in the existing candidate; stop without generating a leaf certificate.

3. Retire the superseded public Runtime inputs

Preserve the old public leaf material for diagnosis while keeping it out of the active input paths:

retired_runtime="$pki_root/retired/runtime-leaf-before-conformance-$(date -u +%Y%m%dT%H%M%SZ)"
install -d -m 0700 "$retired_runtime"

for path in \
  "$pki_root/runtime-certificate.csr" \
  "$pki_root/runtime-certificate.ext" \
  "$pki_root/public/runtime-certificate.pem" \
  "$pki_root/public/runtime-certificate-chain.pem"; do
  if test -e "$path"; then
    mv -- "$path" "$retired_runtime/"
  fi
done

This does not retire or replace the deployment CA, PostgreSQL CA, or their private keys.

4. Generate the fresh Runtime key and request

The preparation wizard requires the protected Runtime key at this exact WSL2 path:

/home/kobus/ahri-tre-pki/private/runtime-private-key.pem

Create it only if it is absent:

runtime_key="$pki_root/private/runtime-private-key.pem"
runtime_csr="$pki_root/runtime-certificate.csr"
runtime_extension="$pki_root/runtime-certificate.ext"
runtime_certificate="$pki_root/public/runtime-certificate.pem"
runtime_chain="$pki_root/public/runtime-certificate-chain.pem"

install -d -m 0700 "$pki_root/private" "$pki_root/public"
test ! -e "$runtime_key"
test ! -e "$runtime_csr"
test ! -e "$runtime_extension"
test ! -e "$runtime_certificate"
test ! -e "$runtime_chain"

openssl genpkey \
  -algorithm RSA \
  -pkeyopt rsa_keygen_bits:3072 \
  -out "$runtime_key"
chmod 0600 "$runtime_key"
openssl pkey -in "$runtime_key" -check -noout

The key is deliberately not passphrase-encrypted because the Runtime service cannot answer a prompt. Its protected WSL2 storage and the wizard’s guarded Runtime-only projection provide its access boundary.

Create and verify a certificate-signing request:

openssl req \
  -new \
  -sha256 \
  -key "$runtime_key" \
  -subj "/CN=$runtime_name" \
  -addext "subjectAltName=DNS:$runtime_name" \
  -addext "keyUsage=critical,digitalSignature,keyEncipherment" \
  -addext "extendedKeyUsage=serverAuth" \
  -out "$runtime_csr"

openssl req -in "$runtime_csr" -verify -noout
openssl req -in "$runtime_csr" -noout -subject
openssl req -in "$runtime_csr" -noout -text | grep -F "DNS:$runtime_name"

The final output must show DNS:runtime.svrltreapcc02.home.arpa.

5. Sign and verify the Runtime certificate

Create the certificate extension file:

nano "$runtime_extension"

Paste exactly:

[server_certificate]
basicConstraints = critical, CA:false
keyUsage = critical, digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @subject_alt_names
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid,issuer

[subject_alt_names]
DNS.1 = runtime.svrltreapcc02.home.arpa

Save and exit. Issue the certificate for 365 days beneath the retained CA:

openssl x509 \
  -req \
  -sha256 \
  -days 365 \
  -in "$runtime_csr" \
  -CA "$ca_certificate" \
  -CAkey "$ca_key" \
  -CAserial "$ca_serial" \
  -extfile "$runtime_extension" \
  -extensions server_certificate \
  -out "$runtime_certificate"
chmod 0644 "$runtime_csr" "$runtime_extension" "$runtime_certificate"

Enter the CA-key passphrase only at the OpenSSL prompt. Verify the certificate chain, hostname, purpose, and dates:

openssl verify \
  -CAfile "$ca_chain" \
  -purpose sslserver \
  "$runtime_certificate"
openssl x509 \
  -in "$runtime_certificate" \
  -noout \
  -checkhost "$runtime_name"
openssl x509 \
  -in "$runtime_certificate" \
  -noout \
  -subject \
  -issuer \
  -dates \
  -ext subjectAltName

Prove that the certificate contains the public key derived from the new private key:

runtime_key_sha256="$({
  openssl pkey -in "$runtime_key" -pubout -outform DER
} | sha256sum | cut -d ' ' -f 1)"
runtime_certificate_sha256="$({
  openssl x509 -in "$runtime_certificate" -pubkey -noout |
    openssl pkey -pubin -outform DER
} | sha256sum | cut -d ' ' -f 1)"
test "$runtime_key_sha256" = "$runtime_certificate_sha256"
unset runtime_key_sha256 runtime_certificate_sha256

Because this deployment CA has no intermediate CA, the Runtime chain contains only the leaf certificate:

install -m 0644 "$runtime_certificate" "$runtime_chain"
openssl verify -CAfile "$ca_chain" "$runtime_chain"
test "$(grep -c 'BEGIN CERTIFICATE' "$runtime_chain")" -eq 1
if grep -q 'PRIVATE KEY' "$runtime_chain" "$ca_chain"; then
  echo 'STOP: a public certificate input contains private-key material'
  false
fi

6. Stage the public packager inputs

Only the public Runtime and deployment chains enter the repository working tree. They are local ignored build inputs and must not be committed:

cd "$repository_root"
install -d -m 0755 pki
install -m 0644 "$runtime_chain" pki/runtime-certificate-chain.pem
install -m 0644 "$ca_chain" pki/deployment-ca-chain.pem

openssl verify \
  -CAfile pki/deployment-ca-chain.pem \
  pki/runtime-certificate-chain.pem
cmp --silent pki/deployment-ca-chain.pem \
  "$source_root/site/public-ca-chain.pem"
git check-ignore pki/runtime-certificate-chain.pem \
  pki/deployment-ca-chain.pem

Both paths must be reported by git check-ignore. A CA comparison failure is authority drift, not Runtime leaf renewal; stop and do not reuse the existing clients.

7. Prepare a fresh component root

Still in the ordinary WSL2 shell, create a new component root and copy only the verified server publication into it:

new_component_root="$repository_root/dist/conformance-candidate-rebuild-0.3.14"
test ! -e "$new_component_root"
install -d -m 0700 "$new_component_root"
cp -a "$source_root/server" "$new_component_root/server"

test -s \
  "$repository_root/dist/private-kit-inputs-0.3.14/site-inputs.minisforum.v3.json"
test -s \
  "$repository_root/dist/private-kit-inputs-0.3.14/postgresql-ca-chain.pem"
test -s \
  "$repository_root/dist/release-v0.10.8/validator/release-metadata.json"

The component root contains no installed package and must not be transferred to a conformance host.

8. Regenerate the site bundle

Switch to the persistent VS Code development-container terminal. Run:

cd /workspaces/ahri-tre-rs

cargo run --locked -p xtask -- test-datastore-kit package-site \
  --contract deployment/test-datastore-kit/minisforum-contract-v0.3.14.json \
  --inputs dist/private-kit-inputs-0.3.14/site-inputs.minisforum.v3.json \
  --runtime-certificate-chain pki/runtime-certificate-chain.pem \
  --public-ca-chain pki/deployment-ca-chain.pem \
  --postgresql-ca-chain dist/private-kit-inputs-0.3.14/postgresql-ca-chain.pem \
  --server-component-manifest dist/conformance-candidate-rebuild-0.3.14/server/component-versions.json \
  --validator-release-root dist/release-v0.10.8/validator \
  --artifact-root dist/conformance-candidate-rebuild-0.3.14

The command must report kit 0.3.14, release 0.10.8, host svrltreapcc02, and Datastore ahri-tre-test. It must not install any package or invoke the conformance harness.

9. Prove the regenerated site matches the fixed candidate identity

Return to the ordinary WSL2 shell:

cd "$repository_root"
new_component_root="$repository_root/dist/conformance-candidate-rebuild-0.3.14"
new_site="$new_component_root/site"

jq -e '
  .kit_version == "0.3.14" and
  .server.artifact_base_release == "0.10.8" and
  .server.artifact_source_revision ==
    "a292eab3d21a02060b115927da3266f62ad982a0" and
  .hostname == "svrltreapcc02" and
  .ipv4_address == "192.168.31.75" and
  .runtime_dns_name == "runtime.svrltreapcc02.home.arpa" and
  .deployment_id == "f2ef37c5-7430-468a-a439-b3ba1b0527c1" and
  .datastore_id == "ahri-tre-test"
' "$new_site/site-inputs.json" >/dev/null

Extract only the public Runtime leaf from the regenerated Application configuration and bind it to the new key:

runtime_review="$(mktemp -d /tmp/ahri-tre-runtime-review.XXXXXX)"
chmod 0700 "$runtime_review"
python3 -c \
  'import sys,tomllib; document=tomllib.load(open(sys.argv[1], "rb")); print(document["services"]["trusted_runtime"]["certificate_chain"][0], end="")' \
  "$new_site/application.toml" \
  >"$runtime_review/runtime-certificate.pem"

openssl verify \
  -CAfile "$new_site/public-ca-chain.pem" \
  "$runtime_review/runtime-certificate.pem"
openssl x509 \
  -in "$runtime_review/runtime-certificate.pem" \
  -noout \
  -checkhost "$runtime_name"
issued_certificate_sha256="$(
  openssl x509 -in "$runtime_certificate" -outform DER |
    sha256sum | cut -d ' ' -f 1
)"
new_certificate_sha256="$(
  openssl x509 -in "$runtime_review/runtime-certificate.pem" -outform DER |
    sha256sum | cut -d ' ' -f 1
)"
test "$issued_certificate_sha256" = "$new_certificate_sha256"

new_certificate_public_key_sha256="$({
  openssl x509 -in "$runtime_review/runtime-certificate.pem" -pubkey -noout |
    openssl pkey -pubin -outform DER
} | sha256sum | cut -d ' ' -f 1)"
new_key_public_key_sha256="$({
  openssl pkey -in "$runtime_key" -pubout -outform DER
} | sha256sum | cut -d ' ' -f 1)"
test "$new_certificate_public_key_sha256" = "$new_key_public_key_sha256"
unset issued_certificate_sha256 new_certificate_sha256
unset new_certificate_public_key_sha256 new_key_public_key_sha256
rm -rf -- "$runtime_review"

The certificate digest comparison uses canonical DER bytes so that harmless PEM trailing-newline differences do not look like certificate drift. The separate public-key comparison proves that the same certificate is bound to the freshly generated Runtime private key.

The temporary directory has an explicit /tmp/ahri-tre-runtime-review. prefix and contains public certificate material only.

10. Prove the existing client packages remain exact

Compare every site input bound into the WSL2 and macOS client manifests:

for relative_path in \
  client.toml \
  public-ca-chain.pem \
  client-publication/wsl2.json \
  client-publication/macos.json; do
  cmp --silent \
    "$source_root/site/$relative_path" \
    "$new_site/$relative_path"
done

All comparisons must return silently. If any comparison fails, stop. That is broader site or trust drift and requires newly packaged clients; it is not the leaf-key recovery described here.

Copy the unchanged, checksum-verified client publications into the fresh component root and verify them again:

cp -a "$source_root/clients" "$new_component_root/clients"
(cd "$new_component_root/clients/wsl2" && \
  sha256sum --check --strict SHA256SUMS)
(cd "$new_component_root/clients/macos" && \
  sha256sum --check --strict SHA256SUMS)

test "$(find "$new_component_root" -mindepth 1 -maxdepth 1 \
  -printf '%f\n' | sort | paste -sd ' ' -)" = "clients server site"
test -z "$(find "$new_component_root" -type l -print -quit)"

11. Compose and verify the replacement candidate

Compose into a new protected staging directory. Do not overwrite the current candidate yet:

candidate_staging="$HOME/ahri-tre-conformance/candidate-staging-0.3.14-runtime-reissue"
test ! -e "$candidate_staging"
install -d -m 0700 "$candidate_staging"

cd "$repository_root"
./scripts/compose-installed-conformance-candidate.sh \
  --component-root "$new_component_root" \
  --output-directory "$candidate_staging"

cd "$candidate_staging"
sha256sum --check --strict "$checksum"

Verify the complete internal checksum inventory in a fresh temporary extraction:

candidate_review="$(mktemp -d /tmp/ahri-tre-candidate-review.XXXXXX)"
chmod 0700 "$candidate_review"
tar --extract --gzip \
  --file "$candidate_staging/$archive" \
  --directory "$candidate_review" \
  --no-same-owner \
  --no-same-permissions
candidate_root="$candidate_review/ahri-tre-test-datastore-deployment-kit-0.3.14"

(cd "$candidate_root" && sha256sum --check --strict SHA256SUMS)
test -x "$candidate_root/conformance/run.sh"
test -f "$candidate_root/clients/wsl2/ahri-tre-client.tar"
test -f "$candidate_root/clients/macos/ahri-tre-client.tar"
jq -e '
  .kit.version == "0.3.14" and
  .release.version == "0.10.8" and
  .qualification_evidence_included == false and
  .validation.outcome == "go"
' "$candidate_root/candidate-manifest.json" >/dev/null

rm -rf -- "$candidate_review"

This verification reads package files but does not install them or invoke the conformance harness.

12. Activate the exact replacement bytes

Retire the old candidate only after the replacement archive and checksum have passed both verification layers:

retired_candidate="$HOME/ahri-tre-conformance/retired/candidate-before-runtime-reissue-$(date -u +%Y%m%dT%H%M%SZ)"
install -d -m 0700 "$retired_candidate"

test -s "$input_root/$archive"
test -s "$input_root/$checksum"
(cd "$input_root" && sha256sum --check --strict "$checksum")
mv -- "$input_root/$archive" "$retired_candidate/"
mv -- "$input_root/$checksum" "$retired_candidate/"

mv -- "$candidate_staging/$archive" "$input_root/"
mv -- "$candidate_staging/$checksum" "$input_root/"

cd "$input_root"
sha256sum --check --strict "$checksum"
sha256sum --check --strict "$predecessor_checksum"

The two final checks must report OK. From this point onward, the replacement candidate archive and its adjacent checksum are the only 0.3.14 bytes used by the server, WSL2, macOS, recovery, cleanup, and finalization phases. Keep the fixed predecessor archive and checksum unchanged.

Return to the repository root and restart Secret preparation:

cd "$repository_root"
./scripts/minisforum-conformance-preparation-wizard.sh

The wizard will independently verify that the new Runtime private key matches the certificate embedded in the replacement candidate. It still stops after printing HOST READY; it never invokes the conformance harness.

Updating the Test Datastore Server

This procedure updates an existing MinisForum Test Datastore without recreating the Datastore. For an empty host, use Installing the Test Datastore Server. Every release-specific command comes from one immutable Test Datastore Deployment Kit. Do not combine the server directory from one kit with the site directory or validator image from another.

The worked example below updates the confirmed MinisForum svrltreapcc02 (192.168.31.75) to:

• AHRI TRE server v0.10.8;
• PostgreSQL ORCID validator v0.10.8; and
• MinisForum site kit 0.3.14.

Publication gate: do not start this update until the GitHub v0.10.8 release actually lists both the 0.3.14 archive and its .sha256 asset. The source implementation alone is not an installable release.

There are two command locations:

• WSL2 is the Ubuntu terminal on the Windows 11 desktop.
• MinisForum is the remote Ubuntu shell whose prompt starts with sysadmin@svrltreapcc02.

Copy each command as one complete line. Do not copy the displayed prompt. If less opens a file, press q to return to the command prompt.

When this repository is available in WSL2, the interactive equivalent of Sections 1 through 12 is run from the repository root with:

./scripts/minisforum-update-wizard.sh

The wizard uses these same guarded kit operations and prints the password location before every remote sudo stage. This chapter remains the authoritative explanation and recovery reference.

Before starting

The supported predecessor is the current site lifecycle at kit 0.3.13, the v0.10.6 server package first shipped in kit 0.3.12, and the healthy v0.10.6 validator. The update briefly restarts PostgreSQL and the Trusted runtime. Sections 6 through 8 include exact restart checks in case an earlier attempt completed one component before stopping. Do not start if an ingest, export, Session, or other Datastore operation is active.

The update preserves Application configuration, Secret material, Managed secrets, Lake files, PostgreSQL data, and the Datastore identity. It does not ask for or transfer passwords, private keys, ORCID tokens, or the ORCID client secret.

1. WSL2: download the published update kit

Open the WSL2 Ubuntu terminal and create a version-specific download directory:

mkdir -p ~/ahri-tre-updates/v0.10.8

Download the kit and its checksum directly from the GitHub release:

gh release download v0.10.8 --repo AHRIORG/ahri-tre-rs --pattern 'ahri-tre-test-datastore-deployment-kit-0.3.14.tar.gz' --pattern 'ahri-tre-test-datastore-deployment-kit-0.3.14.tar.gz.sha256' --dir ~/ahri-tre-updates/v0.10.8

Verify the download:

cd ~/ahri-tre-updates/v0.10.8

sha256sum --check ahri-tre-test-datastore-deployment-kit-0.3.14.tar.gz.sha256

Stop unless the result is exactly:

ahri-tre-test-datastore-deployment-kit-0.3.14.tar.gz: OK

2. WSL2: transfer the verified kit

Create a version-specific transfer directory on the MinisForum:

ssh sysadmin@192.168.31.75 'umask 077; mkdir -p ~/ahri-tre-transfer/v0.10.8'

Copy both files:

scp ahri-tre-test-datastore-deployment-kit-0.3.14.tar.gz ahri-tre-test-datastore-deployment-kit-0.3.14.tar.gz.sha256 sysadmin@192.168.31.75:ahri-tre-transfer/v0.10.8/

Connect to the MinisForum:

ssh sysadmin@192.168.31.75

All remaining commands run on the MinisForum.

3. MinisForum: verify and extract the kit

Verify the transferred bytes again:

cd ~/ahri-tre-transfer/v0.10.8

sha256sum --check ahri-tre-test-datastore-deployment-kit-0.3.14.tar.gz.sha256

Stop unless the result says OK. Extract the kit beside earlier versions:

tar -xzf ahri-tre-test-datastore-deployment-kit-0.3.14.tar.gz

Enter the new kit directory:

cd ~/ahri-tre-transfer/v0.10.8/ahri-tre-test-datastore-deployment-kit-0.3.14

Do not copy files into an older kit directory. The scripts, manifests, server binaries, and validator image in this directory are one release-bound unit.

4. MinisForum: prove the current installation is healthy

Check PostgreSQL:

docker inspect --format '{{.Config.Image}} {{.State.Health.Status}}' ahri-tre-postgresql

The current image version may be older, but its status must be healthy.

Check the Trusted runtime:

sudo systemctl is-active ahri-tre-runtime.service

Expected: active.

Check the existing Datastore binding:

D=ahri-tre-test; R=/usr/libexec/ahri-tre/ahri-tre-runtime; sudo -u ahri-tre-runtime env LD_LIBRARY_PATH=/usr/lib/ahri-tre "$R" datastore reconcile "$D"

Expected: JSON containing "status":"ready".

Do not use an update to repair an installation that fails these checks. Record the failing output and diagnose the current installation first.

5. MinisForum: create a pre-update backup

Run the new kit’s bounded backup procedure before changing either component:

sudo install -d -o root -g root -m 0700 /var/backups/ahri-tre/manual

sudo site/backup.sh /var/backups/ahri-tre/manual/pre-kit-0.3.14.dump

Confirm that new backup files exist:

sudo find /var/backups/ahri-tre -maxdepth 2 -type f -printf '%TY-%Tm-%Td %TH:%TM %p\n' | sort

The backup wrapper refuses to overwrite an existing output. If that exact path already exists, choose a new absolute filename containing the maintenance date; do not remove an older backup. Stop if the backup command fails or no new file appears. Keep this backup through live qualification and the later admitted and unadmitted client qualification.

6. MinisForum: upgrade the PostgreSQL validator

First inspect the running image:

docker inspect --format '{{.Config.Image}} {{.State.Health.Status}}' ahri-tre-postgresql

If this prints ahri-tre/postgresql-orcid-validator:0.10.8 healthy, the validator is already current: skip the upgrade command below and continue to Section 7. If it prints the healthy v0.10.6 image, run the upgrade. Any other image or a non-healthy state is a stop condition.

The upgrade verifies the exact healthy v0.10.6 predecessor, release OCI checksum, image labels, Docker bridge, ownership, Secret projections, and configuration. It briefly stops PostgreSQL and the Runtime, creates the v0.10.8 container over the preserved data, and retains the stopped predecessor as ahri-tre-postgresql-v0.10.6-rollback. A failed start restores v0.10.6.

sudo site/postgresql/upgrade-postgresql.sh --confirm-host svrltreapcc02 --confirm-address 192.168.31.75

Expected:

PostgreSQL validator upgraded to v0.10.8; rollback container retained as ahri-tre-postgresql-v0.10.6-rollback

Confirm the running image and health:

docker inspect --format '{{.Config.Image}} {{.State.Health.Status}}' ahri-tre-postgresql

Expected: ahri-tre/postgresql-orcid-validator:0.10.8 healthy.

7. MinisForum: upgrade the AHRI TRE server

Inspect the installed server manifest before changing it:

sudo jq -r '.base_release + " kit=" + .kit_version' /usr/share/ahri-tre/server/component-versions.json

If this prints v0.10.8 kit=0.3.14, the server is already the exact package required by kit 0.3.14: skip the upgrade command below and continue to Section 8. Otherwise, the expected predecessor is v0.10.6 kit=0.3.12.

The bundled server directory is the new release-bound v0.10.8/0.3.14 server package. Its upgrader retains the previous server files under /var/backups/ahri-tre/server-previous and restarts the Runtime against the upgraded PostgreSQL service.

sudo server/upgrade.sh

sudo systemctl is-active ahri-tre-runtime.service

Expected: active.

8. MinisForum: activate kit 0.3.14 metadata

The persistent Secret authority and reboot-qualified 0.3.6 lifecycle already exist. Before running the command, inspect the installed plan:

sudo jq -r '.kit_version' /etc/ahri-tre/secret-projection-plan.json

If it prints 0.3.14, skip the upgrade command and continue with verification. Otherwise it must print 0.3.13. The guarded operation admits only that exact predecessor plan, the installed v0.10.8/0.3.14 server package, the v0.10.8 container, and three active services. It replaces only the plan and rolls it back on failure.

sudo site/upgrade-secret-projector.sh --confirm-host svrltreapcc02 --confirm-address 192.168.31.75

Expected:

site lifecycle metadata upgraded to kit 0.3.14; v0.10.8 authorization mapping is ready

Verify the projector and ordered services immediately:

sudo /usr/libexec/ahri-tre/secret-projector.sh verify

sudo systemctl is-active ahri-tre-secret-projector.service ahri-tre-postgresql.service

Expected: verification succeeds and both services are active. Stop if it does not. Never inspect files below /var/lib/ahri-tre/injected-secret-authority with a content-printing command. Exclude that authority from broad file backups; it is Secret material.

Confirm the Runtime remained active:

sudo systemctl is-active ahri-tre-runtime.service

If the update reports a failure, it restores the prior installed projector and plan. Confirm that the container and Runtime remain healthy, then stop and diagnose. Do not reboot, delete, or recreate the container.

9. MinisForum: confirm admitted ORCID authority

Rerun admission for each intended ORCID identity. The operation is repeat-safe and confirms its canonical role, identity-map entry, bounded group membership, and Datastore privileges after the validator replacement.

For the confirmed admitted test identity, run:

sudo site/admit-user.sh 0009-0005-0445-6675

Expected:

admitted 0009-0005-0445-6675 as orcid_0009-0005-0445-6675

The ORCID identifier passed to admit-user.sh retains its hyphens. Do not manually create or rename a PostgreSQL role. Repeat this command with the hyphenated ORCID identifier of every other identity that should remain admitted.

10. MinisForum: verify the complete installation

Run the kit’s full readiness check:

sudo site/verify-readiness.sh

Expected: MinisForum v3 site is ready. Confirm its exit status immediately:

echo $?

Expected: 0.

If readiness instead reports that the PostgreSQL OAuth deployment is not ready, run the detailed check for the admitted identity:

sudo env PATH="$PWD/site/postgresql/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" site/postgresql/contract/operator.sh site/postgresql/deployment-contract.json ready 0009-0005-0445-6675

Positive validation is JSON containing "ready":true with the transport, issuer, validator, audience, hba, and admission checks all marked "status":"ready". An admission check with class missing_admission means the role or identity-map entry is absent. Class incomplete_admission means Section 9 must be repeated to repair bounded authority. Stop and diagnose any other failed check rather than continuing to client qualification.

Reconcile the preserved Datastore once more:

D=ahri-tre-test; R=/usr/libexec/ahri-tre/ahri-tre-runtime; sudo -u ahri-tre-runtime env LD_LIBRARY_PATH=/usr/lib/ahri-tre "$R" datastore reconcile "$D"

Expected: JSON containing "status":"ready" and the same Datastore identity as before the update.

11. Confirm the existing reboot qualification remains intact

Kit 0.3.14 replaces the application and validator with v0.10.8 but does not change the projector implementation, readiness waiter, or systemd ordering. Kit 0.3.6 already passed that boot-lifecycle gate. Do not reboot merely to activate this patch update. Positive server validation is the active services, exact Secret projection, site readiness, and preserved Datastore identity confirmed in Sections 6 through 10.

12. Complete release qualification

For this authentication correction, repeat the release kit’s real admitted and valid-but-unadmitted ORCID checks. For kit 0.3.14 these are in site/HITL.md, Section 11. Never save an ORCID token or Secret value in a command transcript or ticket. The unadmitted account is 0009-0007-0768-5937; never pass it to site/admit-user.sh.

Keep the pre-update Datastore backup until both qualification checks pass. Removing older rollback material is a separate, explicitly approved maintenance action.

Applying this procedure to a future release

Do not mechanically replace version numbers in this page. For each future update:

1. read the release notes and identify the exact server version, validator version, and site-kit version;
2. download that release’s site-kit archive and checksum together;
3. verify the checksum in WSL2 and again after transfer;
4. follow the site/HITL.md inside that exact kit for release-specific prerequisites and commands; and
5. retain rollback material until readiness and any required live qualification and reboot-persistence gates pass.

If a release does not publish a site kit for this MinisForum profile, it is not an authorized MinisForum update.

Developer Guide

Developers adding TRE features or fixing bugs use one non-root VS Code development-tool container over the authoritative host checkout. Opening the editor starts no PostgreSQL, Lake, Trusted runtime, Web, Browser, OIDC, or Integration workload.

The complete operating, platform, migration, command, diagnostic, and recovery reference is README-dev-env.md.

Understand the three scopes

The development environment deliberately separates editing, disposable integration testing, and durable product evaluation:

Opening VS Code starts only the first scope. The development container has no Docker socket, service credential, database authority, or Trusted-runtime authority. Run Local and Integration lifecycle commands from a host terminal; run Cargo and mdBook commands in the development-container terminal.

Start

1. Install a supported native Docker Engine with Compose v2, VS Code, and the Dev Containers extension.
2. On WSL2, keep the checkout in the distribution filesystem and open it through WSL integration. Native Linux and macOS use their host checkout.
3. Run ./dev-env doctor in a host terminal.
4. Choose Dev Containers: Rebuild and Reopen in Container.
5. Confirm host and container Git HEAD, branch, remotes, staged changes, and unstaged changes match.

The checkout is bind-mounted at /workspaces/ahri-tre-rs; it is not cloned inside the container and there is no separate Git metadata volume. A commit, branch change, staged file, or generated file is therefore the same object on the host and in the container. Stop and investigate if these checks differ:

git rev-parse HEAD
git symbolic-ref --quiet --short HEAD || printf 'detached\n'
git remote -v
git status --short --branch
git diff --cached --stat
git diff --stat

GitHub credentials and commit signing remain host-owned. The container may use VS Code’s supported HTTPS-helper or SSH-agent forwarding, but it receives no private key, credential store, GPG agent, GitHub CLI state, or Docker socket. If forwarding is unavailable, make authenticated Git operations and signed commits from a host terminal against this same checkout.

Validate a change

Run the standard loop inside the container as the mapped non-root user:

cargo fmt --all
cargo clippy --workspace --all-targets --all-features -- -D warnings
cargo test --workspace
mdbook test docs/book
mdbook build docs/book

The default test suite is non-live. It does not require PostgreSQL, a Lake, OAuth, ORCID, or Managed-secret authority. Build output is placed beneath /tmp/ahri-tre-rs-target in the container.

Run Integration tests

Run live adapter behavior from a host terminal through the disposable fixture:

./dev-env integration run
./dev-env integration run -- -p ahri_tre_pgmeta

The runner proves verified PostgreSQL TLS and a writable container-visible Lake before tests. It is a test runner, not governed compute. Diagnose retained failures only with the run ID printed by the command:

./dev-env integration status int-<id>
./dev-env integration logs int-<id>
./dev-env integration logs int-<id> runner
./dev-env integration shell int-<id>
./dev-env integration reset int-<id>

integration reset previews the exact owned fixture and requires its run ID as confirmation. Do not diagnose or remove fixtures with copied container IDs, raw Compose, or direct volume deletion.

Exercise Local TRE

Use Local TRE only when a change requires the durable non-production product topology:

./dev-env local up
./dev-env local status
./dev-env local test smoke
./dev-env local test e2e
./dev-env local down

local up builds or verifies immutable artifacts, validates the retained foundation, starts PostgreSQL and the Trusted runtime, reconciles configured Datastores through the protected administration socket, and starts the Web and Browser path. Repeating it validates and reuses retained state. It does not mount this checkout, Cargo output, a Docker socket, or a toolchain into the long-running workloads.

local status prints the checkout-scoped environment identifier, selected profile, active root/store slot, declared versions, service health, and loopback Browser and Trusted-runtime client origins without printing credentials or Restricted local references. local down stops workloads and removes temporary capability projections but retains PostgreSQL, Lake, configuration, trust, and Secret state. After a reboot, run local up again; Local services do not restart automatically.

For the configuration/Secrets cutover, local status reports exactly one of no state change, service rebuild, or guarded reset. Ticket 21 does not change the development-tool image, so it does not require rebuilding or reopening VS Code. Use local up for the reported Local rebuild, or local reset followed by local up for pre-cutover retained state; do not operate the topology with raw Compose.

To rotate the Local Managed-secret root, run:

./dev-env local rotate-root
./dev-env local up

The first command revokes temporary database exposure, stops all services, and stages into a distinct empty identity and store pair. It atomically selects the new pair, reconstructs the active mount topology, then runs secrets verify --all as the post-cutover startup gate and leaves services stopped. A recovery marker blocks local up if that sequence is interrupted; rerun rotate-root to resume. Long-running workloads never receive the next identity or staging-store mounts.

Use only the supported service selectors when inspecting redacted logs:

./dev-env local logs
./dev-env local logs trusted-runtime
./dev-env local logs --follow web

To rebuild one backend workload after a source change:

./dev-env local rebuild trusted-runtime
./dev-env local rebuild web

To test a companion TRE Browser checkout without mounting or retaining its Restricted local reference:

./dev-env local browser rebuild --source /path/to/ahri-tre-web

The command builds an immutable Browser image, verifies its compatibility, and records only the companion Git revision.

Use synthetic demo data

Ordinary local up leaves the durable development Datastore empty and does not create the separately declared demo Datastore. Create or recreate only the reviewed synthetic demo with:

./dev-env local seed demo

Reseeding replaces only the demo’s versioned Dataset tables. It does not alter the development or research Datastores.

Exercise real ORCID sandbox interoperability

The default Local profile uses deterministic test identities. Real ORCID sandbox interoperability is an explicit, human-only profile. Register exactly https://127.0.0.1:8443/auth/orcid/callback with the sandbox client and put the three required values in the owner-only file outside the checkout described in the development-environment reference. Then run from a host terminal:

./dev-env local up --profile orcid-sandbox
./dev-env local test orcid-sandbox
./dev-env local down

Only one sandbox checkout can use fixed port 8443. local down returns the deployment to the deterministic default profile and removes the projected capabilities without changing the contributor-owned credential file. Never paste authorization codes, tokens, cookies, ORCID identities, credentials, or credential paths into logs or evidence.

Inspect Local PostgreSQL

Database access is closed by default. For a temporary, read-only DBeaver diagnostic endpoint:

./dev-env local trust export
./dev-env local db expose
./dev-env local db status
./dev-env local db unexpose

trust export writes only the public Local CA to the ignored .dev-env/local-ca.crt. Configure DBeaver with the host, dynamic loopback port, role, CA, and password-file location printed by db expose; use SSL verify-full and keep hostname verification enabled. The password itself is never printed. Always run db unexpose when finished. local down also attempts the same revocation.

Recover and clean up

Start diagnosis with:

./dev-env doctor
./dev-env local status
./dev-env local logs <service>

Correct the named prerequisite or phase and rerun the same high-level command. The lifecycle is designed to reconcile partial startup safely. Do not edit .dev-env, copy a Docker ID into a repair command, invoke the underlying Compose files, or copy volumes.

Remove disposable Cargo output without touching Local or Integration state:

./dev-env clean artifacts

To irreversibly remove the current checkout’s entire Local TRE, run ./dev-env local reset. It first previews the owned resources, verifies their labels and network membership, and requires the full Local identifier as confirmation. It fails closed on ambiguous or mismatched ownership.

The most common host remedies are:

• move a WSL2 checkout from /mnt/<drive> into the distribution filesystem;
• start Docker and install or enable the Compose v2 plugin;
• free port 8443 before using the ORCID sandbox profile;
• rebuild and reopen the development container after changing its definition, Git credential helper, or SSH agent; and
• use the exact retained Integration run ID or Local service name printed by the failing command.

Boundaries

Read the relevant crate README before changing code and preserve the ownership rules in AGENTS.md. Do not reopen configuration or Secret stores from workflow code, leak Restricted local references into runtime code, mount source into Local TRE, or operate supported lifecycle state with raw Compose. Local TRE has no production-isolation, backup, observability-platform, RustFS, or governed-compute guarantee. Docker Engine or Docker Desktop with Compose v2 and native linux/amd64 or linux/arm64 containers is the qualified runtime; Podman, Rancher Desktop, Kubernetes, rootless variants, and architecture emulation are not qualified by this development environment.

Configuration and secrets

AHRI TRE has two versioned configuration documents and one Secret subsystem. They are deliberately separate authorities: trusted services use the complete Application configuration, user-side processes use the generated Client bootstrap, and Secret material never belongs in either document.

Application configuration

The operator-authored Application configuration is the deployment-wide, non-secret authority for trusted services and operator commands. Its canonical path is /etc/ahri-tre/config.toml. It contains the immutable Deployment identity, named infrastructure and policy registries, service declarations, logical Secret references, and reusable Execution profiles.

Schema version 1 is one closed TOML file. It has no includes, overlays, fragments, interpolation, or environment substitution. Unknown fields fail validation. The document contains no credential values, Restricted local references, user state, or live handles. Deployment tooling may render it from another system, but AHRI TRE validates and explains only the final file.

Client bootstrap configuration

The Client bootstrap is a generated, deployment-wide projection for Managed runtimes, the C ABI, language bindings, CLI users, and the JupyterHub broker. Its canonical path is /etc/ahri-tre/client.toml. The allowlist is intentionally narrow: schema version, the same Deployment identity, one canonical Trusted-runtime HTTPS origin, and the public Deployment CA chain.

It contains no Execution-profile definitions, datastore or worker topology, policy, Secret reference, private key, or user state. Operators generate it from the validated Application document:

ahri-tre --config /etc/ahri-tre/config.toml \
  config render-client --output client.toml

Use --replace only for an intentional atomic replacement. Deployment tooling distributes the completed file read-only and restarts Managed runtimes after a change. Rendering or parsing this file does not contact the Trusted runtime and is not Client-readiness evidence.

Effective configuration and fingerprints

An Effective configuration is the immutable, validated non-secret projection for one startup or operation. Resolution follows typed references, applies built-in defaults, selects the requested Execution profile or command target, and retains the origin of every effective value. Runtime and workflow code consume that projection rather than reopening the Application document.

The configuration fingerprint is a stable SHA-256 digest of the canonical Effective configuration. It includes logical Secret references and declared Injected-secret versions, but excludes comments, TOML ordering, the selected file path, Secret material, and mutable active Managed-secret versions. Actual resolved Managed versions are recorded separately as Secret-use provenance.

Use config show-effective to inspect the safe projection and fingerprint:

ahri-tre --config /etc/ahri-tre/config.toml config show-effective \
  --for execution-profile --profile research

Selection and selected-path validation

--config PATH is strict: a missing, unreadable, unsafe, wrong-kind, or invalid explicit document fails without fallback. Without it, trusted entry points use the canonical Application path and user-side entry points use the canonical Client path. The complete configuration-independent command set is help, top-level version, shell-completion generation, schema emission, and config init; those commands select neither document. Diagnostics are not an implicit configuration-independent fallback.

config validate checks the whole Application document offline. In contrast, config show-effective and config preflight select one startup or operation path. Unselected optional incompleteness may remain local to another path, while a selected path must be complete. Preflight resolves only that target’s required Secret capabilities and performs bounded dependency checks; it does not create, repair, migrate, authenticate a user, or prove service readiness.

Secret authorities

Application configuration stores only canonical injected:// and managed:// references. Injected secrets are deployment-projected, read-only value plus non-secret version files beneath /run/secrets; a trusted process snapshots its required projections at startup. Managed secrets are encrypted, Deployment-bound, audited artifacts beneath /var/lib/ahri-tre/secrets, protected by the separately projected X25519 root identity at /run/secrets/ahri-tre/root-identity/value.

Secret values enter only through a protected prompt, standard input, or a built-in generator. They never appear in configuration, arguments, public output, diagnostics, protocol envelopes, provenance, or plaintext export. See Sensitive Material Handling and the System Integrator Guide for lifecycle and recovery procedures.

No operational environment authority

Production AHRI TRE processes do not use operational environment variables or dotenv files as configuration or credential authority. Runtime, Web, daemon, CLI, C ABI, datastore, Lake, OAuth, and OIDC behavior comes from the selected document, its Effective configuration, and narrow Secret capabilities.

Recognized names from retired interfaces are fatal. The product names only the offending variable and exits before configuration selection or side effects; it never reads or prints the value.

Repository orchestration is a different boundary. dev-env and Compose use DEV_ENV_ names, and the disposable Integration runner may pass explicitly command-scoped fixture inputs to its test process. Those exceptions are not accepted by Local TRE services, installed packages, or production workloads.

Documentation and CI

Documentation tooling may use build metadata such as DOCS_CHANNEL, DOCS_SOURCE_REF, and release/workflow identifiers to assemble and label the published site. These values affect documentation output and traceability only. They are not Application or Client configuration and are never read by product bootstrap paths. CI test selectors and Integration-fixture inputs have the same repository-only status: keep them command-scoped and do not project them into Local TRE or a deployed service.

Ownership boundaries

• ahri_tre_config owns document parsing, validation, selection, Effective projection, origin tracking, and fingerprints.
• ahri_tre_secrets owns Secret references, Injected snapshots, encrypted Managed storage, audit integrity, rotation, and recovery checks.
• ahri_tre_runtime retains one immutable trusted or client bootstrap state and supplies narrow Secret capabilities to newly authorized operations.
• ahri_tre_app orchestrates workflows from resolved inputs; adapters own PostgreSQL, Lake, OAuth/OIDC, and other physical dependencies.
• Deployment tooling owns document and Secret projection, PKI, service lifecycle, backups, mount cutover, and rollback. It cannot ask AHRI TRE to export plaintext Secret material or repair a failed store.

Sensitive material handling

Secret material is owned by ahri_tre_secrets. Application configuration may contain canonical logical references and declared Injected versions; Client bootstrap contains no Secrets.

Injected material is snapshotted at startup where required. Managed material is resolved through narrow capabilities, authenticated envelopes, and exact version provenance. A newly authorized operation may resolve its selected Managed reference again; workflow code does not reopen the store.

Public output, logs, diagnostics, protocol envelopes, Session metadata, provenance summaries, and package artifacts must never contain passwords, tokens, authorization codes, private keys, signed credential values, inline connection credentials, credential-bearing paths, or live handles.

Replacement and removal are owner-aware transactions. Active Session and Datastore references are durable evidence. If authoritative inspection is unavailable or commit state is uncertain, the operation fails closed and keeps the recoverable material.

Tests use explicit canary literals inside test modules and assert they are absent from every public representation. Test helpers do not create public credential-field DTOs in production crates.

Governance And Provenance

Governance and provenance are first-class parts of the AHRI_TRE_RS model. They are not presentation-layer concerns and should not be bypassed by the CLI, daemon, examples, or language bindings.

Governance Separation

The governance model separates four concerns:

• authentication: who the caller is
• datastore entry: whether the authenticated caller may open the datastore
• study authorization: whether the caller may read, write, or administer a particular study
• DUO restrictions: what data-use restrictions apply to a study or asset version

DUO metadata informs policy and review, but it does not grant access by itself. Study access grants, custodianship, and explicit policy checks remain separate concepts.

The Authenticated TRE user is the user identity proven by a live Session’s authentication context. Auditable destructive operations record this identity as their actor when it is available. It is not the same thing as the PostgreSQL current user, a DuckLake catalog user, or study custodianship. Authenticated users with study access may perform eligible study-scoped content lifecycle deletes; deleting a study itself remains a study administration action governed by custodianship or administrator capability. Unused semantic catalog deletes require an authenticated actor and dependency-safe targets, and study-scoped references block deletion rather than becoming authorized through custodianship.

Study Custodianship

Study custodianship is the workflow-facing accountability model for study authorization. A study has one primary custodian and may have delegate custodians. Custodians can list visible study access grants and custodian assignments, and can grant or revoke ordinary study access through the application/database workflow path. They should not write raw governance tables directly.

The primary custodian carries ownership accountability for the study. Only the current primary custodian can add delegate custodians, remove delegate custodians, or transfer primary custodianship. Delegates can help administer ordinary access grants, but they cannot change primary ownership or custodian membership.

Adding a delegate custodian automatically grants that delegate study access when the access row is missing. Removing delegate custodianship removes only the study_custodians row; it does not revoke study_access. This keeps administrative responsibility separate from research access. If a former delegate should also lose study access, revoke the access grant explicitly.

Primary transfer is a handoff between custodians. The transfer target must already be an existing delegate custodian, so ownership cannot be transferred directly to an arbitrary user. After transfer, the new custodian is primary, the previous primary remains a delegate, and study access is preserved for both principals.

Study access and delegate targets must be existing datastore principals. The PostgreSQL workflow functions validate that the target is an ORCID-backed login role and that it inherits a datastore-entry group role. This prevents governance metadata from naming users who cannot enter the datastore.

study_access uses an open-access fallback: zero rows for a study means the study is public/open to connected TRE users. One or more rows means the study is restricted to the listed principals. The grant workflow protects the public-to-restricted transition by inserting the current primary custodian and the requested target in the same database operation when the first non-primary access grant is added to an open study. The revoke workflow also prevents the primary custodian from being removed from a still-restricted study, while still allowing the final access row to be removed deliberately to return a study to public/open access.

Governed Querying

Lake queries are not just SQL execution. The app-layer governed query path requires an authorizer before DuckDB statements are prepared. This keeps access decisions visible at the workflow boundary and prevents the lake adapter from becoming a permissive shortcut around metadata policy.

Provenance Model

AHRI_TRE_RS records lineage around data movement and transformation. The current model uses transformation records plus transformation input/output links for workflow provenance.

Provenance appears around operations such as:

• file ingest
• file-to-dataset conversion
• SQL-to-dataset ingest
• dataset and datafile export
• transformation workflows
• archive-first deletion flows as they are implemented

The transform_assets concept should remain an explicit higher-order workflow concept. It should not be collapsed into generic CRUD operations, because the workflow boundary is where governance, inputs, outputs, and review semantics are easiest to preserve.

Transformation Source Provenance

Transformation records can carry source-code provenance in addition to input/output lineage:

• file_path: the workflow source file, script, or notebook responsible for the transformation
• repository_url: the Git repository containing that source when discoverable
• commit_hash: the Git commit used when discoverable

The shared application provenance path enriches missing Git fields before transformation records are persisted. If a transformation has a file_path, the app layer uses it as the source context for Git discovery, reads HEAD, reads the origin remote, normalizes SSH-style remotes to HTTPS-style URLs, and trims a trailing .git. Explicit caller-provided values are preserved and are not overwritten.

This enrichment is intentionally best effort. Non-Git execution environments, packaged examples, missing remotes, or unavailable git commands do not block the workflow; the transformation is recorded with the provenance fields already available.

There is an important Rust-specific nuance. Unlike the Julia git_commit_info helper, Rust async service functions do not reliably expose the top-level caller source location through #[track_caller]. Relying on an async app-service frame would risk recording an internal ahri_tre_app source file instead of the workflow that caused the transformation. The mitigation is to record the top-level workflow source path in file_path when constructing the transformation. The shared app layer then derives repository_url and commit_hash from that path while preserving the workflow-level file_path.

Documentation Status

Some governance and provenance foundations are implemented, but not every public workflow is complete. When a page describes planned CLI, daemon, binding, or example behavior, it should say so directly and link back to the ordered backlog or issue tracker.

Data Contracts

AHRI_TRE_RS uses a small set of canonical data contracts so the Rust core, adapters, control plane, examples, and future language bindings do not invent competing formats.

Canonical Formats

The workspace version baseline for Arrow-family crates, Parquet, DuckDB, and the Rust toolchain is maintained in the root Cargo.toml workspace dependency table and mirrored in docs/tabular_contract.md.

Arrow At Crate Boundaries

Arrow RecordBatch values are the neutral in-memory tabular boundary. The ahri_tre_tabular crate provides reusable helpers for schema-validated Arrow tables, representative typed-column construction, Parquet read/write behavior, and Arrow IPC stream/file byte helpers.

Python, Julia, and R dataframe types are client-local representations. They are not the canonical wire model and should not drive Rust service or adapter interfaces.

Parquet For Persistence

Parquet is the canonical persisted dataset format for lake-managed tabular data and exported dataset artifacts where a columnar binary format is needed. CSV and newline-delimited JSON may exist as export formats for interoperability, but they do not replace Parquet as the primary persisted dataset contract.

Arrow IPC For Binary Transport

Arrow IPC is the binary tabular transport when JSON is not appropriate. It preserves Arrow schemas and batches across process and language boundaries without treating a language-specific dataframe format as the shared contract.

JSON Control Plane

The control-plane direction is JSON over HTTP. JSON should carry commands, status, diagnostics, metadata envelopes, and machine-readable responses. Large analytical tables should use Arrow IPC or persisted datasets rather than being forced through JSON.

The current protocol, daemon, and CLI crates provide the structure for this direction. The implemented CLI schema registry now exposes local JSON Schemas for stable protocol envelopes, shared protocol objects, session, datastore, daemon readiness, domain, study, governance, asset, datafile, dataset, lifecycle/delete, ingest, transformation, workflow, dictionary, tag, semantic model, and bounded CLI-local readiness/lifecycle payloads. Inspect them with ahri-tre schema list --format json and ahri-tre schema get protocol.dataset.catalog.v2 --format json.

These schemas describe JSON control-plane DTOs owned by ahri_tre_protocol or bounded local CLI DTOs. They do not replace Arrow IPC or Parquet for analytical data. Dataset row streams, binary table transfer, and persisted lake/export artifacts should continue to use Arrow IPC and Parquet where those formats are the correct contract.

For substantive TRE workflow operations, the JSON payload ownership model is protocol-first: public request and response bodies belong in ahri_tre_protocol and are carried in stable protocol envelopes. The local daemon is an execution and session adapter over those protocol contracts. It may own local process-control and runtime-only DTOs, but it should not define a second public JSON grammar for reusable workflow results.

TRE Variable Value Types

The PostgreSQL datastore table public.value_types is the source of truth for supported TRE variable value types. Runtime code should resolve these values through the canonical mapping in ahri_tre_types::TreValueType instead of introducing ad hoc ValueTypeId constants in workflow or adapter code.

The six XSD-backed types are scalar variable types. enumeration is for single-response categorical variables represented by a TRE vocabulary, such as REDCap radio, dropdown, yes/no, and true/false fields. multiresponse is for multiple-response categorical variables represented by a TRE vocabulary, such as REDCap checkbox fields.

Boolean and binary are not supported TRE variable value types today. Boolean source fields may be normalized to an existing scalar or categorical TRE type by an ingest workflow, and binary concerns belong to tabular transport or file storage contracts such as Arrow IPC, Parquet, or governed datafiles. They should not be recorded as public.value_types rows unless a future schema change explicitly adds them.

Example Workflows

Example workflow documentation is organized as separate pages so each workflow can describe its own maturity, inputs, authentication behavior, outputs, and validation path without becoming the template for every other workflow.

Use the workflow documentation pattern when adding a new example page. The pattern is intentionally status-first: a page must say whether it describes complete user-facing behavior, provisioning-only behavior, smoke-test-only behavior, or roadmap intent.

Use Configuration And Secrets for product authority. Example helper inputs must remain visibly repository-only and command-scoped; they cannot become runtime configuration or credential fallbacks.

Current And Expected Pages

Page Rules

• Keep each workflow on its own page.
• Link every workflow page to the relevant backlog item, issue, example crate, README, or smoke test.
• Describe command-line usage only for commands that exist.
• If a crate, command, daemon route, or binding is scaffolded but incomplete, label the documented behavior as planned or partial.
• Treat live-service smoke tests as validation evidence, not as ordinary user-facing workflow commands.

Workflow Documentation Pattern

Use this pattern for every example workflow page. The goal is consistency: a reader should be able to compare HDSS, REDCap, future import workflows, and future analysis workflows without guessing which parts are implemented.

Required Header

Start with the workflow name and a short status block.

# <Workflow Name>

Status: <complete | partial | provisioning-only | smoke-test-only | roadmap>
Implementation owner: <crate, example, command, or planned surface>
Backlog context: <ordered backlog item or local issue path>

Choose the narrowest status that is true today:

Purpose

Explain what research or operational job the workflow performs. Keep this section user-facing and brief, but name the AHRI_TRE concepts involved: domain, study, datastore session, metadata store, lake, asset, datafile, dataset, variable, vocabulary, transformation, provenance, governance, or export.

Current Implementation Status

List what is implemented, what is partial, and what remains planned. Link to source material rather than duplicating backlog text.

Recommended links:

• docs/ahri_tre_rust_ordered_backlog.md for product and architecture status
• local issue files under docs/issues/ for recent implementation slices
• example crate README files under examples/
• crate README files under crates/
• focused smoke tests when the behavior is intentionally opt-in

Use direct language such as “implemented in the app layer”, “available only as an example binary”, “validated by an opt-in live smoke test”, or “planned for a future CLI/daemon surface”.

Prerequisites

Document the required runtime context before commands appear.

Typical prerequisites:

• a bootstrapped PostgreSQL metadata datastore
• a Datastore binding with its canonical Lake location available to the runtime
• DuckDB/DuckLake support in the development container or deployment target
• local fixture files, external exports, or source-system credentials
• an existing domain or study ID when the workflow does not create one
• live-service opt-in flags for smoke tests

If a workflow needs datastore schema compatibility, say which datastore schema-status state is required. Prefer datastore schema-plan and datastore schema-migrate for supported metadata-only upgrades; require recreation only when status is unsupported or the workflow needs a future storage-aware migration.

Authentication Modes

State which datastore authentication modes the workflow supports.

Use this checklist when relevant:

• direct PostgreSQL credentials
• interactive OAuth
• stored OAuth artifact
• injected bearer token
• daemon-backed session
• unauthenticated local fixture parsing
• external source-system credentials, such as REDCap API tokens

Do not imply that all modes are equivalent unless the implementation proves it. If a workflow has selected-session behavior, say whether all metadata and lake operations use the selected datastore session.

Inputs

List workflow inputs in a table.

Use exact flag names, logical identifiers, document fields, files, table names, or ID types when they exist. Treat repository-only runner variables as fixture inputs, never product configuration. Use placeholders only for roadmap pages.

Commands

Show commands only for implemented entry points. Prefer copy-pasteable commands that match the repository’s current package names.

cargo run -p <package> -- <flags>

For smoke tests, keep the opt-in variable visible:

RUN_LIVE_<WORKFLOW>_SMOKE=true cargo test -p <package> <test_name> -- --nocapture

If a workflow will eventually be run through the CLI or daemon but that surface is incomplete, place those commands under a “Planned Commands” subsection and label them as planned.

Outputs

Document expected outputs by store and by concept.

Separate ordinary outputs from cleanup artifacts, debug logs, and temporary staging paths.

Validation

Describe how maintainers prove the workflow still works.

Recommended validation sections:

• fast unit or fixture tests that run without live services
• integration tests that need PostgreSQL or DuckLake
• opt-in live smoke tests and their exact repository-only, command-scoped fixture inputs
• expected skip behavior when live credentials are absent
• manual verification queries or file checks when no automated test exists yet

Do not make live smoke tests mandatory for normal documentation builds.

Governance And Provenance Notes

State where governance is checked and what provenance is recorded. If the workflow crosses PostgreSQL metadata and lake storage, explain the observable workflow steps and any compensating cleanup expectations instead of describing the operation as one distributed transaction.

Limitations And Follow-Up

End with known limitations and links to follow-up issues. This section is required for partial, provisioning-only, smoke-test-only, and roadmap pages.

Common limitations to call out:

• a workflow has app-layer support but no dedicated polished CLI command yet
• a workflow can use selected datastore sessions, but has no dedicated daemon workflow route yet
• language bindings are mostly roadmap thin clients over existing contracts
• live external systems require credentials and opt-in tests

HDSS example workflow

An HDSS workflow starts from a configured execution profile and an authorized Session. Infrastructure and credentials are not part of the workflow script.

1. Select the operator-published profile through the authenticated runtime.
2. Create or select the HDSS domain and study by logical identifier.
3. Register governed source assets and immutable versions.
4. Materialize datasets, variables, provenance, entities, and relations through stable protocol requests.
5. Read bounded output through the Session capability.

The CLI validation example demonstrates these protocol-shaped operations with checked-in non-secret data. Deployment-specific PostgreSQL, Lake, OAuth, and Secret details remain in Application configuration and the Secret subsystem.

Importing a REDCap project

Select a Managed Session and authorized Study, then upload project information JSON, metadata JSON and records EAV CSV from the client:

ahri-tre --risk high --study HDSS ingest redcap project \
  --domain REDCap \
  --project-info-path ./project.json \
  --metadata-path ./metadata.json \
  --records-eav-path ./records.csv \
  --dataset-prefix survey --format json

Use repeated --form or --skip-form selectors. --no-register-dictionary materializes typed forms without creating dictionary entries or variable links. Compression/encryption default to enabled. Inputs are read by the client and transferred as three bounded, integrity-checked roles.

Artifacts retain explicit ingest risk; derived forms are High. Inspect form_outcomes for partial results before starting another acquisition. Dataset content still requires separate disclosure admission. Live API acquisition also feeds this workflow.

For a live export, replace the three paths with --api-url and explicitly choose client acquisition/upload or direct Trusted acquisition. Supply a fresh token on nonterminal stdin using the sensitive-input options:

ahri-tre --risk high --study HDSS --acquisition trusted \
  --source-auth redcap --source-material-stdin ingest redcap project \
  --api-url https://redcap.example/api/ --domain REDCap \
  --dataset-prefix survey --form demographics --format json

Use --acquisition client for client-side fetch/upload. No source registration is needed; the endpoint must satisfy deployment outbound policy. Never put the token in command arguments, public request JSON or the endpoint. The runtime discards it after acquisition and requires fresh material next time; there is no service credential fallback. Both paths use the same authorized Study/Domain and writer.

--response-encoding utf8|iso88592, --max-attempts 1..5 and --timeout-secs 1..120 control live exports within the acquisition’s overall budget. Project/metadata bodies are capped at 8 MiB each. Redirects and compressed responses are refused; interrupted writers never replay automatically. The C acquire_https functions accept the same public live intent and separate token.

Control plane, Managed runtime, and CLI

AHRI_TRE exposes stable JSON protocol envelopes through authenticated trust boundaries. Domain semantics belong in ahri_tre_types and ahri_tre_core, workflow orchestration in ahri_tre_app, configuration projection in ahri_tre_config, and physical dependencies behind adapter crates.

Web

The Web service selects one Application document at startup and receives an immutable Effective configuration plus narrow Secret capabilities. It owns browser HTTP sessions and the documented metadata/access-request surface. It does not accept runtime topology or credentials from requests or environment.

Trusted and Managed runtimes

The Trusted runtime bootstraps all service dependencies from Application configuration. The user-side Managed runtime selects one Client bootstrap and forwards protocol requests over authenticated HTTPS. It cannot discover an alternate endpoint, start a local Trusted runtime, or execute workflows locally.

Named Session metadata is safe, durable journal state. Live PostgreSQL and Lake handles remain process-local. After process loss, a persisted open record is closed and its owner-bound Managed-secret references are released through the authenticated internal reconciliation path; it is never reopened from a cache or plaintext recipe.

CLI and C ABI

The CLI and C ABI are thin clients over the same protocol. Their public output uses protocol DTOs and safe diagnostics. Arrow IPC carries binary tabular data, and Parquet is the persisted dataset format. Credentials, signed URLs, Restricted local references, and runtime handles stay outside public schemas.

The CLI schema registry remains available through schema list and schema get. Product startup rejects retired environment names before operational work. Repository-only DEV_ENV_ settings and isolated Integration fixtures are not product configuration.

Web Control Plane

The web control plane is the deployed, network-facing HTTP service for TRE Browser. It is distinct from the local daemon and from the separate TypeScript frontend that consumes its JSON contract. This Rust workspace owns the service, its protocol DTOs, and the versioned fixtures; it does not contain the frontend product.

Service Boundary

The service exposes a deliberately bounded browser surface:

• ORCID/OIDC login and a server-side browser session;
• datastore, study, resource, dictionary, and semantic/provenance discovery metadata, including bounded catalogue search across those discovery families;
• study access-request submission, requester status and withdrawal, and the custodian queue, rejection, and approval actions;
• /health, /ready, /diagnostics, /version, and a limited /protocol diagnostic surface.

The browser does not receive a daemon Session, raw database access, dataset rows, datafile bytes, live runtime handles, or lake access. Discovery is metadata-only. An approved request is a governed datastore workflow entry; it is not asset-level access control and does not itself expose protected content.

The local daemon and CLI remain the interfaces for local live-session and automation workflow execution. The C ABI remains the lower-level interface for language bindings. All three faces use the same domain and application boundaries, but they have different callers, authority, and operational scope. Every substantive Web route crosses the authenticated stable-protocol boundary to the Trusted runtime; the Web process does not open PostgreSQL or Lake capabilities.

Identity And Authority

Five identity tiers must not be conflated:

The Trusted runtime alone resolves and uses the read-only browser and narrow access-request datastore authorities. The invoking Web service identity is recorded separately from the represented visitor. Custodian approval runs the governed study-access workflow through the matching live Session and remains auditable under the Authenticated TRE user.

Browser Contract

The substantive routes are resource routes, not a browser wrapper around the local daemon protocol. Their representative, synthetic responses are versioned under docs/contracts/web-control-plane/v2. The fixture manifest and stable IDs are the frontend handoff for the separate web-application repository.

The companion web application specification and handoff maps supported browser journeys to these routes, specifies the same-origin session integration, and gives the receiving repository fixture and release expectations without prescribing its frontend stack.

Network responses provide X-Request-Id and X-Protocol-Version for support and protocol correlation. POST /protocol is limited to daemon version and doctor diagnostics; it is not a second grammar for discovery or access requests. Public error responses use the documented stable error shape and do not reveal authorization details.

The authenticated Domain collection route supports server-owned filtering with optional text and text_mode parameters plus repeatable tag labels and tag_id public references. Text matches the browser-safe name, description, or URI using exact, prefix, or contains mode. Text and tags combine as text AND any tag; filtering occurs before deterministic ordering and paging. The response returns normalized active filters, page count, has_more, and cursors bound to the selected Datastore, Domain collection, filters, limit, and protocol version. No client-side partial-page filtering is required.

Catalogue Search

Catalogue search is a Web v2 discovery capability. It lets an ORCID-authenticated browser visitor submit structured, protocol-owned search queries to one selected ready datastore; it is not a daemon search API, a federated/global catalogue, or an unbounded client-side filter over a downloaded catalogue.

Each route accepts the corresponding stable protocol search request as JSON. Study search uses query.text and the nested facets.any_tags and facets.any_domains groups; the other families preserve their noun-specific predicates. This keeps text modes, selectors, validation, page bounds, and deterministic ordering protocol-owned. Empty queries or predicate sets are rejected. The existing unfiltered discovery routes remain the way to browse an entire available collection. Where a reused request DTO has a daemon Session field, a client-supplied value is rejected: the browser session and server-held Discovery authority are the only Web identity and authority inputs.

Search is scoped to the path’s selected ready Datastore and uses server-held, read-only Discovery authority. It requires an ORCID-authenticated browser session, but its visibility is the broad TRE Browser Discovery view: Study access grants, TRE OAuth-group membership, custodianship, and access-request state neither broaden nor narrow results. A search result must be eligible for the equivalent browser Discovery projection, so search cannot bypass the ordinary route’s visibility or redaction decisions.

Results, Pagination, And Privacy

Every successful response identifies the selected datastore and search family, returns the normalized active query or predicates, a deterministic ordering identifier and description, the returned-page count, requested limit, has_more, and an optional opaque next_cursor. It does not provide an exact total count. An empty result is a successful 200 with an empty result array, zero returned records, has_more: false, no next cursor, and a neutral server-provided empty state.

Treat a returned cursor as opaque and use it only with the same search family, datastore, normalized query or predicates, page limit, and protocol version. Do not decode, manufacture, modify, silently substitute, or persist it beyond the active pagination journey. A malformed or mismatched cursor is a 400 validation error; the caller must handle that explicitly before restarting pagination.

Results remain metadata-only. Resource versions expose authoritative available or withdrawn lifecycle state. Datafile resource and search records may expose safe creation dates, stored size, format, and compression/encryption booleans. They exclude daemon session state, protected rows or bytes, previews and downloads, dataset row counts, datafile digests, Content-derived summaries, storage paths, credentials, raw SQL, diagnostics, membership lists, and custodian internals. Operational logging retains only correlation and bounded operational data—such as request ID, search family, datastore identity, status, duration, page count, and control outcome—not bodies, predicates, search text, cursors, result identifiers, cookies, ORCID identity, or authority details.

Failure And Resource Boundaries

Search requests are JSON bodies capped at 64 KiB. A request has at most 20 query terms or predicates and each text value is at most 256 characters; the family-specific page bounds remain protocol-owned. The default controls are 300 searches per minute per browser session with burst capacity 30, six concurrent searches, and a ten-second datastore timeout; an operator may tighten these controls without changing protocol limits.

Search requests may send the protocol version accepted from /version in X-Protocol-Version; omission uses the current supported version. All responses are Cache-Control: no-store and continue to provide X-Request-Id and X-Protocol-Version for safe support correlation.

The Web control-plane v2 contract is the authoritative payload and fixture index. Its companion web-application handoff defines frontend consumption, fixture-driven tests, and retry presentation. The protocol 2.0.0 cutover replaces the prior reference encoding and fixture IDs. The companion must adopt web-control-plane.v2; a v1 image is incompatible. The existing HTTP route structure and authority classes are preserved.

Domain Browsing

Domain browsing is the datastore-scoped discovery journey for moving from a Domain to its associated Studies, canonical Variables, and their Vocabulary metadata. In the Web control-plane v2 contract, every reference returned by a route is a stable, opaque public reference to use in the next route. It is not a daemon Session API and it does not expose Dataset or Datafile content.

All Domain-browser routes start with a selected ready datastore and require an ORCID-authenticated browser visitor. The Trusted runtime resolves that datastore through its Runtime-held discovery authority before reading metadata. An ORCID-authenticated browser visitor does not send a datastore credential, OIDC token, role, or other authority.

The public projection is deliberately narrow. Domain rows contain only their descriptive facts and tags. Domain Study rows contain shared browser-safe Study facts, not access-request actions or search-match evidence. Domain Variable rows are canonical definitions—not Dataset-version membership—and expose name, value type, description, and an optional Vocabulary link. Variable detail can add value format, tags, and ontology identifiers, but never Dataset membership, key role, note, or Dataset row role.

Collection Traversal

The Domain list, Domain Study list, Domain Variable list, Vocabulary item list, and Vocabulary mapping list are independently paginated collections. Each response has a returned count, requested limit, stable ordering identifier and description, optional opaque previous_cursor and next_cursor, and an optional neutral empty_state. The contract intentionally does not publish an exact total count.

Use limit and a returned cursor only on the same collection route. The default limit is 100 and the maximum is 500; zero, oversized, malformed, or otherwise invalid page parameters produce a public validation error. A returned cursor is bound to its datastore, route family, parent references, limit, ordering, direction, boundary, and protocol version. It must not be edited, decoded, or replayed for a different Domain, Vocabulary, collection direction, or page size.

The first page has no Previous cursor; the terminal page has no Next cursor; an empty collection has neither. An empty collection is a successful 200 response with an empty item array, returned count zero, and a stable neutral empty-state code and message. A collection URL with its returned cursor and unchanged limit is bookmarkable while the catalogue remains unchanged, but it is not a permanent snapshot. If the catalogue changes, the service returns 409 with browser_domain_cursor_stale; restart only that collection from its first page. Keep the Study and Variable cursors independent of each other and of the Domain list, so moving between these tables never changes another table’s traversal state.

Vocabulary Categories And Mappings

A Vocabulary category uses product-facing fields: integer code, string label, and optional definition. The projection is explicit: stored integer value becomes public code, stored string code becomes public label, and optional stored description becomes public definition. The storage field names are not alternative API fields. Categories are ordered by code then stable item identity and contain no observed frequencies, counts, percentages, distributions, or other Content-derived summaries.

Mappings retain their stored source and target orientation. A mapping row includes its own stable reference and, at each endpoint, the Vocabulary reference, item reference, integer code, and label. It does not duplicate item definitions. The selected Vocabulary can therefore appear as the source or the target, and the opposite Vocabulary can belong to a different Domain. This is not a claim that the mapping is symmetric. Mapping ordering is stable across source Vocabulary and item, target Vocabulary and item, then mapping identity.

Authority, Privacy, And Failure Boundaries

Domain browsing is broad Discovery metadata. It is independent of Study access grants, custodianship, TRE OAuth-group membership, and access-request state; those facts do not broaden or narrow a result. The service applies permission trimming before public projection and uses neutral not-found responses for unknown, cross-parent, unauthorized, or trimmed objects so the route cannot be used to probe hidden metadata.

Responses are metadata-only and must not expose protected rows or bytes, previews, downloads, Dataset memberships, Content-derived summaries, storage or Restricted local references, credentials, raw SQL, adapter detail, runtime handles, private governance data, Study membership, custodianship, or access-request state. They are Cache-Control: no-store. Operational logs retain only request ID, route family, datastore identity, response status, duration, returned-page count, requested limit, and limit outcome—not object references or names, cursors, bodies, cookies, ORCID identity, tags, ontology values, or category or mapping content.

Domain-browser callers may send the protocol version accepted from /version in X-Protocol-Version; omission uses the current supported version. A malformed or unsupported value fails safely. The operational defaults are 300 reads per minute per authenticated session with a burst capacity of 30, six concurrent reads, and a ten-second datastore timeout; deployments may tighten those values without changing protocol page maxima.

Separate Discovery Journeys

Domain Study browsing is not Study search. It has no search predicate, search-specific result evidence, aggregate search result, or access-request action; it simply follows the Domain–Study association and returns the shared safe Study facts. Its HTTP route has no runtime dependency on the Study search route.

Likewise, Domain Variables are canonical Domain-owned definitions, not the Dataset-scoped dictionary. The dictionary journey is GET /browser/datastores/{datastore_id}/studies/{study_id}/datasets/{dataset_version_id}/dictionary and describes its owning Dataset version. Datafiles do not own dictionaries. The legacy Datafile-shaped URL remains recognized as an explicit obsolete-route error and returns 410 / browser_datafile_dictionary_deprecated; new navigation must not offer it. Neither journey calls or depends on the other’s HTTP route. A Variable reused by Dataset versions or Studies is still represented once through its owning Domain rather than by a reverse Dataset or Study usage listing.

Deployment And Operations

Deploy the frontend and service at the same HTTPS origin. Browser session cookies are server-side, HttpOnly, Secure, and SameSite=Lax or Strict. The short-lived login-state cookie always uses SameSite=Lax so it can return on the external provider’s top-level GET redirect. Both callback routes require that cookie to match the pending transaction before consuming state or exchanging a code, and clear it on success. With Strict session cookies, the frontend landing page loads first and then fetches /session from the same origin; the callback itself does not need a session cookie.

Pending logins are bounded by services.web.max_pending_logins (default 1,024). Each new login reclaims expired records before admission; live transactions remain usable. Capacity rejection returns 503 orcid_login_capacity_exceeded with Retry-After: 60 and no cookie or redirect. Honour the delay before a fresh attempt. The deployment security guide details expiry, idle retention, and configuration limits.

Unsafe requests require a matching Origin; cross-origin credentialed CORS is not enabled. The complete operator policy is in the web control-plane deployment security guide. The datastore deployment guide describes the reverse-proxy and Runtime topology, identity and authority split, configuration categories, verification, and current session-store constraint.

The standalone service starts only with explicit confidential-client OIDC configuration: issuer, client ID, externally visible callback URI, scopes, cookie policy, and a server-side client-secret file reference. It exchanges the callback code and keeps token material in the service process. The callback is the same-origin /auth/orcid/callback URL; production deployments require HTTPS and secure cookies. The frontend never receives OIDC tokens or the secret.

The diagnostics operator guide describes request correlation, the ten-second Runtime check, shared admission limits, evidence bases and Local TRE log commands. /diagnostics retains HTTP 200 even when a returned check fails; /ready returns 200 or 503 for its check result. Admission can reject either route. Neither surface exposes owner-Session findings.

/health is a process-liveness check. /ready authenticates to the Trusted runtime and verifies the required stable-protocol capabilities, returning failure when that sole substantive path is unavailable or incomplete. It does not probe PostgreSQL and has no direct fallback. /diagnostics exposes only stable check names and status, redacting credentials, runtime references, Restricted local references, Lake locations, and protected content.

Scope And Follow-up

This MVP is a browser discovery and access-request surface, not a general remote control plane or a content viewer. It intentionally excludes cross-origin browser deployment, frontend framework choices, Rust/Wasm, dataset/datafile content delivery, and asset-level grants. The frontend repository specification and release handoff are tracked in issue 20.

Rust CLI reference

The installed CLI is a configuration-driven protocol client. Use ahri-tre --help and ahri-tre <group> --help as the exact grammar.

Local commands

• version, doctor, schema, and completion inspect the installed client.
• config initializes, validates, projects, preflights, and renders versioned Application or Client documents.
• daemon controls the user-side Managed runtime selected by Client bootstrap.

Trusted administration

• secrets performs audited Managed-secret administration.
• datastore create and credential rotation use the protected Trusted-runtime administration channel and configured logical identifiers.

Authenticated workflow commands

session, datastore, domain, study, asset, datafile, dataset, variable, vocabulary, tag, entity, entity-relation, transformation, and ingest operate through an authorized configured Session.

Removed environment profiles, direct database opens, cached-token opens, discovery overrides, and Lake mutation commands are not hidden compatibility features. Supplying a retired environment name is fatal.

CLI manual

Governed content control documents are available through ahri-tre schema get protocol.content-transfer.v1. This schema describes admission requests, immutable input identities, exact effective budgets, deployment limits, and terminal evidence. Content is delivered incrementally; admission success alone does not establish completion. See the content transfer contract for destination and C payload lifetime semantics.

For an explicit Dataset destination:

ahri-tre --session analysis --study demo dataset data \
  --dataset visits --format arrow --to visits.arrow

Direct queries declare each input’s owning Study or immutable reference; they do not accept the global --study selector:

ahri-tre --session analysis query \
  --inputs '[{"alias":"visits","asset":{"kind":"name","study":{"kind":"name","name":"demo"},"name":"visits","asset_type":"dataset"}}]' \
  --sql 'SELECT * FROM visits LIMIT 100' --format parquet --to sample.parquet

The shared request budgets are --max-payload-bytes, --max-decoded-file-bytes, --max-rows and --max-transfer-seconds. Omitted values use deployment defaults. Explicit positive values retain their exact value within the advertised ceiling; invalid values fail before work. --limit or SQL LIMIT selects rows independently of these service budgets. Dataset commands accept --version; a complete Asset-version reference also pins content. Datafile export supports --no-decrypt, --no-decompress and logical-byte recompression with --compress.

Use --to - for payload-only stdout. For file destinations, existing files are preserved unless --overwrite is explicit, and even then replacement follows verified completion. An error on stdout can leave partial bytes and returns a nonzero exit status. Disclosure identity and notices go to stderr.

All commands accept --config PATH for explicit document selection and most reporting commands accept --format text|json. Explicit selection is strict.

Top-level command groups are:

• version, doctor, schema, and completion;
• config for schema, initialization, validation, Effective rendering, preflight, and Client rendering;
• secrets for privileged Managed-secret administration;
• daemon for the Managed runtime lifecycle;
• session and datastore for authenticated protocol operations;
• governed metadata and data groups such as domain, study, asset, datafile, dataset, variable, vocabulary, tag, entity, entity-relation, transformation, and ingest.

Use ahri-tre <group> --help for the exact installed grammar. Removed direct database profiles, dotenv files, endpoint overrides, cached-token opens, passfiles, Lake moves, and adoption/reset commands are intentionally absent.

Detached Dataset materialization

ingest dataset from-datafile waits for completion by default. To receive an operation ID as soon as the server accepts the work:

ahri-tre --session analysis --study StudyA ingest dataset from-datafile \
  --domain HDSS --dataset observations --source-asset source_csv \
  --description "Materialize observations" --no-wait --output-format json
ahri-tre --session analysis operation list --scope datastore --status completed --limit 25 --format json
ahri-tre --session analysis operation get OPERATION_ID --event-limit 25 --format json
ahri-tre --session analysis operation cancel OPERATION_ID --format json
ahri-tre --session analysis operation result get OPERATION_ID --format json

Acceptance JSON contains kind: "operation"; the normal successful wait returns kind: "completed" with the materialization result. Text detach output names the accepted operation. Interrupting the CLI wait leaves accepted work running while its original Session stays available. Failed or cancelled waits exit nonzero; successful inspection of a failed operation exits zero. Current authorization is required for each inspection, including from an explicitly reopened Session.

--source-asset accepts the returned canonical Asset reference or immutable version reference as JSON, as well as a Datafile name. Source and Output must belong to the same resolved Study. A separate --source-version must agree with a pinned reference. Study/Domain names and references can be mixed; equivalent selectors retain the same operation for the same idempotency key.

Existing governed Datafiles can be parsed as CSV, JSON, Arrow IPC, Parquet or XLSX, with the existing parser options. An explicit --format must agree with the Datafile’s declared format. This parser support does not extend the current CSV-only client upload transport; broader acquisition is tracked separately. The derived Dataset remains High and uses the existing version allocation.

This producer requires configured bounded execution and independent lifecycle metadata access. Configured runtimes advertise the four implemented operation commands.

Cancellation returns the current operation summary. cancel_requested means the request was accepted; the operation may still be waiting for an adapter call or cleanup. Inspect it until it becomes cancelled, completed, or failed. Repeated pending cancellation returns the same state. Terminal operations and final admission return conflict. Cancellation requires current owner, Session, Datastore, and resource authority; an invisible ID behaves like an absent ID.

Operation history requires explicit --scope datastore or --scope session. Session scope currently returns an empty collection. Datastore scope lists only the current owner’s authorized work, newest first, with descending operation ID breaking creation-time ties. Repeat --status to include several states; --operation-kind ingest.dataset.from_datafile, --created-after, and --created-before narrow history. Time bounds are exclusive RFC 3339 timestamps.

List --limit and get --event-limit accept 1–500 (default 100). Continue list pages with --cursor and event pages with --event-cursor, copying the returned next_cursor and retaining the query context and filters. Events sort by ascending sequence. New operations do not enter an existing list traversal; statuses and authorization are checked on every page. Cursors are opaque and expire when the Trusted runtime restarts; start a fresh traversal then.

Progress reports only optional stages. A terminal operation does not establish Dataset availability for another write: retained output reservations may still require cleanup. History retention is enforced on every read.

Operation retention

Terminal summaries, events and typed receipts expire exactly 30 days after finished_at; active work has no deadline. Get, event and result reads return not-found at operation expiry, and history omits expired rows before pagination and cursor lookahead, even when physical cleanup is delayed. Completed status is preserved if its result expires early, is removed, or becomes unavailable. The summary’s result reference and operation_result_unavailable detail distinguish expired, removed and unavailable; current source and output authority still apply. Successful result receipts include retention and availability alongside kind and data. Status and result text output display these protocol fields.

Idempotency protection lasts while active and until both acceptance plus 24 hours and the terminal operation deadline have passed. If public history expires inside that minimum key window, a retry returns not-found without executing new work. Expired keys may be reused only after the normal output-reservation checks. Startup and explicit Session opening perform bounded metadata housekeeping, at most 100 expired operations per pass. Housekeeping never releases reservations or deletes private cleanup evidence, Datafiles, Dataset versions, Transformation provenance or audit records. No public prune command or retention setting exists.

CLI Option Values

This page lists command option values that are selected from a bounded set. Free-form names, paths, URLs, reasons, SQL text, timestamps, and environment variable names are documented in the CLI Manual instead.

Control Output Format Values

Used by --format on most control-plane commands and by --output-format when --format is already used for a data or source format.

Dataset Data Format Values

Used by dataset data --format and dataset export --format.

Dataset Table Format Values

Used by ingest dataset table --format and ingest dataset from-datafile --format.

JSON Data Format Values

Used by dataset table parsing option --json-format.

Boolean Option Values

Used by explicit boolean-valued options such as dataset table parsing option --header.

Asset Type Values

Used by asset selectors such as asset --type and tag selector --asset-type.

PostgreSQL SSL Mode Values

Used by datastore maintenance option --sslmode. The value is passed through to PostgreSQL/libpq connection handling.

Tag Target Values

Used by tag get --target and tag set --target.

Protocol automation can also target semantic domain, study, variable, entity, and entity-relation tag targets by their public refs. Data catalog tag targets support asset, datafile, and dataset refs on the implemented automation and daemon-backed protocol paths; missing refs fail as not found or validation errors rather than unsupported selector errors.

Shell Values

Used by completion <shell>.

Study Type Values

Used by study add --study-type. The CLI accepts either the integer id or the symbolic name and normalizes the value to the integer id before sending a stable protocol request. Stable protocol and app-layer requests still use the integer id. These values come from the seeded study_types reference table.

Variable Value Type Values

Used by variable add --value-type and variable update --value-type. These values come from the seeded value_types reference table.

Variable Keyrole Values

Used by variable add --keyrole and variable update --keyrole.

SQL Source Flavour Values

Used by ingest dataset from-sql --flavour.

Remote sources require explicit --acquisition client or trusted and typed credentials through --source-auth postgresql/mssql with --source-material-stdin, or an entitled --shared-source for Trusted acquisition. Credentials and connection-string options are forbidden in the endpoint. See the SQL source contract.

Datafile Format Values

Used by ingest datafile --format. This option stores an EDAM format value for the managed datafile. It is not a closed enum: explicit EDAM identifiers or MIME types are accepted. The CLI also normalizes the common shortcuts below.

Asset Version Selector Values

Used by ordinary asset, datafile, and dataset version selectors. When these options are optional and omitted, commands normally select the latest version.

Delete Version Selector Values

Used by datafile delete --version and dataset delete --version.

Dataset Withdrawal Version Values

Used by dataset withdraw --version.

Rust CLI Command Spec

The canonical Rust CLI command implementation contract is maintained at docs/ahri_tre_cli_command_spec.md.

That document supersedes the earlier Julia-reference CLI notes. The Julia CLI implementation is not planned.

Binding Contracts And Roadmap

AHRI_TRE_RS is designed so language bindings stay thin. Bindings should expose the application services through stable interface contracts; they should not create a second domain model, a second persistence format, or a second control plane for each client language.

ADR-0005 sets the repository boundary for those bindings. Full Python, Julia, and R package implementations live in separate repositories:

This Rust workspace owns the public protocol, stable C ABI, Client bootstrap runtime, runtime artifacts, package manifests, compatibility fixtures, and wrapper-author contract. The language repositories own package code, language-specific CI, publication, documentation, dataframe ergonomics, and ongoing host-language issue tracking. The shared Binding Repository Seed Contract is the handoff contract each external repository consumes.

Shared Boundary Model

The Rust workspace treats Arrow as the neutral tabular boundary. Python, Julia, and R users may naturally work with pandas, Polars, Julia DataFrame, Arrow.jl tables, tibbles, or R data.frame values, but those are binding-local representations. They are adapters around the shared Arrow, Parquet, Arrow IPC, and JSON contracts rather than canonical wire or storage models.

This keeps the Rust service independent of any single client dataframe library and lets each binding choose idiomatic conversion points without changing the core workflow semantics.

Implemented Foundation

The binding foundation currently exists below the language wrapper layer:

• ahri_tre_tabular owns reusable Arrow RecordBatch validation, Parquet read/write helpers, Arrow IPC helpers, and AHRI_TRE Arrow metadata handling.
• ahri_tre_app owns workflow-facing services that bindings should call through a stable lower-level interface instead of reimplementing workflows.
• ahri_tre_protocol, ahri_tre_daemon, and ahri_tre_cli provide the local JSON/control-plane direction over app workflows. The final JSON-over-HTTP service remains roadmap work.
• ahri_tre_ffi_c exposes the stable C protocol-adapter ABI: startup introspection, immutable Client bootstrap selection, opaque client/session/result handles, authenticated Trusted-runtime protocol JSON execution, response JSON access, optional attached payload access, and matching cleanup functions. It intentionally exposes no local daemon lifecycle or transport overrides and is not one C function per workflow.
• ahri_tre_python, ahri_tre_julia, and ahri_tre_r are scaffold crates retained only as transitional workspace placeholders and compatibility shims. Their exported Rust functions are not public language APIs and should not grow into package implementations in this repository.

C ABI First

The stable C ABI is the implemented language-neutral low-level interface. It defines:

• opaque handles for sessions and result objects
• explicit memory ownership and cleanup rules
• stable error and diagnostic envelopes
• JSON control-plane payload exchange for commands and metadata
• Arrow IPC or file-oriented handles for tabular payloads
• smoke tests proving the ABI is usable outside Rust

The C ABI is stable enough for Python, Julia, and R wrappers to share. The wrappers can focus on idiomatic packaging and dataframe conversion rather than each binding discovering its own unsafe Rust boundary.

Wrapper and C ABI compatibility checks should use Protocol version compatibility, not the Rust crate version, Rust package version, C ABI library identity, or host-language package version. Local installations expose that contract through ahri-tre version --format json: the JSON version field identifies the CLI executable package, C ABI startup introspection identifies the loaded shared library and ABI surface, each language package has its own ecosystem version, and protocol.current plus protocol.compatibility identify the public control-plane contract the executable speaks.

Wrapper authors can inspect the same JSON payload contract through ahri-tre schema list --format json and ahri-tre schema get <schema-id> --format json. The protocol schema IDs, such as protocol.response-envelope.v2, protocol.dataset.catalog.v2, and protocol.semantic-model.catalog.v2, are the binding-facing DTO contract for control-plane responses. They are useful for generated typed result objects, contract fixtures, and cross-language smoke tests, while Arrow IPC and Parquet remain the data-plane contracts for tabular payloads.

Backlog: P1.21 Build stable C ABI.

Language Wrapper Roadmap

The first useful Rust-workspace outcome for each host language is a seed and contract handoff: artifact discovery, protocol compatibility checks, managed runtime lifecycle coverage, generic protocol JSON execution, safe diagnostics, payload handling, and smoke fixtures. Full host-language APIs, dataframe conversion matrices, package-manager checks, documentation sites, CI matrices, and publication workflows belong in the external binding repositories.

C ABI Wrapper Author Contract

The stable C ABI is the layer Python, Julia, R, and other wrappers should bind to before exposing idiomatic host-language APIs. Wrapper startup should inspect the loaded library with ahri_tre_abi_version() and ahri_tre_library_version(), then decide behavioral compatibility from ahri_tre_protocol_version(), ahri_tre_protocol_compatibility_minimum(), ahri_tre_protocol_compatibility_maximum(), and ahri_tre_protocol_compatibility_rule(). Wrappers should fail fast when their required protocol support is outside that range; package and crate versions are support metadata, not the protocol compatibility contract.

The ABI owns unsafe-boundary lifecycle only. Client handles wrap adapter state, session handles wrap selected session context, and result handles own protocol response JSON plus any attached payload descriptors or bytes. Borrowed response JSON, payload descriptor strings, and borrowed byte views stay valid only until the owning result handle is freed. Owned string and byte copies must be released with ahri_tre_string_free() and ahri_tre_bytes_free() respectively. Daemon-backed data-plane outputs use those result-owned payload accessors for Arrow IPC bytes and descriptor-only export artifacts; runtime target paths and lake internals are not wrapper-facing data.

Cleanup functions accept NULL as a no-op. Double-free, freeing foreign pointers, freeing with the wrong function, freeing copied bytes with the wrong length, and concurrent mutation or cleanup of the same handle are caller errors. First-stable same-handle concurrent use is not thread-safe unless a future ABI symbol explicitly documents otherwise.

Wrapper construction accepts only optional Client bootstrap selection. A wrapper passes an explicit projected document through client_bootstrap_path or leaves it NULL for /etc/ahri-tre/client.toml. Construction validates Deployment identity, Trusted-runtime origin, and TLS trust before returning the opaque handle. Profiles are server-discovered and selected by logical identifier. Explicit selection failures do not fall back. Wrappers must not reintroduce endpoint, local daemon, binary override, never-start, or auto-start configuration above ABI v2.

Workflow semantics stay in public protocol envelopes. C callers submit serialized protocol request JSON with ahri_tre_client_execute_protocol_json() or ahri_tre_session_execute_protocol_json() and read the returned protocol response JSON from the result handle. Python, Julia, and R packages may add typed convenience functions above that generic layer, but those functions should generate protocol requests and parse protocol responses rather than depending on a workflow-specific C ABI.

New TRE workflow semantics must land in ahri_tre_app and ahri_tre_protocol first. A language wrapper may expose a convenient host API only after the behavior is available through the shared protocol contract.

The dedicated book chapter Stable C ABI explains the ABI’s functioning and wrapper-author contract in more detail. The generated header and minimal C examples live in crates/ahri_tre_ffi_c.

External Python, Julia, and R repositories must also follow the shared Binding Repository Seed Contract. That contract is the discoverable handoff for ADR-0005: it defines the runtime artifact layout, manifest discovery rules, protocol compatibility checks, Client bootstrap policy, C ABI ownership rules, redaction expectations, payload handling, development overrides, and representative smoke scenarios that every language repository should share.

Backlog:

• P2.8 Python binding repository handoff
• P2.9 Julia binding repository handoff
• P2.10 R binding repository handoff
• P2.11 Build and test release matrix

Binding Rules

Bindings should preserve the same architectural boundaries as the Rust workspace:

• domain types remain free of runtime database, lake, or OAuth handles
• application workflows remain the semantic source of truth
• PostgreSQL metadata access remains behind the libpq-based adapter
• DuckDB, DuckLake, and Lake location logic remain behind the lake adapter
• OAuth/OIDC behavior stays separate from libpq wiring
• JSON control-plane responses remain machine-readable and stable
• dataframe conversion happens at the edge of each language binding

These rules are especially important while the binding crates are still scaffolds. A thin wrapper is allowed to make AHRI_TRE convenient in a host language; it is not allowed to bypass governance, provenance, datastore session, or lake semantics.

Protocol 2 cutover

Current source clients require protocol 2.0.0 (minimum and maximum), independently of C ABI 2 and package 0.11.0. Match a source-built library and CLI by revision and artifact digest; old released runtime artifacts do not acquire v2 support through a package-version comparison. Read streaming results through the verified empty completion using the existing content-transfer API.

The Rust Python/R/Julia crates and bindings/ folders are placeholders. ADR-0005 seed directories under tmp/ are absent in this checkout; no new host-language wrapper or native artifact publication is claimed. The implemented Python ctypes qualification consumer uses the exported C functions and protocol introspection. External packages must adopt these same references, Session semantics and stream ownership before declaring support for a v2 runtime artifact.

Binding Repository Seed Contract

This contract is the shared handoff surface for external AHRI TRE language binding repositories. It applies to the Python, Julia, and R repositories created from ADR-0005:

The language repositories own host-language package code, dependency lockfiles, CI, publication, dataframe conversion, and documentation. This Rust workspace owns the runtime artifacts, public protocol, stable C ABI, managed local runtime behavior, package manifests, fixtures, and compatibility contract that those repositories consume.

Ordinary package installation must consume prebuilt AHRI TRE runtime artifacts. It must not require Cargo, Zig, Rust toolchains, PostgreSQL development headers, this repository’s target directory, or this repository’s dev container. Developer overrides may point at a staged package or local checkout, but those overrides are for wrapper development only.

Required Runtime Artifacts

Every binding repository must be able to consume a staged or released AHRI TRE runtime package with this layout:

ahri-tre-<version>-<target>/
  bin/
    ahri-tre
    ahri-tred
  include/
    ahri_tre_ffi_c.h
  lib/
    libahri_tre_ffi_c.so or libahri_tre_ffi_c.dylib
    libduckdb.so or libduckdb.dylib
    <other bundled runtime libraries>
  share/ahri-tre/
    manifest.json
    installer.json
    README.md

The package manifest is the machine-readable contract for wrapper runtime discovery. Binding repositories should discover package-relative paths from share/ahri-tre/manifest.json and installer metadata rather than assuming Cargo target paths, repository-relative paths, or global PATH entries.

The required wrapper-facing artifacts are:

Package-relative paths in manifests and installer metadata must not expose Restricted local references. Binding packages may copy, bundle, or depend on these runtime artifacts according to their language ecosystem, but they should keep the same artifact names and discovery semantics.

Protocol Compatibility

Protocol compatibility is the behavioral compatibility boundary. Binding package versions do not need to move in lockstep with the Rust workspace, runtime package version, CLI version, or C ABI crate version.

At startup, a binding must load the C ABI library and inspect:

• ahri_tre_abi_version()
• ahri_tre_library_version()
• ahri_tre_protocol_version()
• ahri_tre_protocol_compatibility_minimum()
• ahri_tre_protocol_compatibility_maximum()
• ahri_tre_protocol_compatibility_rule()

Each binding release must declare:

• the AHRI TRE protocol version range it supports
• the runtime artifact version it bundles or expects
• the C ABI surface version it was written against

The binding should fail fast when the loaded runtime’s protocol range does not cover the binding’s required public protocol behavior. Library package versions and language package versions are support and provenance metadata; they are not substitutes for protocol compatibility checks.

Host-language convenience functions must generate public protocol request JSON and parse public protocol response JSON. New datastore, governance, provenance, lake, or workflow semantics must be added to the app and protocol layers in this Rust workspace before a binding exposes them.

Client Bootstrap And Trusted Runtime

A binding exposes optional explicit Client bootstrap selection at construction. Omission selects /etc/ahri-tre/client.toml; an explicit missing or invalid document fails without fallback. The C ABI validates Deployment identity, Trusted-runtime HTTPS origin, and TLS trust, then retains the immutable transport capability in its opaque client and session handles. Profiles are server-discovered and selected by logical identifier.

Binding repositories must not recreate the removed local runtime surface. They do not expose endpoint-only or never-start modes, daemon binary overrides, daemon discovery, local auto-start, or runtime lifecycle helpers. Protocol operations and profile selection go through the authenticated Trusted runtime described by the selected Client bootstrap.

C ABI Ownership And Concurrency

The C ABI owns unsafe-boundary memory and handle rules. Each binding must wrap those rules in idiomatic host-language objects without weakening them.

Borrowed JSON strings, payload descriptor strings, and borrowed payload bytes are valid only until the owning result handle is freed. Bindings must copy data before releasing the result when host-language objects need to outlive the call scope.

Cleanup functions accept NULL as a no-op. Double-free, freeing foreign pointers, freeing with the wrong function, freeing copied bytes with the wrong length, and concurrent mutation or cleanup of the same handle are caller errors. First-stable same-handle concurrent use is not thread-safe unless a future ABI symbol explicitly documents otherwise. Different handles may be used concurrently only within the guarantees of the daemon and selected-session layer.

Host-language finalizers should be defensive cleanup aids, not the primary runtime ownership model. They may release client, session, result, string, and byte-buffer resources; they must not stop the user-scoped daemon unless the user explicitly requested a runtime stop operation.

Safe Diagnostics

Binding diagnostics should preserve the structured AHRI TRE protocol and lifecycle failure shapes while keeping sensitive material out of logs, exceptions, test snapshots, and package artifacts.

Diagnostics must not expose:

• raw request bodies
• passwords, OAuth tokens, refresh tokens, bearer tokens, passfiles, or credential material
• raw PostgreSQL connection strings that contain secrets
• Restricted local references or developer workstation paths
• lake internals, DuckLake catalog internals, runtime storage roots, sockets, or secret-bearing runtime paths
• raw handle addresses or other process-local implementation details

Wrappers should surface safe fields from public protocol failure envelopes, safe protocol warnings, ABI status classes, and private lifecycle diagnostics. When a wrapper adds host-language exceptions or logs, the exception message should remain useful without reintroducing secrets or local runtime internals.

Payload Handling

Control-plane requests and responses use JSON. Large or tabular payloads use protocol data-plane references plus Arrow IPC, Parquet, or descriptor-only export artifacts.

Bindings should:

• inspect protocol response JSON before assuming a payload exists
• inspect ahri_tre_result_payload_count() and payload descriptors
• accept descriptor-only payloads as valid results
• copy borrowed payload bytes before freeing the result when needed
• convert Arrow IPC bytes to host-language dataframe objects at the binding edge
• keep Parquet as the durable exported dataset format

Payload descriptors may contain safe protocol refs, media types, suggested names, sizes, and byte-availability flags. They must not contain runtime Lake locations, datastore connection details, credentials, or local temporary paths.

Development Overrides

The ordinary binding install path is a prebuilt runtime artifact. Development overrides are allowed only as explicit local-development hooks.

A binding repository may support overrides for:

• a local staged runtime package under dist/
• an unpacked release archive
• a local C ABI shared library and generated header
• an explicit projected Client bootstrap fixture

Override configuration must be visibly separate from normal package discovery. Bindings may use their own visibly test-only runner inputs, but never a retired AHRI TRE product name or ordinary package-discovery fallback. Override diagnostics should report the override class without leaking Restricted local references or secret-bearing locations.

Representative Smoke Scenarios

Every binding repository should include a small smoke suite that can run against a staged or released AHRI TRE runtime artifact without cloning this Rust workspace.

The shared smoke expectations are:

These smoke tests prove the shared contract. Full host-language API coverage, dataframe conversion matrices, package-manager checks, and publication workflows belong in each external binding repository.

Seed Repository Checklist

Each external seed repository should start with:

• a language-native package skeleton
• a .devcontainer/ folder for that language and its tools
• a PostgreSQL service modelled on this repository’s development environment
• an Application-selected Lake mount at the canonical container-visible path
• runtime artifact discovery code based on package manifests and installer metadata
• C ABI declarations generated from or checked against ahri_tre_ffi_c.h
• a thin unsafe wrapper layer with deterministic cleanup
• protocol compatibility checks at startup
• private runtime lifecycle helpers
• generic public protocol JSON execution helpers
• Arrow IPC and Parquet payload entrypoints
• smoke scripts for the representative scenarios above
• handoff documentation that points back to this contract, ADR-0004, ADR-0005, the C ABI chapter, and the installation package guide

After seeding, ongoing package implementation should happen in the external repository’s issue tracker rather than in this Rust workspace.

Stable C ABI

The AHRI TRE C ABI is the language-neutral adapter for Python, Julia, R, C++, and other host-language wrappers. It gives wrappers a stable unsafe boundary without asking every language package to bind directly to Rust crates or recreate TRE workflow semantics.

The ABI deliberately offers only the low-level pieces that every wrapper needs: checking which library and protocol version were loaded, selecting a Client bootstrap, creating client/session/result handles, sending protocol JSON, reading response JSON, accessing optional payloads, and freeing anything the ABI allocated. It does not provide one C function for every TRE workflow. Typed functions such as list_datasets() or read_dataset_arrow() belong in host-language wrappers above this layer.

Contract Layers

Wrappers should treat the C ABI as three related but separate contracts:

Protocol compatibility is the behavioral compatibility contract. A wrapper should load the library, call the introspection functions, parse the reported protocol range, and fail fast when its required protocol is outside that range. Rust crate versions, CLI package versions, and dynamic library package versions are provenance metadata rather than substitutes for protocol checks.

How The ABI Works

At runtime, a wrapper creates an opaque ahri_tre_client and submits serialized public protocol request envelopes with ahri_tre_client_execute_protocol_json(). The C ABI forwards those envelopes to the configured Trusted runtime, then returns an opaque ahri_tre_result containing the serialized public protocol response JSON and any attached payload metadata or bytes.

Client construction selects and validates the projected Client bootstrap, including Deployment identity, HTTPS origin, and TLS trust, before returning a handle. Profiles are discovered from the authenticated runtime and selected by logical identifier.

The high-level flow is:

host-language wrapper
  -> C ABI library
  -> authenticated Trusted-runtime HTTPS transport
  -> public protocol request router
  -> application workflows
  -> protocol response JSON plus optional payloads
  -> C ABI result handle
  -> host-language objects

The C ABI keeps database, lake, OAuth/OIDC, daemon, and session internals out of the host-language process. Wrappers operate on protocol JSON, result handles, and copied or borrowed bytes; they do not receive raw PostgreSQL, DuckDB, DuckLake, token, Lake location, or runtime storage handles.

Client Bootstrap And Runtime

Client creation validates one projected Client bootstrap and keeps it immutable for the lifetime of the opaque handle. Callers may set client_bootstrap_path in ahri_tre_client_config. A NULL value selects /etc/ahri-tre/client.toml. If an explicit path is missing or invalid, creation fails without trying the canonical path.

The bootstrap supplies Deployment identity, Trusted-runtime HTTPS origin, and TLS trust. Profile metadata is server-discovered and selection sends only a logical identifier. Every client and selected-session protocol call uses that same authenticated remote transport. ABI v2 does not expose local runtime configuration, daemon discovery or lifecycle functions, endpoint or binary overrides, never-start flags, or daemon auto-start behavior. Host language wrappers should surface bootstrap selection and normal handle ownership only.

Protocol Execution

Public workflow semantics stay in protocol envelopes. C callers submit request JSON and receive response JSON. Python, Julia, R, and C++ wrappers may expose typed convenience APIs, but those APIs should generate protocol requests and parse protocol responses rather than depending on workflow-specific C symbols.

Use ahri_tre_client_execute_protocol_json() for requests that do not require selected-session context. Use ahri_tre_client_select_session_protocol_json() with a session.use request to obtain an opaque selected-session handle, then call ahri_tre_session_execute_protocol_json() for selected-session requests.

When execution returns AHRI_TRE_STATUS_OK, the result handle owns a response envelope. That envelope can represent either workflow success or protocol-shaped workflow failure. Wrappers should inspect the response JSON to determine workflow outcome.

The ABI status channel is reserved for unsafe-boundary failures such as null pointers, invalid config, unsupported ABI config, invalid handles, invalid UTF-8 before a protocol envelope can be parsed, or allocation failure. Protocol validation errors, unsupported request kinds, governance failures, daemon transport failures that can be represented as protocol errors, and managed runtime unavailability are returned as response JSON whenever the ABI can produce a protocol-shaped envelope.

Result Ownership

The ABI uses explicit ownership classes:

Borrowed views are owned by the result handle:

• ahri_tre_result_response_json_borrowed() returns borrowed response JSON.
• ahri_tre_result_payload_descriptor() returns borrowed descriptor strings.
• ahri_tre_result_payload_bytes_borrowed() returns borrowed payload bytes when bytes are attached.

Borrowed data is valid only until the owning result is freed. Host-language wrappers should copy values before releasing the result when those values need to outlive the immediate call scope.

Cleanup functions accept NULL as a no-op. Double-free, freeing a pointer with the wrong cleanup function, freeing foreign pointers, freeing copied bytes with the wrong length, and concurrent mutation or cleanup of the same handle are caller errors. First-stable same-handle concurrent use is not thread-safe unless a future ABI symbol explicitly documents otherwise.

Payloads

Response JSON carries protocol refs and descriptors. Large or tabular data is attached to result handles only when the executor provides a payload. Arrow IPC is the first in-memory or streaming tabular transfer format, and Parquet remains the durable exported dataset format.

Wrappers should first inspect ahri_tre_result_payload_count(), then read each ahri_tre_payload_descriptor. Descriptors expose safe protocol refs, media types, suggested names, size values, and byte-availability flags. They must not expose runtime Lake locations, datastore connection details, credentials, or local temporary paths.

Some payloads are descriptor-only export artifacts. Wrappers must handle data == NULL && len == 0 as a normal no-bytes case rather than a failure. When bytes are present, wrappers can either consume a borrowed view before freeing the result or request an owned copy and release it with ahri_tre_bytes_free().

Diagnostics And Redaction

ABI diagnostics and protocol diagnostics are safe for wrapper logs. They must not include raw request bodies, passwords, tokens, auth artifacts, Restricted local references, lake internals, runtime storage locations, or raw handle addresses.

Status messages from ahri_tre_status_message() describe status classes only. After client construction fails, ahri_tre_client_create_error_message() returns the safe name-only diagnostic for that thread’s most recent construction attempt; wrappers should copy it before another construction call on the same thread. Detailed workflow and transport outcomes belong in protocol response JSON. Wrappers should surface those structured JSON fields through host-language exceptions, result objects, logs, or notebook displays without assuming internal path or secret values will be present.

Wrapper Guidance

Wrapper packages should keep the C layer thin and deterministic:

1. Load the library and perform introspection before using credentials or opening sessions.
2. Compare the wrapper’s required protocol version against the reported compatibility range.
3. Select an explicit projected Client bootstrap or use the canonical /etc/ahri-tre/client.toml document.
4. Create the client once and retain its immutable authenticated Trusted-runtime transport capability.
5. Serialize public protocol request envelopes with the host language’s JSON library.
6. Treat AHRI_TRE_STATUS_OK as “a result envelope is available”, not “workflow succeeded”.
7. Parse public protocol response JSON for workflow success, failure, warnings, and resource refs.
8. Convert Arrow IPC or Parquet payloads at the host-language edge.
9. Copy borrowed data before freeing the owning result when the host language object needs to persist.
10. Release every owned handle, string, and byte buffer with the matching C ABI cleanup function.

Wrappers should not bypass the protocol by binding directly to application workflow internals, metadata adapters, lake adapters, or auth internals. They should also avoid adding wrapper-specific persistence formats or alternate control planes. The stable cross-language contract is public protocol JSON plus Arrow IPC, Parquet, and the C ABI ownership rules.

The generated header and the lower-level C examples live with the crate in crates/ahri_tre_ffi_c.

External source acquisition

Use ahri_tre_client_acquire_source or ahri_tre_session_acquire_source for public acquisition intent and separate bounded sensitive bytes. The historical *_acquire_https names remain compatible aliases. This common interface covers HTTPS, SQL and live REDCap acquisition; it does not add a C function per workflow. Caller-owned credential bytes remain borrowed for the synchronous call and must be cleared by the caller afterwards. Always inspect the returned protocol response even when the ABI status is successful.

SQL ingestion supports Client DuckDB/SQLite query-result uploads and explicit Client/Trusted PostgreSQL/MSSQL reads. Local database paths never cross into the Trusted runtime. See the SQL source contract for endpoint, credential, risk, schema, version and cleanup rules. The tested source-built Linux bundle includes its matching private query worker; other native package and binding releases require their own installed qualification.

Developer Installation Package Guide

Test Datastore Deployment Kit contract

The ordinary ahri-tre-<version>-<target> archives described below are release inputs; they are not the complete AHRI TRE Test Datastore Deployment Kit. The 0.2.0 contract and its v0.10.3-based server remain superseded historical artifacts in contract-v0.2.0.json.

The current MinisForum installed-conformance successor is the closed v3 0.3.23 composition contract in minisforum-contract-v0.3.23.json. It binds the published v0.10.12 server, client, and validator artifacts containing the numeric OAuth Dataset Variable-order correction. Earlier candidate archives and published kits through 0.3.22 remain immutable.

The original fixed-site contract remains frozen at deployment/test-datastore-kit/contract-v0.1.0.json. The current repository deployment/test-datastore-kit/contract.json records v3; P2.11c already isolates v1, v2, and v3 parsing and rendering.

The complete required artifact surface is versioned in deployment/test-datastore-kit/contract.json. It covers the Linux server, specialized site bundle, WSL2 and Apple Silicon macOS clients, conformance harness, clean-host evidence, SBOMs, dependency reports, licences, recovery tools, and artifact-bound runbooks. Its base-release record pins each selected archive, its .sha256 publication asset, and the expected archive digest:

Maintainers validate the source contract with:

cargo run -p xtask -- test-datastore-kit check-contract

Once every declared artifact has been produced under one staging root, assembly is explicit and requires the clean checkout’s full 40-character kit source revision:

KIT_SOURCE_REVISION="$(git rev-parse --verify HEAD)"
cargo run -p xtask -- test-datastore-kit assemble \
  --artifact-root dist/ahri-tre-test-datastore-deployment-kit-0.3.0 \
  --source-revision "$KIT_SOURCE_REVISION"

Assembly writes manifest.json and SHA256SUMS only after every required file exists and the Linux server, clean WSL2 client, and clean Apple Silicon macOS client evidence all report go. The supplied kit source revision must equal the current clean committed checkout; source-generated and site-generated artifacts bind to it. Release-origin artifacts separately retain the v0.10.4 base-release revision pinned by the contract. Assembly rejects absolute or parent paths, symlinks, missing artifacts, unmanifested executables, build-time Restricted local references, private-key and credential markers in textual public artifacts, inconsistent kit versions, unsupported targets, and no-go qualification evidence. Pinned binaries, libraries, and archives are opaque checksum-bound inputs because runtime libraries can legitimately embed parser marker strings. CI validates this contract on every change, but it deliberately has no kit publication step until the later installed-package and clean-host issues supply all required evidence.

Test Datastore Linux server package

Ticket 02 supplies the deterministic server-deliverable command:

cargo run --locked -p xtask -- test-datastore-kit package-server \
  --contract "$RELEASE_CONTRACT" \
  --release-archive downloads/ahri-tre-0.10.4-x86_64-unknown-linux-gnu.tar \
  --release-checksum downloads/ahri-tre-0.10.4-x86_64-unknown-linux-gnu.tar.sha256 \
  --artifact-root dist/ahri-tre-test-datastore-deployment-kit-0.3.0 \
  --source-revision "$RELEASE_SOURCE_REVISION"

The packaging command verifies the pinned archive name and SHA-256, its published checksum file, exact release source revision, and x86-64 ELF identity. It extracts the release-origin CLI, local daemon, Trusted runtime, Web control plane, C ABI, header, and DuckDB without rebuilding them, then generates the public Application and protocol schemas through the staged release CLI, audits every executable and shared-library dependency, and launches every staged executable without a Cargo target-directory path. Publication of server/ is atomic; a bad digest, source revision, architecture, dependency, launch check, or schema response leaves no partial deliverable.

The server deliverable contains systemd, sysusers, and tmpfiles definitions plus install, upgrade, rollback, diagnostic, and uninstall commands. To stage a filesystem image without activating identities or services, use:

DESTDIR=/path/to/image ./server/install.sh

On a new supported host, run sudo ./server/install.sh; on the MinisForum with the superseded server package already present, run sudo ./server/upgrade.sh. It creates or preserves the dedicated ahri-tre-runtime and ahri-tre-web identities and the declared configuration, runtime, Managed-secret, Injected-secret, log, scratch, and backup boundaries. It installs only ahri-tre in /usr/bin; ahri-tred remains a local Managed runtime component under /usr/libexec/ahri-tre and is never exposed as the multi-user Trusted runtime.

The installer does not invent Application configuration or Secret values and does not start either service. After the site bundle has installed /etc/ahri-tre/config.toml and its declared prerequisites, validate and start the Trusted runtime with:

sudo /usr/libexec/ahri-tre/diagnose.sh preflight-runtime
sudo /usr/libexec/ahri-tre/diagnose.sh dependencies
sudo systemctl enable --now ahri-tre-runtime.service
sudo /usr/libexec/ahri-tre/diagnose.sh readiness

Diagnostics return JSON and classify artifact, configuration, permission, library, Deployment-identity, TLS, Secret, and readiness failures. The Web unit is installed separately but deliberately remains disabled until ticket 04 binds and qualifies its site ingress and OIDC inputs.

The v0.10.4 runtime owns /run/ahri-tre/admin.sock and /run/ahri-tre-login/login.sock, validates their ownership and modes, and recovers only safe stale sockets. It does not implement systemd socket activation, so the kit ships no competing .socket unit. The runtime service creates its private runtime directories and waits for the process-owned sockets as its readiness boundary. systemd applies restart-on-failure, a bounded start and stop interval, SIGTERM, restrictive service permissions, and explicit configuration and writable paths.

Superseded 0.2.0 Test Datastore site bundle

The first Ticket 12 pass generated a non-secret site bundle from one closed v2 input document plus public TLS material. The confirmed MinisForum profile is recorded in deployment/test-datastore-kit/site-package/site-inputs.minisforum.json. It identifies Deployment UUID f2ef37c5-7430-468a-a439-b3ba1b0527c1, Runtime origin https://runtime.svrltreapcc02.home.arpa, the reserved address 192.168.31.75, and the /data/ahri-tre storage boundary.

The following command is retained only to reproduce historical evidence; do not use its output to provision the MinisForum. It supplied a public Runtime certificate chain whose SAN contains runtime.svrltreapcc02.home.arpa and its public issuing CA chain:

cargo run --locked -p xtask -- test-datastore-kit package-site \
  --inputs deployment/test-datastore-kit/site-package/site-inputs.minisforum.json \
  --runtime-certificate-chain pki/runtime-certificate-chain.pem \
  --public-ca-chain pki/deployment-ca-chain.pem \
  --server-component-manifest \
    dist/ahri-tre-test-datastore-deployment-kit-0.1.0/server/component-versions.json \
  --artifact-root dist/ahri-tre-test-datastore-deployment-kit-0.2.0

Do not pass --web-certificate-chain for this CLI-only profile. A Web-enabled profile must instead declare its separate authority and supply a matching public Web chain. Private keys, database passwords, OAuth secrets, encryption keys, tokens, and Runtime credentials are never packager inputs and must not be placed in the repository or generated bundle.

Generation rejects a profile with an unsupported Ubuntu release or artifact, non-private or ambiguous listener, mismatched address or client CIDR, invalid TLS authority, direct workstation database exposure, unsafe or insufficient storage, incomplete Web authority, private-key input, or partial contract. It validates the Application graph through ahri_tre_config, derives the public Client projection, validates the exact server component manifest, and publishes all 21 site/ artifacts atomically. Generated provisioning compares the installed manifest against the digest of that structured, package-selected input; it does not search JSON text or require an ambient JSON parser.

Review these generated files before copying the bundle:

• site/site-inputs.json — the exact confirmed profile and Deployment UUID;
• site/name-resolution.md — one matching hosts-file entry for the MinisForum, Windows, WSL2, and macOS;
• site/firewall-plan.json — Runtime ingress from 192.168.31.0/24, Web disabled, and non-loopback PostgreSQL denied;
• site/filesystem-plan.json — Lake and scratch beneath the /data mount;
• site/client-publication/wsl2.json and macos.json — the same Runtime origin and public CA publication; and
• site/injected-secrets.json and site/secret-reference-inventory.json — descriptors and ownership only, never values.

After the Ticket 02 server package is installed, provision the private values through the external authorities and paths named by those two Secret inventories. Apply the bundle only after the generated name-resolution instructions work on every client and after confirming both live host identifiers:

sudo ./site/provision.sh \
  --confirm-host svrltreapcc02 \
  --confirm-address 192.168.31.75

The provisioner verifies Ubuntu 26.04, x86-64, the installed v0.1.0 server package provenance, live hostname and reserved address, canonical Runtime DNS, PostgreSQL 18 TLS, the /data mounts and capacity, required Runtime Secret projections, Application validity, and Datastore-create preflight before live mutation. It refuses an existing configuration or active AHRI TRE service and has no reset, force, or adoption option. It installs configuration and public Client trust atomically, starts only the Trusted runtime, and submits only ahri-tre-test through the protected administration socket. verify-readiness.sh then checks the persisted ready binding, schema, PostgreSQL database, DuckDB/DuckLake, Lake location, and trusted scratch.

Identity admission remains a server-side PostgreSQL operation and distributes no database credential. The supplied commands are idempotent and accept only the canonical ORCID iD; the operator derives the PostgreSQL role by adding the orcid_ prefix while preserving the canonical iD’s hyphens:

sudo ./site/admit-user.sh 0000-0000-0000-0001
sudo ./site/remove-user.sh 0000-0000-0000-0001

When upgrading a site whose earlier operator replaced the iD’s hyphens with underscores, repeat admit-user.sh with the canonical iD. The corrected operator recognizes only that exact legacy mapping, renames the existing role so its grants are preserved, rewrites the identity map, and reloads it. It refuses an ambiguous or unrelated existing role instead of creating another admission.

Admission grants membership in tre_oauth_users plus schema usage, connect, read, insert, update, and sequence authority needed by the bounded gate. It does not grant superuser, database creation, role creation, DDL ownership, function-wide execution, delete, or an inherited database password. Web Secret projection and service activation are absent from the CLI-only Application configuration and the Web service remains disabled.

For regression or recovery work against the original Ticket 03 profile, use contract-v0.1.0.json together with the v1 example input and both Runtime and Web public certificate chains. Do not edit a generated v1 bundle into a MinisForum bundle.

0.3.23 MinisForum site and client bundles

minisforum-contract-v0.3.23.json advances the complete v0.3.22 composition to patch release v0.10.12. That release qualifies the integer source column in the OAuth Dataset metadata query, preserving numeric Variable order even though the wire fields remain text. The server, site, WSL2, and controlled macOS packages otherwise retain the accepted recovery and diagnostic contracts from v0.3.22.

The release assets are bound to source revision 1ae413547dae23b7fcbb659d064c704cce34cfa7. Compose the server and site with minisforum-contract-v0.3.23.json, then package both clients with minisforum-client-contract-v0.3.23.json. Exact commands, including the Linux/AMD64 controlled macOS generation path, are in deployment/test-datastore-kit/client-packaging.md.

The published archive and its adjacent checksum are release assets of v0.10.12. Two isolated Linux compositions from candidate source revision 1eee63105446396bf53c13c89e22239bda1408b4 were byte-for-byte identical at SHA-256 8c4664cd7fb48c040869c09172cf15cba2bb324c2958b0d97407c424ce20fcea.

0.3.22 MinisForum site and client bundles

minisforum-contract-v0.3.22.json diagnoses the v0.3.21 native macOS dataset.verify no-go without changing the accepted 0.3.12 lifecycle predecessor. It binds release v0.10.11, whose administration client permits bounded clean-host Datastore creation to complete and whose PostgreSQL-validator publication passed both portable adapter qualifications. Two independent Linux-container compositions from source revision 85167cb99a5780e37537de50a67204838d540f28 produced the identical archive SHA-256 8193c540887c2bb39782befc5eab742f84fa816bf6f4fb2cc664cf92073d69b1.

After checksum-verifying the v0.10.11 Linux, macOS, and validator release assets, compose the server in the persistent development container:

cargo run --locked -p xtask -- test-datastore-kit package-server \
  --contract deployment/test-datastore-kit/minisforum-contract-v0.3.22.json \
  --release-archive dist/release-v0.10.11/ahri-tre-0.10.11-x86_64-unknown-linux-gnu.tar \
  --release-checksum dist/release-v0.10.11/ahri-tre-0.10.11-x86_64-unknown-linux-gnu.tar.sha256 \
  --artifact-root dist/ahri-tre-test-datastore-deployment-kit-0.3.22 \
  --source-revision 032a37c60e3fc2d3d03cc7dfcd5f680527aebc2f

Create dist/private-kit-inputs-0.3.22/site-inputs.minisforum.v3.json with mode 0600 from the current v3 template, filling only its REPLACE_... values. Package the site in the same container:

cargo run --locked -p xtask -- test-datastore-kit package-site \
  --contract deployment/test-datastore-kit/minisforum-contract-v0.3.22.json \
  --inputs dist/private-kit-inputs-0.3.22/site-inputs.minisforum.v3.json \
  --runtime-certificate-chain pki/runtime-certificate-chain.pem \
  --public-ca-chain pki/deployment-ca-chain.pem \
  --postgresql-ca-chain dist/private-kit-inputs-0.3.22/postgresql-ca-chain.pem \
  --server-component-manifest dist/ahri-tre-test-datastore-deployment-kit-0.3.22/server/component-versions.json \
  --validator-release-root dist/release-v0.10.11/validator \
  --artifact-root dist/ahri-tre-test-datastore-deployment-kit-0.3.22

Hash the generated site/client.toml, site/public-ca-chain.pem, and both client-publication documents into minisforum-client-contract-v0.3.22.json before packaging either client. Exact client commands and the controlled cross-platform generation path are in deployment/test-datastore-kit/client-packaging.md.

0.3.14 MinisForum site and client bundles

minisforum-contract-v0.3.14.json advances the live 0.3.13 site from the v0.10.6 application and validator to v0.10.8. Release v0.10.8 classifies the raw libpq OAuth rejection before diagnostic sanitization, discards the raw diagnostic at the adapter boundary, and carries only the stable unadmitted-user variant into the Application and Runtime protocol layers.

The site upgrader admits only the exact healthy v0.10.6 validator, retains it as the rollback container, and loads the checksummed v0.10.8 OCI archive. The server package is newly composed from the v0.10.8 Linux release archive. The lifecycle activator accepts only the installed 0.3.13 projection plan and the new server and validator before advancing it to 0.3.14. The reboot-qualified projector, readiness waiter, and systemd ordering remain unchanged.

After downloading and checksum-verifying the published v0.10.8 Linux archive and validator release assets, compose the server in the persistent development container:

cargo run --locked -p xtask -- test-datastore-kit package-server \
  --contract deployment/test-datastore-kit/minisforum-contract-v0.3.14.json \
  --release-archive dist/release-v0.10.8/ahri-tre-0.10.8-x86_64-unknown-linux-gnu.tar \
  --release-checksum dist/release-v0.10.8/ahri-tre-0.10.8-x86_64-unknown-linux-gnu.tar.sha256 \
  --artifact-root dist/ahri-tre-test-datastore-deployment-kit-0.3.14 \
  --source-revision a292eab3d21a02060b115927da3266f62ad982a0

Create dist/private-kit-inputs-0.3.14/site-inputs.minisforum.v3.json with mode 0600 from the current v3 template, filling only its REPLACE_... values. Package the site in the same container:

cargo run --locked -p xtask -- test-datastore-kit package-site \
  --contract deployment/test-datastore-kit/minisforum-contract-v0.3.14.json \
  --inputs dist/private-kit-inputs-0.3.14/site-inputs.minisforum.v3.json \
  --runtime-certificate-chain pki/runtime-certificate-chain.pem \
  --public-ca-chain pki/deployment-ca-chain.pem \
  --postgresql-ca-chain dist/private-kit-inputs-0.3.14/postgresql-ca-chain.pem \
  --server-component-manifest dist/ahri-tre-test-datastore-deployment-kit-0.3.14/server/component-versions.json \
  --validator-release-root dist/release-v0.10.8/validator \
  --artifact-root dist/ahri-tre-test-datastore-deployment-kit-0.3.14

Package companion clients with minisforum-client-contract-v0.3.14.json. The WSL2 client packager runs in the development container; the macOS client packager runs on native Apple Silicon because both launch their staged binaries. Exact commands are in deployment/test-datastore-kit/client-packaging.md. Server update and live qualification steps are in the generated site/HITL.md.

0.3.13 MinisForum site and client bundles

minisforum-contract-v0.3.13.json corrects the generated lifecycle activator from the unpublished live 0.3.12 activation attempt. The 0.3.12 packager silently replaced a required server-manifest digest marker with an empty string. The corrected packager injects the exact digest and its package test requires that digest in the final script. Missing digest wiring now leaves an unresolved marker and fails package validation.

Kit 0.3.13 deliberately reuses the exact v0.10.6 server package from 0.3.12. Its site input therefore declares site kit 0.3.13 and installed server kit 0.3.12. The guarded activator accepts only the installed 0.3.11 projection plan, the exact 0.3.12 server manifest, the healthy v0.10.6 validator, and the active boot-ordered services before advancing lifecycle metadata to 0.3.13.

Start a clean artifact root by copying the already packaged, checksummed 0.3.12 server directory byte-for-byte; do not rebuild or edit it:

artifact_root=dist/ahri-tre-test-datastore-deployment-kit-0.3.13
test ! -e "$artifact_root"
mkdir -p "$artifact_root"
cp -a dist/ahri-tre-test-datastore-deployment-kit-0.3.12/server "$artifact_root/server"
test "$(sha256sum "$artifact_root/server/component-versions.json" | cut -d ' ' -f 1)" = f6686e8d3d5199a4cb280755d57df334de9e3f52d73dc52ba035c5d912a3f363

Create dist/private-kit-inputs-0.3.13/site-inputs.minisforum.v3.json with mode 0600 from the current v3 template, filling only its two REPLACE_... values. Package the site in the persistent development container:

cargo run --locked -p xtask -- test-datastore-kit package-site \
  --contract deployment/test-datastore-kit/minisforum-contract-v0.3.13.json \
  --inputs dist/private-kit-inputs-0.3.13/site-inputs.minisforum.v3.json \
  --runtime-certificate-chain pki/runtime-certificate-chain.pem \
  --public-ca-chain pki/deployment-ca-chain.pem \
  --postgresql-ca-chain dist/private-kit-inputs-0.3.13/postgresql-ca-chain.pem \
  --server-component-manifest "$artifact_root/server/component-versions.json" \
  --validator-release-root dist/release-v0.10.6/validator \
  --artifact-root "$artifact_root"

Then package companion clients with minisforum-client-contract-v0.3.13.json. Exact commands are in deployment/test-datastore-kit/client-packaging.md; update and qualification steps are in the generated site/HITL.md and each client’s WORKSTATION.md.

0.3.12 MinisForum site and client bundles

minisforum-contract-v0.3.12.json advances the live 0.3.11 site to the published v0.10.6 source revision. Its guarded upgrade replaces the exact healthy v0.10.5 validator container while retaining a rollback container, installs the v0.10.6 server package, and finally advances only the persistent projection plan. The reboot-qualified projector, readiness waiter, systemd ordering, Datastore, PostgreSQL data, Lake, configuration, and Secret authority remain unchanged.

Compose the site with minisforum-contract-v0.3.12.json, then package clients with minisforum-client-contract-v0.3.12.json. Exact commands are in deployment/test-datastore-kit/client-packaging.md; update and qualification steps are in the generated site/HITL.md and each client’s WORKSTATION.md.

After downloading and checksum-verifying the published v0.10.6 Linux archive and validator release files, run the Cargo commands in the persistent development container from /workspaces/ahri-tre-rs:

cargo run --locked -p xtask -- test-datastore-kit package-server \
  --contract deployment/test-datastore-kit/minisforum-contract-v0.3.12.json \
  --release-archive dist/release-v0.10.6/ahri-tre-0.10.6-x86_64-unknown-linux-gnu.tar \
  --release-checksum dist/release-v0.10.6/ahri-tre-0.10.6-x86_64-unknown-linux-gnu.tar.sha256 \
  --artifact-root dist/ahri-tre-test-datastore-deployment-kit-0.3.12 \
  --source-revision 915dd9cb89940d20588b0c9f1a1fe11069255ca0

Create dist/private-kit-inputs-0.3.12/site-inputs.minisforum.v3.json with mode 0600 as a private copy of the current v3 template, filling only its two REPLACE_... values. The ignored dist/ path is visible in both WSL2 and the development container. Then package the site; the PostgreSQL CA input is public certificate material, not its signing key:

cargo run --locked -p xtask -- test-datastore-kit package-site \
  --contract deployment/test-datastore-kit/minisforum-contract-v0.3.12.json \
  --inputs dist/private-kit-inputs-0.3.12/site-inputs.minisforum.v3.json \
  --runtime-certificate-chain pki/runtime-certificate-chain.pem \
  --public-ca-chain pki/deployment-ca-chain.pem \
  --postgresql-ca-chain /home/kobus/ahri-tre-pki/public/postgresql-ca-chain.pem \
  --server-component-manifest dist/ahri-tre-test-datastore-deployment-kit-0.3.12/server/component-versions.json \
  --validator-release-root dist/release-v0.10.6/validator \
  --artifact-root dist/ahri-tre-test-datastore-deployment-kit-0.3.12

The validator directory must contain the OCI archive and adjacent checksum, release metadata and checksum, both schemas, and qualification result from the same v0.10.6 release. Do not copy a private key or Secret into either input directory.

0.3.11 MinisForum site and client bundles

minisforum-contract-v0.3.11.json retains the 0.3.10 server behavior and the reboot-qualified service bytes. Its guarded updater accepts only the installed 0.3.10 plan and advances lifecycle metadata without restarting services or replacing binaries. The shared WSL2/macOS qualification gate reads the packaged CSV header and data-row count: derive verification checks its dimensions, while metadata verification checks the exact variable count, names, and order.

Compose the site with minisforum-contract-v0.3.11.json, then package clients with minisforum-client-contract-v0.3.11.json. Exact commands are in deployment/test-datastore-kit/client-packaging.md; update and qualification steps are in the generated site/HITL.md and each client’s WORKSTATION.md.

0.3.10 MinisForum site and client bundles

minisforum-contract-v0.3.10.json retains the 0.3.9 site behavior and the reboot-qualified 0.3.6 service bytes. Its guarded updater accepts only the installed 0.3.9 plan and advances lifecycle metadata without restarting services or replacing binaries. The companion WSL2 and macOS packages retain the separately validated logical Datastore ID and physical PostgreSQL database name from 0.3.9. Their shared qualification gate now expects the canonical CSV fixture’s three data rows and four columns. The package-contract test derives those dimensions from the fixture, preventing the assertion from drifting again.

Compose the site with minisforum-contract-v0.3.10.json, then package clients with minisforum-client-contract-v0.3.10.json. Exact commands are in deployment/test-datastore-kit/client-packaging.md; update and qualification steps are in the generated site/HITL.md and each client’s WORKSTATION.md.

0.3.8 MinisForum site and client bundles

minisforum-contract-v0.3.8.json composes the current site from the unchanged v0.10.5 server and validator release assets. Its generated operator preserves canonical hyphenated ORCID subjects, creates the bounded tre_oauth_users authority, and repairs membership and grants on repeat admission. The guarded updater accepts the exact installed 0.3.7 plan with the reboot-qualified 0.3.6 projector and waiter bytes, then updates plan metadata without restarting services or replacing binaries.

Use site-inputs.minisforum.v3.template.json to create the private input, then compose the v0.10.5 server and package the site with the 0.3.8 contract and artifact-root names. Package WSL2 and macOS clients with minisforum-client-contract-v0.3.8.json; the exact commands are in deployment/test-datastore-kit/client-packaging.md. The published 0.3.8 site archive must exist before either companion client is composed.

The archive and adjacent checksum are published with v0.10.5. Their kit source revision is 33e005dff84abff3c58f2d912ec7f0ba582902ed and the archive SHA-256 is 6c88258f84e17e047bb76eefc72becf399faba64483689fbefdf0c69d686670f.

Review the generated site/HITL.md. An existing 0.3.7 MinisForum runs the guarded metadata update, reruns site/admit-user.sh for the admitted canonical ORCID iD, and then completes both the admitted and valid-but-unadmitted client gates. A fresh installation follows server-installation.md with the 0.3.8 archive.

0.3.6 MinisForum site bundle

minisforum-contract-v0.3.6.json is the reboot-qualified composition. It pins the v0.10.5 Linux server archive and production validator OCI archive, qualification matrix, schemas, image digest, and full release source revision. The default contract.json preserves the published 0.3.0 record.

The checked-in site-inputs.minisforum.v3.template.json is deliberately not a deployable profile: replace only REPLACE_ORCID_SANDBOX_CLIENT_ID and REPLACE_POSTGRESQL_CA_SHA256 in a private working copy after completing the first two actions in the generated HITL guide. The packager rejects either placeholder and publishes no partial site.

Compose the v0.10.5 server package first. Then generate the final site with:

cargo run --locked -p xtask -- test-datastore-kit package-site \
  --contract deployment/test-datastore-kit/minisforum-contract-v0.3.6.json \
  --inputs /tmp/site-inputs.minisforum.v0.3.6.json \
  --runtime-certificate-chain pki/runtime-certificate-chain.pem \
  --public-ca-chain pki/deployment-ca-chain.pem \
  --postgresql-ca-chain /path/to/postgresql-public-ca-chain.pem \
  --server-component-manifest \
    dist/ahri-tre-test-datastore-deployment-kit-0.3.6/server/component-versions.json \
  --validator-release-root dist/release-v0.10.5/validator \
  --artifact-root dist/ahri-tre-test-datastore-deployment-kit-0.3.6

Review site/HITL.md and use its commands in order. The generated site carries the OCI bytes and their release evidence; it never rebuilds the validator and contains no Secret value or private key. Its additional site artifacts define the persistent Injected-secret source contract, boot projector, managed PostgreSQL service, Trusted runtime dependency drop-in, and guarded upgrade-secret-projector.sh. Fresh installs capture verified projections through install-secret-projector.sh; an installed 0.3.5 lifecycle uses the guarded boot-service upgrader. The projector recreates /run/secrets, including explicitly traversable shared namespaces, before PostgreSQL and the Runtime on every boot. The bounded readiness waiter tolerates transient Docker health states but does not release the Runtime until PostgreSQL is healthy.

The archive and checksum are published with v0.10.5. Their kit source revision is 2fd59cc5ea287b070c406c2e328104d92caa143f and the archive SHA-256 is 84cee3dd7dc1ef798673909c6cefc9b9801a1337986a342467f7b5afa31a5780. Live MinisForum qualification proved projection, PostgreSQL health gating, Runtime ordering, and Datastore persistence across a clean boot. Real admitted and valid-but-unadmitted ORCID Session checks remain human qualification steps, not infrastructure publication gates.

This page is for maintainers who need to build installable AHRI TRE local runtime packages. It is not required for ordinary users. End users should start with the Getting Started chapter.

The implemented maintainer interface is the workspace xtask package-release command:

cargo run -p xtask -- package-release --target x86_64-unknown-linux-gnu

The command builds or reuses release artifacts, stages an installer-ready package under dist/, audits dynamic dependencies, runs staged launch validation, writes package metadata and external-datastore examples, and archives the result. The package provides the local runtime used by the public CLI and language wrapper packages. It does not provision PostgreSQL server infrastructure or lake storage; users provide connectivity and credentials through profile and secret files.

Package Targets

Target selection is explicit. Pass one or more --target values:

cargo run -p xtask -- package-release \
  --target x86_64-unknown-linux-gnu \
  --target aarch64-unknown-linux-gnu

Missing targets are rejected by the CLI parser. Unsupported targets fail during planning before build or staging work begins. Repeated target selection lets a maintainer produce multiple packages from one invocation when the host has the required toolchain support.

Supported package targets are:

Native Windows runtime packages are out of scope for this packager slice.

CI Release Packaging

The CI release packaging path calls the same maintainer command as local packaging:

cargo run -p xtask -- package-release --target "<target-triple>"

The release-package matrix runs only for manual dispatches, GitHub release events, and version tag refs. Ordinary pull-request, branch, and scheduled CI runs still build documentation and run normal checks, but they do not produce release package candidates. This keeps untrusted or routine validation from publishing installer-ready runtime artifacts.

Each matrix entry builds one target on the runner family shown in the package target table, verifies the staged package with .github/scripts/verify-release-package-artifact.sh, writes a SHA-256 checksum for the target tarball, and uploads a workflow artifact named:

ahri-tre-<version>-<target>

The workflow artifact is the review surface for a release candidate. It contains the staged package directory, the deterministic tar archive, and the archive checksum:

dist/ahri-tre-<version>-<target>/
dist/ahri-tre-<version>-<target>.tar
dist/ahri-tre-<version>-<target>.tar.sha256

Inside the staged package, reviewers should expect the manifest, dependency-audit reports, staged validation summaries, installer metadata, generated README, generated profile and secrets examples, runtime libraries, C ABI header, CLI, daemon, and install script. The manifest records the validation commands, dependency report paths, installer contract, archive name, workspace version, package target, and package-relative runtime paths.

GitHub Release Assets

The release-package-publish job consumes the matrix workflow artifacts and attaches public assets only from a GitHub release event or a version tag ref. Pull-request, branch, scheduled, and manual-dispatch package runs do not attach assets to GitHub Releases. Manual dispatch is for producing reviewable workflow artifacts before a release, not for publication.

Before upload, .github/scripts/prepare-release-package-assets.sh verifies every supported target is present, rechecks the package artifact surface, requires the ahri-tre.package.v1 manifest schema and matching target triple, requires dependency reports and staged validation summaries, rejects known Restricted local references and secret material, and writes release checksums. Publication uses gh release upload against the existing release or tag. It does not create release notes, create GitHub releases, or infer versions from binding repositories.

Release assets use the stable names:

ahri-tre-<version>-<target>.tar
ahri-tre-<version>-<target>.tar.sha256

Binding repositories should select a runtime artifact by the AHRI TRE runtime version they support and the host target triple they are packaging for. For example, a Python, Julia, or R binding build targeting Apple Silicon macOS should download ahri-tre-<version>-aarch64-apple-darwin.tar, verify the matching .sha256, read share/ahri-tre/manifest.json, confirm schema_version is ahri-tre.package.v1, confirm the target matches the requested host triple, and then use the manifest and installer metadata to locate the C ABI library, header, CLI, and sibling daemon.

CI packaging intentionally remains a runtime-artifact publication path only. Container images, bundled PostgreSQL server infrastructure, lake provisioning, database migrations, filesystem mounting, .deb, .rpm, .pkg, Homebrew, PyPI, CRAN, Julia registry, and other installer-specific package formats stay out of scope for this guide and for the release package workflow.

Common Release Checks

Start from the release commit and run the normal repository validation before publishing release candidates:

cargo fmt --all
cargo clippy --workspace --all-targets --all-features -- -D warnings
cargo test --workspace

For a release candidate that changes Local deployment behaviour, use the supported deployment interface for its live checks:

./dev-env local up
./dev-env local test smoke
bash .devcontainer/scripts/check-lake.sh

Live PostgreSQL, lake, OAuth, ORCID, or REDCap checks are not default package-release steps because those services are external to installation.

The workflow contract check .github/scripts/check-release-package-workflow.sh runs in CI to make release workflow drift visible. It fails if the matrix stops invoking xtask package-release, if expected targets or runner families disappear, if workflow artifacts stop including staged packages, tarballs, and checksums, or if release asset publication is no longer gated to published GitHub Release events.

Published prereleases use the same package validation and asset publication gates. Only non-prerelease releases build and publish the stable documentation channel; a release candidate leaves the existing stable documentation in place.

Command Behaviour

By default, package-release invokes the target build, stages the package, audits dependencies, validates the staged binaries, writes package materials, and creates the archive:

cargo run -p xtask -- package-release --target x86_64-unknown-linux-gnu

Use --dry-run to review target planning without building or staging:

cargo run -p xtask -- package-release \
  --target x86_64-unknown-linux-gnu \
  --dry-run

Release packaging always builds its binaries from the clean checked-out revision. Reusing ignored Cargo target outputs is intentionally unsupported because it could attach current provenance to stale binaries.

Linux targets use cargo zigbuild --release --locked. The Apple Silicon macOS target uses native cargo build --release --locked. Every target includes the public CLI, local daemon, and stable C ABI crate used by language wrappers. The x86-64 Linux server target additionally builds the Trusted runtime and Web control plane.

The devcontainer image installs Zig and the workspace-pinned cargo-zigbuild version declared in the root Cargo.toml under workspace.metadata.ahri_tre.release_tools. If you are working outside the devcontainer, run:

bash .devcontainer/scripts/install-cross-tools.sh

That script reads the same workspace pin and installs the matching cargo-zigbuild binary before release packaging.

The script also accepts install-zig, install-rust-targets, and install-cargo-zigbuild phases. The devcontainer Dockerfile invokes these separately so a later failure does not invalidate completed cross-tool setup. Zig archives are resumed from a build cache, verified against the pinned upstream SHA-256 digest, and downloaded with visible progress.

Staged Layout

Each selected target stages an isolated package directory:

dist/ahri-tre-<version>-<target>/
  bin/
    ahri-tre
    ahri-tred
    ahri-tre-query-worker
    ahri-tre-runtime (x86-64 Linux server target)
    ahri-tre-web (x86-64 Linux server target)
  include/
    ahri_tre_ffi_c.h
  lib/
    libahri_tre_ffi_c.so or libahri_tre_ffi_c.dylib
    libduckdb.so or libduckdb.dylib
  share/ahri-tre/
    README.md
    installer.json
    manifest.json
    RELEASE_NOTES.md
    sbom.spdx.json
    licenses/
      LICENSE
      DUCKDB-LICENSE.txt
    dependency-audit/
      bin-ahri-tre.txt
      bin-ahri-tred.txt
      bin-ahri-tre-query-worker.txt
      bin-ahri-tre-runtime.txt (x86-64 Linux server target)
      bin-ahri-tre-web.txt (x86-64 Linux server target)
      lib-ffi.txt
  install.sh

The bin/ directory contains the user-facing CLI, local daemon, and private query/acquisition worker. The worker is installed beside the runtime binaries under libexec/ahri-tre, with a launcher for binding-side discovery. Its isolation is Linux-specific; packaging it on macOS does not enable unsupported execution. The x86-64 Linux server target additionally contains the Trusted runtime and Web control plane; workstation and Arm64 packages do not carry those services. The lib/ directory contains the bundled redistributable runtime libraries needed by those binaries and wrappers. DuckDB is located from the target release outputs and copied into lib/, so installed users do not need Cargo’s target directory or a development checkout.

The stable C ABI artifacts are staged as the shared library in lib/ and the public header in include/. Python, Julia, R, and future language wrapper packages can use these artifacts together with the sibling daemon path bin/ahri-tred recorded in installer metadata.

Runtime Libraries

The package bundles DuckDB and the stable AHRI TRE C ABI shared library as bundled-redistributable runtime libraries. Additional redistributable runtime libraries can be staged into lib/ by the packager when they become required. The generated installer copies lib/ into the install prefix and writes launchers for every binary present in that target package.

Installed launchers execute runtime binaries from libexec/ahri-tre and set the platform library path before dispatch:

The dependency audit allowlist treats staged libraries and expected platform libraries as acceptable. Host-specific or unexpected dynamic links are reported as warnings in command output and in dependency reports so maintainers can decide whether to bundle, install, or document them. Unresolved dependencies, for example libname.so => not found, fail packaging.

Manifest Contract

Every staged package writes share/ahri-tre/manifest.json with schema version ahri-tre.package.v1. The manifest is intended for release review and future CI automation, so paths are package-relative and must not expose Restricted local references.

At maintainer level, the manifest contains:

• schema_version, package_name, workspace_version, target, platform, git_commit, full source_revision, and build_profile.
• staged_binaries for bin/ahri-tre, bin/ahri-tred, and bin/ahri-tre-query-worker, plus the Trusted runtime and Web control plane on the x86-64 Linux server target.
• staged_libraries for the C ABI shared library and DuckDB.
• c_abi_artifacts for the wrapper-facing shared library and public header.
• required_runtime_libraries with library names, paths, and classifications.
• dependency_reports pointing at package audit files.
• validation_commands showing the staged smoke checks and library path environment used to run them.
• installer metadata for prefix-based installation, generated launchers, support files and wrapper runtime discovery.
• archive_path, written as ahri-tre-<version>-<target>.tar after the archive path is known.

Generated package material includes release notes, an SPDX SBOM, the workspace licence, and the DuckDB licence. It is checked for known secret canaries and patterns for Restricted local references before it is written. Non-fixture packaging requires a clean, committed checkout.

Installer Materials

The staged package includes human and machine installer material:

• share/ahri-tre/README.md describes package layout, prefix installation, external datastore configuration, smoke checks, and wrapper runtime paths.
• share/ahri-tre/installer.json records the ahri-tre.installer.v1 installer contract, install directories, launcher library-path strategy, and wrapper runtime locations.
• install.sh installs into AHRI_TRE_PREFIX or a prefix passed as the first argument. It copies lib/, include/, and share/ahri-tre/, stores runtime binaries under libexec/ahri-tre, and writes bin/ launchers.
• The package contains no profile dotenv file, secret overlay, token file, or credential template. Operators hand off a projected Client bootstrap through a protected deployment path; authentication artifacts remain in the Injected/Managed Secret subsystem.

The installer does not create databases, run migrations, mount filesystems, or own lake storage. Physical PostgreSQL database names, DuckLake catalog settings, configured container-visible Lake location, and DuckLake catalog credentials remain operator, create/adoption, diagnostic, or repair concerns rather than ordinary installation profile fields.

Staged Validation

Validation runs by default from the staged package, not from the build tree. The command sets the package lib/ directory on the platform library path and runs:

dist/ahri-tre-<version>-<target>/bin/ahri-tre version
dist/ahri-tre-<version>-<target>/bin/ahri-tre doctor --format json
dist/ahri-tre-<version>-<target>/bin/ahri-tre schema list --format json
dist/ahri-tre-<version>-<target>/bin/ahri-tred --help
dist/ahri-tre-<version>-x86_64-unknown-linux-gnu/bin/ahri-tre-runtime --help
dist/ahri-tre-<version>-x86_64-unknown-linux-gnu/bin/ahri-tre-web --help

The final two checks apply only to the x86-64 Linux server target.

Any validation command failure fails packaging for that target. When repeated targets are requested, the packager reports each target outcome and preserves successful packages for targets that completed before another target failed.

Archive Outputs

After dependency audit and staged validation pass, the packager writes a tar archive beside the staged package:

dist/
  ahri-tre-<version>-x86_64-unknown-linux-gnu/
  ahri-tre-<version>-x86_64-unknown-linux-gnu.tar

The archive contains the staged package under a top-level ahri-tre-<version>-<target>/ directory and preserves executable bits for commands and installer scripts. Multiple requested targets produce isolated staged directories, manifests, dependency reports, and archives.

Out Of Scope

This packager slice deliberately does not produce Dockerfiles, Compose bundles, container images, registry pushes, GitHub releases, Homebrew formulae, Debian packages, RPMs, macOS installer packages, or native Windows packages. It also does not bundle PostgreSQL server infrastructure, provision a lake filesystem, or ship operator-specific credentials.

Local Documentation Build

The local documentation build assembles the narrative mdBook and workspace rustdoc into one channel-shaped site tree.

From the repository root:

docs/scripts/build-local-docs.sh

The default channel is dev, and the generated site is written to:

target/docs/site/dev/

Open target/docs/site/dev/index.html in a browser to inspect the assembled site. Set DOCS_CHANNEL to build a different local channel name:

DOCS_CHANNEL=stable docs/scripts/build-local-docs.sh

Prerequisites:

• mdbook
• jq
• the Rust toolchain configured by this workspace
• native build dependencies needed by the workspace crates

Install mdBook with Cargo when it is not already available:

cargo install mdbook

The local build does not publish anything and does not require live PostgreSQL, OAuth, DuckLake, HDSS, REDCap, or browser-auth services.

See the documentation and CI controls for DOCS_CHANNEL and other documentation/CI controls.

Public protocol API

ahri_tre_protocol owns the versioned JSON request and response envelopes used by CLI, daemon, Trusted runtime, C ABI, and language bindings.

Every request carries a protocol version, request identifier, kind, and typed body. Every response is either a typed success payload or a stable safe error. Unknown or removed operations fail explicitly.

Open the generated ahri_tre_protocol Rust API documentation.

The public contract includes:

• runtime/session status and configured profile selection;
• Datastore health and safe binding identity;
• domain, Study, governance, asset, datafile, dataset, variable, vocabulary, tag, entity, relation, transformation, and ingest workflows;
• operation receipts, pagination, warnings, and safe provenance;
• optional Arrow IPC payload descriptors for tabular results.

It excludes Secret material, credential names or paths, inline credentials, connection strings, raw SQL authority, Restricted local references, and live handles. Logical Secret references and exact resolved versions may appear only where safe provenance requires them.

The CLI exposes the implemented JSON schemas with schema list and schema get <schema-id>. The registry includes cli.schema.coverage-map.v1 for the current command-to-contract coverage map. Client bootstrap and Application documents have their own versioned configuration schemas.

The current registry identifiers are:

• cli.schema.coverage-map.v1
• protocol.request-envelope.v2
• protocol.response-envelope.v2
• protocol.success-envelope.v2
• protocol.failure-envelope.v2
• protocol.error.v2
• protocol.warning.v2
• protocol.mutation.v2
• protocol.pagination.v2
• protocol.refs.v2
• protocol.operation.v2
• protocol.tabular-ref.v2
• protocol.session.lifecycle.v2
• protocol.session.current-study.v2
• protocol.datastore.lifecycle.v2
• protocol.datastore.schema-status.v2
• protocol.daemon.readiness.v2
• protocol.domain.metadata.v2
• protocol.study.metadata.v2
• protocol.study.governance.v2
• protocol.asset.catalog.v2
• protocol.datafile.catalog.v2
• protocol.dataset.catalog.v2
• protocol.lifecycle.delete.v2
• protocol.ingest.workflow.v2
• protocol.transformation.audit.v2
• protocol.workflow.summary.v2
• protocol.dictionary.catalog.v2
• protocol.tag.registry.v2
• protocol.semantic-model.catalog.v2
• cli.daemon.lifecycle.v1
• cli.diagnostics.output.v1
• cli.version.output.v1
• cli.doctor.output.v1

Materialization acceptance

ingest.dataset.from_datafile returns a pending StartResult::Operation after atomic persistence of the operation, private attempt, and shared Dataset reservation. It pins the concrete source before acceptance. Poll operation.get or operation.result.get with the selected live Session; polling remains available while Lake work blocks. Results are operation_not_ready until terminalization. Worker saturation returns rate_limited without accepting work; Dataset contention returns a safe conflict with no other owner’s identifiers. An absent executor configuration returns unsupported_operation. The protocol request does not carry a wait flag: --no-wait is a CLI presentation choice.

Materialization cancellation

operation.cancel takes { "session": { "name": "analysis" }, "operation": { "id": "..." } } and returns OperationSummary. Every request rechecks the owner and required resource access. The ID is never an access capability. Cancellation remains responsive through an independent metadata capability.

cancellable reflects the current window. Pending and running work can accept cancel_requested; retries return that summary without another cancellation event. A blocking adapter call may finish before the next checkpoint. Cleanup and safe executor exclusion precede cancelled and output-reservation release. Cleanup failure returns a terminal safe cleanup_failed classification and retains private evidence and ownership. Terminal status alone cannot make the Dataset reusable.

Entering final admission atomically closes cancellation and publishes the committing stage. Cancellation then returns conflict. A request accepted after the last cooperative checkpoint may still finish completed or failed; the authoritative Dataset admission outcome wins. Completed, failed, and cancelled operations reject cancellation. A policy without cancellation support returns unsupported_operation. Configured runtimes advertise all four operation request kinds.

Operation retention and results

Terminal summaries, events and typed receipts expire exactly 30 days after finished_at; active work has no deadline. Get, event and result reads return not-found at operation expiry, and history omits expired rows before pagination and cursor lookahead, even when physical cleanup is delayed. Completed status is preserved if its result expires early, is removed, or becomes unavailable. The summary’s result reference and operation_result_unavailable detail distinguish expired, removed and unavailable; current source and output authority still apply. Successful result receipts include retention and availability alongside kind and data. Status and result text output display these protocol fields.

Idempotency protection lasts while active and until both acceptance plus 24 hours and the terminal operation deadline have passed. If public history expires inside that minimum key window, a retry returns not-found without executing new work. Expired keys may be reused only after the normal output-reservation checks. Startup and explicit Session opening perform bounded metadata housekeeping, at most 100 expired operations per pass. Housekeeping never releases reservations or deletes private cleanup evidence, Datafiles, Dataset versions, Transformation provenance or audit records. No public prune command or retention setting exists.