Expand description
DuckLake-backed storage adapter for AHRI TRE datasets and data files.
This crate owns Lake location layout, DuckDB/DuckLake attachment, dataset table loading/export, and staged data-file materialization.
Re-exports§
pub use adapter::AwsWorkloadIdentitySource;pub use adapter::DuckLakeAdapter;pub use adapter::DuckLakeAttachPlan;pub use adapter::DuckLakeCredentialKind;pub use adapter::DuckLakeHealth;pub use adapter::DuckLakeLocalLayout;pub use adapter::DuckLakeOpenOptions;pub use adapter::DuckLakeOpenedCatalog;pub use adapter::DuckLakeTlsGuard;pub use adapter::FilesystemCatalogTls;pub use adapter::FilesystemLakeSessionConfig;pub use adapter::ObjectLakeSessionConfig;pub use adapter::ObjectStorageAuthentication;pub use adapter::ObjectStorageLocation;pub use adapter::ObjectStorageTls;pub use adapter::S3UrlStyle;pub use alias::LAKE_ALIAS;pub use dataset_loading::DatasetCsvSeed;pub use dataset_loading::DatasetExportFormat;pub use dataset_loading::DatasetFileFormat;pub use dataset_loading::DatasetFileReadOptions;pub use dataset_loading::DatasetTableRelation;pub use dataset_loading::ExportDatasetTableRequest;pub use dataset_loading::ExportedDatasetTable;pub use dataset_loading::ImportedColumn;pub use dataset_loading::LoadDatasetTableFromFileRequest;pub use dataset_loading::LoadDatasetTableFromSqlRequest;pub use dataset_loading::LoadRedcapFormDatasetRequest;pub use dataset_loading::LoadedDatasetTable;pub use dataset_loading::PreviewDatasetTableRequest;pub use dataset_loading::PreviewedDatasetTable;pub use dataset_loading::RedcapEavChoice;pub use dataset_loading::RedcapEavField;pub use dataset_loading::RedcapSyntheticColumn;pub use error::LakeError;pub use file_staging::DataFileCompression;pub use file_staging::DataFileEncryption;pub use file_staging::DeleteDataFileRequest;pub use file_staging::DeletedDataFile;pub use file_staging::ExportDataFileRequest;pub use file_staging::ExportedDataFile;pub use file_staging::MaterializeDataFileRequest;pub use file_staging::MaterializedDataFile;pub use file_staging::StageDataFileRequest;pub use file_staging::StageDataFileStreamRequest;pub use file_staging::StagedDataFile;pub use scratch::ScratchAttempt;pub use scratch::ScratchAttemptId;pub use scratch::ScratchError;pub use scratch::TrustedScratch;pub use dataset_loading::dataset_output_key;
Modules§
- adapter
- alias
- Shared DuckLake catalog alias used in generated SQL.
- dataset_
loading - Dataset table loading and export helpers for DuckLake relations.
- error
- file_
staging - preflight
- Read-only configured Lake prerequisites. These checks never open DuckDB, attach a catalog, resolve credentials, or create scratch/TLS files.
- scratch
Structs§
- Catalog
Observation - Dataset
Executor - One process generation, held by Sessions and every outstanding Lake attempt. The stable lock file lives in durable coordination storage outside source scratch.
- Disclosure
Capture Authority - Disclosure
Datafile Stream - Disclosure
Dataset Input - Governance
Ledger - Lake
Catalog Observations - Operation-local evidence only. No handles, credentials, driver messages or persistent history. An unattempted stage has no fabricated success.
- Prepared
Dataset Source - Restored source bytes and intermediate files owned by one private attempt. The restricted path remains inside the trusted workflow and Lake adapter.
- Prepared
Disclosure Datafile - Prepared
Disclosure Query - Prepared
Query Output - Redcap
Execution Limits - Session-exclusive form work uses a finite engine budget; the prior settings are restored before the Session can serve another workflow.
- Redcap
Upload - Redcap
Upload Role - Restricted
Query Plan - Restricted
Query Stream - Dropping a stream kills and reaps its worker before releasing private inputs.
- SqlDatabase
Endpoint - One policy-approved socket and its original TLS identity. No connection string, database override, initialization SQL, or ambient login is accepted.
- SqlSource
Plan - SqlSource
Read - Upload
Validation
Enums§
- Catalog
Observation Result - Query
Binding - SqlRead
Source - Local source authority stays in the acquiring process and its private worker. This type is never part of a Trusted request or a persisted source record.
Functions§
- analyze_
dataset_ transform - Bind the existing Dataset SQL-transform source using its canonical relation and logical alias. Any additional or external relation must fail resolution.
- analyze_
disclosure_ query - attest_
datafile - Called by a trusted writer before publishing metadata, or by the operator during legacy conversion. Ordinary metadata UPDATE cannot create this proof.
- dataset_
physical_ identity - Captures the committed immutable physical identity while the caller retains its writer reservation (or the operator holds an upgrade maintenance window).
- run_
datafile_ capture_ worker - Private worker entry point; accepts no user program, URL or destination.