Introducing DuckLake on Layerbase

DuckDB 2 gives you a place to work with data in SQL, whether that data lives in a DuckDB file, a PostgreSQL database, or MariaDB. On Layerbase, you can start with a self-contained DuckDB 2 database, attach databases from your account, and save the results you want to keep in DuckLake.

DuckDB 1 is also available if you need it. You can keep using it for standalone databases and supported account sources. The managed DuckLake integration requires DuckDB 2; enabling it does not upgrade an existing DuckDB 1 database.

Managed DuckLake is available to eligible Layerbase accounts during rollout; general access depends on activation of the managed DuckLake feature flag. PostgreSQL holds the lake's catalog, and a private Cloudflare R2 bucket holds its Parquet data. You query the lake from the same DuckDB query console you use for your local tables and attached sources.

Start with DuckDB, add a lake when you need one

A self-contained DuckDB database is useful on its own. Import data, create tables, run analytical queries, and keep the results in its database file. A lake is optional.

DuckLake adds a different storage arrangement: a SQL catalog tracks the tables and their metadata, while Parquet files hold data in object storage. Layerbase manages the R2 storage and connects the lake to a PostgreSQL catalog you choose.

That gives you three places to work with data in one SQL session: the local DuckDB file, attached source databases, and the lake. You choose whether a query should read a source directly or save a copy for later analysis.

Six workflows, from a single file to an R2-backed lake

These are six ways to use the integration together. PostgreSQL and MariaDB are source databases; the managed lake uses a PostgreSQL catalog and R2 storage.

The source attachments are read-only. Layerbase creates a dedicated SELECT-only login, so your analytical connection cannot insert, update, or delete source rows.

Connect PostgreSQL and MariaDB

Start with a DuckDB 2 database and eligible PostgreSQL and MariaDB databases in the same personal account and server container. In DuckDB's Sources tab, add the PostgreSQL database with the alias pg_source, then add MariaDB as maria_source. Use Test to check each connection.

For a small example, create this table in PostgreSQL's query console, running each statement separately:

CREATE TABLE public.lake_demo (
  id INTEGER PRIMARY KEY,
  name TEXT,
  amount DECIMAL(10, 2)
);

INSERT INTO public.lake_demo VALUES
  (1, 'Ada', 12.34),
  (2, 'Bea', 56.78);

In MariaDB's query console, create a second table:

CREATE TABLE lake_demo (
  id INTEGER PRIMARY KEY,
  city VARCHAR(100),
  amount DECIMAL(10, 2)
);

INSERT INTO lake_demo VALUES
  (1, 'London', 10.25),
  (2, 'Paris', 20.50);

Back in DuckDB, query either source directly:

SELECT * FROM pg_source.public.lake_demo ORDER BY id;

SELECT * FROM maria_source.lake_demo ORDER BY id;

Or join them:

SELECT
  p.name,
  m.city,
  p.amount + m.amount AS combined_amount
FROM pg_source.public.lake_demo AS p
JOIN maria_source.lake_demo AS m USING (id)
ORDER BY p.id;

name  city    combined_amount
Ada   London  22.59
Bea   Paris   77.28

This reads the attached databases at query time. There is no CSV export or intermediate lake table to prepare before running the join.

Save the results in DuckLake

In the DuckLake section of Sources, select an eligible PostgreSQL database for the catalog and choose Enable DuckLake. Once active, the lake is available under the ducklake alias.

Copy the PostgreSQL data into a lake table:

CREATE TABLE ducklake.main.pg_snapshot AS
SELECT * FROM pg_source.public.lake_demo;

Then copy the MariaDB data:

CREATE TABLE ducklake.main.maria_snapshot AS
SELECT * FROM maria_source.lake_demo;

Read the saved tables using ordinary SQL:

SELECT 'PostgreSQL' AS source, count(*) AS rows, sum(amount) AS total
FROM ducklake.main.pg_snapshot
UNION ALL
SELECT 'MariaDB', count(*), sum(amount)
FROM ducklake.main.maria_snapshot;

source      rows  total
PostgreSQL  2     69.12
MariaDB     2     30.75

Each CREATE TABLE ... AS SELECT makes a saved copy. Later source changes do not automatically appear in that table. You control when to load new data; this is not continuous replication or change data capture.

DuckLake can keep small inserts in its catalog before writing Parquet files. In our staging verification, we explicitly flushed the example tables, then read the resulting objects directly from R2 and checked the same row counts and totals.

Your local DuckDB tables stay available

Attaching the lake leaves the local database file in place. Use fully qualified names to make the destination explicit. The managed local catalog is layerbase, and the lake catalog is ducklake:

CREATE TABLE layerbase.main.local_example AS
SELECT 42 AS answer;

SELECT * FROM layerbase.main.local_example;

This matters because the active lake becomes the default catalog in the current integration. An unqualified CREATE TABLE can create a lake table. Qualifying the catalog makes your storage choice clear in the query itself.

What the managed integration includes

The managed integration uses account sources in the same personal account and server container, a PostgreSQL catalog, and Layerbase-managed R2 storage. Shared teams, app backing stores, and databases in another container are not eligible account sources. Update, October 26: External PostgreSQL sources and your own R2/S3 bucket for lake storage are now available on Pro and Dedicated as Beta features, with no enrollment. Both are still a work in progress. See the DuckLake overview.

Read-only account sources are available on Free and paid plans, subject to database limits and the account requirements above. Managed DuckLake is included on Pro and Dedicated for eligible accounts. Managed DuckLake needs no invitation. Use the DuckLake controls in your eligible DuckDB 2 database to configure managed storage.

The lake's catalog and objects also have a different lifecycle from a standalone DuckDB file. Normal backup, branch, restore, move, and delete operations are blocked while a managed lake is retained, including after detaching it. A local DuckDB file backup is not a backup of the lake. Detaching preserves the catalog and objects; self-service lake recovery is not available, and recovery requires operator assistance.

Managed object storage is monitored separately from local database storage. The current 5 GiB operational target is advisory, not an enforced storage cap. Scheduled lake backups and self-service recovery are not included in this launch.

From live queries to saved analysis

Use standalone DuckDB when a single database file fits the job. Attach PostgreSQL or MariaDB when you need to read existing application data. Join those sources when the answer spans both. Save a lake table when you want to keep the result for later analysis.

Explore the managed DuckLake overview for workflows, requirements, and SQL examples. Start with DuckDB on Layerbase, or read the DuckDB documentation for connection details and the current availability of Sources and DuckLake.