Noqta
writing/blog/2026/07
● BlogJul 14, 2026·6 min read

DuckLake 1.0 and Iceberg v3: The 2026 Lakehouse Guide

DuckLake 1.0 is production-ready and DuckDB now reads and writes Iceberg v3. A practical guide to building a modern lakehouse with SQL and Parquet.

Noqta Team
·EN · FR · AR

For years, building a lakehouse meant assembling a small distributed system: a Spark cluster, a catalog service, a table format, and a compaction job to sweep up the mess they made together. In 2026, that assumption finally broke. Two releases a few weeks apart — DuckLake v1.0 in April and DuckDB-Iceberg v1.5.3 with full Iceberg v3 support in May — mean you can now run a serious, ACID-compliant lakehouse from a single-node engine you install with one command.

This guide walks through what changed, how the two formats differ, and how to choose between them for a real project.

A lakehouse, minus the cluster

A lakehouse stores data as open files (almost always Parquet) on object storage, while layering database guarantees on top: ACID transactions, schema evolution, time travel, and concurrent writers. The hard part was never the data — Parquet has been solid for a decade. The hard part is the metadata: knowing which files belong to which table version, and coordinating writers so they do not corrupt each other.

Apache Iceberg solved this with metadata files stored alongside the data, plus a catalog service to point at the current version. DuckLake, released by the DuckDB team, asks a simpler question: if you need a database to coordinate the catalog anyway, why not put all the metadata in that database and skip the file juggling entirely?

DuckLake 1.0: metadata lives in a database

DuckLake v1.0 shipped on April 13, 2026 as a production-ready specification with guaranteed backward compatibility. The reference implementation is the ducklake extension in DuckDB v1.5.2, already among DuckDB's top-10 core extensions by downloads.

The core design: data files are plain Parquet on object storage, and every piece of metadata — snapshots, schemas, file lists, statistics — lives as rows in a SQL catalog database. That catalog can be SQLite for a laptop project, PostgreSQL for a multi-writer team setup, or DuckDB itself.

Getting started takes three statements:

INSTALL ducklake;
 
ATTACH 'ducklake:postgres:dbname=lake_catalog host=db.internal'
    AS lake (DATA_PATH 's3://my-bucket/lake/');
 
CREATE TABLE lake.orders (id INT, status VARCHAR, total DECIMAL(10,2));

From there it behaves like a normal database — inserts, updates, deletes, joins across tables, all transactional — while the data lands as Parquet files anyone can read.

Data inlining kills the small-files problem

The flagship v1.0 feature is data inlining, now on by default. Small inserts, updates, and deletes are staged as rows in the catalog database instead of each producing a tiny Parquet file. A checkpoint later flushes them to object storage in sensible batches:

CREATE TABLE lake.t (id INT, status VARCHAR);
INSERT INTO lake.t VALUES (1, 'en route'), (2, 'shipped');
DELETE FROM lake.t WHERE id = 1;
UPDATE lake.t SET status = 'delivered' WHERE id = 2;
 
FROM ducklake_list_files('lake', 't');  -- returns empty: no files written yet
CHECKPOINT;                             -- now the data is flushed to Parquet

This is the "small changes" problem that plagues every file-based table format, solved by simply having a database available. It also makes streaming ingestion into a lake practical without a compaction service.

Sorted tables, bucket partitioning, and richer types

v1.0 also brings sorted tables (ALTER TABLE ... SET SORTED BY (id ASC)) for better file and row-group pruning, and bucket partitioning that deliberately uses the murmur3 hash for full compatibility with Iceberg:

ALTER TABLE lake.events SET PARTITIONED BY (bucket(8, user_name));

The type system grew too: a GEOMETRY type with bounding-box filter pushdown, and a VARIANT type for semi-structured data — binary-encoded, shreddable to primitive columns, and supporting dates and timestamps that JSON cannot represent. The DuckDB team believes VARIANT will eventually replace JSON as the default type for semi-structured data, and the Iceberg community made the same bet.

Iceberg v3: the spec catches up

Apache Iceberg v3 is the biggest revision of the dominant open table format in years, and 2026 is when it became real: generally available on Snowflake since May and on Databricks Runtime 18, with the wider ecosystem following. The headline features:

  • Binary deletion vectors. Instead of v2's positional delete files, each data file gets a compressed Roaring bitmap (stored in Puffin files) marking deleted rows — dramatically faster merge-on-read and far fewer small files.
  • VARIANT type. The same binary-encoded semi-structured type DuckLake adopted, defined in the Parquet project, with pushdown-friendly shredding.
  • Row lineage. Row-level tracking of when rows were added and modified, enabling incremental processing and change feeds.
  • Default column values and TIMESTAMP_NS. Schema evolution without rewriting files, and nanosecond timestamps.

If you run Iceberg tables today, v3 is the upgrade that makes frequent updates and deletes affordable.

DuckDB as an Iceberg v3 client

The second piece of the story is that DuckDB-Iceberg v1.5.3 turned DuckDB into a genuine read-write Iceberg client — implemented natively in the extension, with no dependency on third-party Iceberg libraries or a JVM. Against any Iceberg REST catalog (Polaris, Lakekeeper, Amazon S3 Tables, Glue), DuckDB now supports MERGE INTO, ALTER TABLE schema evolution, bucket and truncate partition transforms, and full Iceberg v3 reads and writes:

ATTACH 'warehouse' AS my_datalake (TYPE iceberg, ...);
 
-- Create a v3 table with VARIANT and nanosecond timestamps
CREATE TABLE my_datalake.default.v3_table
WITH ('format-version' = 3) AS
FROM (VALUES
    (1, {'kind': 'click', 'x': 10}::VARIANT, TIMESTAMP_NS '2026-05-20 12:00:00.123456789'),
    (2, {'kind': 'view'}::VARIANT,           TIMESTAMP_NS '2026-05-20 12:00:00.987654321')
) t (id, payload, event_time);
 
-- Deletes are written as compact binary deletion vectors
DELETE FROM my_datalake.default.v3_table WHERE id = 1;
 
-- Upgrade an existing table in place
ALTER TABLE my_datalake.default.renamed_table SET ('format-version' = 3);

MERGE INTO deserves a special mention: it is the recommended way to express upserts against tables without primary keys — which is every lakehouse table — and it now works end to end against Iceberg from DuckDB.

DuckLake or Iceberg? Both, actually

The practical decision in 2026 is less either-or than it looks:

  • Choose DuckLake when your team controls the stack and wants the simplest possible operations: one Postgres database as catalog (perhaps the PostgreSQL 18 instance you already run), Parquet on any object store, no extra services. Streaming small writes, multiplayer DuckDB setups, and read-only lakes served over plain HTTPS all work out of the box.
  • Choose Iceberg when you need maximum ecosystem reach — Snowflake, Databricks, Spark, Trino, and Flink all speak it — or when the tables already exist and other teams depend on them.
  • Interop is designed in. DuckLake stores data as Parquet with Iceberg-compatible bucket hashing and Puffin deletion vectors. The iceberg_to_ducklake function performs a metadata-only migration of an entire Iceberg catalog into DuckLake — no data files copied — and copying from DuckLake back to Iceberg is supported too.

A pattern we increasingly recommend: DuckLake as the fast-moving operational layer owned by one team, publishing curated tables to Iceberg where the wider organization consumes them.

Why this matters for lean teams

For startups and SMEs in Tunisia, the Gulf, and beyond, the historical blocker on lakehouse architecture was operational cost: clusters to babysit and catalog services to secure, for data volumes that rarely justified them. That excuse is gone. A DuckDB process, a Postgres catalog, and an S3-compatible bucket now deliver ACID tables, time travel, and open formats — terabytes of data and millions of snapshots are explicitly within spec. If you are building an enterprise data strategy for AI, this stack is the cheapest credible foundation we have ever been able to recommend.

Planning a data platform, or migrating analytics off spreadsheets and into a lakehouse? Talk to the Noqta team — we design and build data infrastructure for teams across MENA.