Architecture

Converts SQL text into an AST. DuckDB uses a fork of PostgreSQL's battle-tested parser, which provides broad SQL compatibility. The parser is completely catalog-unaware — it produces SQLStatement, QueryNode, TableRef, and ParsedExpression nodes without resolving any names or types. This separation exists because parsing is a syntactic concern; mixing in catalog lookups would create circular dependencies.

Resolves names and types against the catalog. Transforms parsed nodes into bound equivalents (SQLStatement → BoundStatement, ParsedExpression → Expression) by looking up table names, resolving column references, checking types, and expanding wildcards (SELECT *). Supports CTEs, subqueries, and correlated references through a hierarchical binder chain where child binders inherit parent context.

Produces a tree of LogicalOperator nodes representing the query's intent without specifying how to execute it. Operators include LogicalGet (scan), LogicalFilter, LogicalProjection, LogicalAggregate, LogicalJoin, and more. This abstract representation is what the optimizer works on.

The heart of query performance. DuckDB runs 30+ sequential optimization passes on the logical plan, including:

• Filter Pushdown — moves predicates close to the data source to reduce I/O
• Join Order Optimization — uses the DPccp algorithm for dynamic-programming-based join enumeration
• Column Pruning — removes columns never referenced downstream
• Common Subexpression Elimination — deduplicates repeated computation
• Late Materialization — defers column reads until actually needed
• Statistics Propagation — uses cardinality estimates to guide operator selection
• TopN Optimization — converts ORDER BY ... LIMIT N into a single operator

The optimizer is extensible — extensions can inject custom pre- and post-optimization passes.

Converts the optimized logical plan into a tree of PhysicalOperator nodes that specify concrete execution strategies. For example, a LogicalJoin might become a PhysicalHashJoin or a PhysicalMergeJoin depending on the join condition, data sizes, and available indexes. The physical planner also resolves column bindings from logical column references to physical vector indices.

Arranges physical operators into pipelines and schedules them for parallel execution. The executor uses an event-driven model where pipeline completion triggers downstream pipelines. Each pipeline has a source, zero or more intermediate operators, and a sink. Worker threads from the task scheduler each grab a morsel of work and push it through the pipeline independently.

Manages all database metadata: schemas, tables, views, functions, types, sequences, and extensions. DuckDB supports attaching multiple databases simultaneously (including read-only access to external databases like PostgreSQL, MySQL, and SQLite via extensions), and the catalog provides a unified namespace across all attached databases.

Manages persistent and in-memory data using a single-file format. Data is organized into row groups (~122,880 rows each), with each column in a row group compressed independently. The storage engine includes a buffer manager for out-of-core processing (datasets larger than RAM), a WAL for crash recovery, and checkpoint logic for compacting the WAL into the main database file.

Manages a thread pool that executes pipeline tasks. The scheduler supports both fully parallel pipelines (multiple threads on different data morsels) and sequential pipelines (when operator semantics require ordered execution). It also handles inter-pipeline dependencies through an event system.