███╗   ███╗███████╗██████╗ ██╗██████╗ ██╗ █████╗ ███╗   ██╗
████╗ ████║██╔════╝██╔══██╗██║██╔══██╗██║██╔══██╗████╗  ██║
██╔████╔██║█████╗  ██████╔╝██║██║  ██║██║███████║██╔██╗ ██║
██║╚██╔╝██║██╔══╝  ██╔══██╗██║██║  ██║██║██╔══██║██║╚██╗██║
██║ ╚═╝ ██║███████╗██║  ██║██║██████╔╝██║██║  ██║██║ ╚████║
╚═╝     ╚═╝╚══════╝╚═╝  ╚═╝╚═╝╚═════╝ ╚═╝╚═╝  ╚═╝╚═╝  ╚═══╝

the agentic iceberg rest catalog

A catalog that lives in the write path and the read path of every engine, and does the other five products there.

Meridian is an open-source, Apache Iceberg-native REST catalog. It's a drop-in IRC for Spark, Trino, Flink, DuckDB, and PyIceberg. It builds autonomous maintenance, cross-engine governance, observability & contracts, an engine-agnostic semantic layer, and a governed gateway for AI agents directly into the catalog, where the transaction actually is.

▶ run it in 60 seconds ★ star on GitHub why it's different

Apache-2.0Rust corePostgreSQL is the only dependencysingle binaryalpha

meridian@localhost · try: help, pillars, whoami, why

51.4779°N
0.0015°W

why it's different

The structural insight the whole thing is built on:

        ┌── writes ──────────────┐        ┌────────────── reads ──┐
 Spark ─┤                        │        │                       ├─ DuckDB
 Trino ─┤   every commit,        ▼        ▼   every load, vend,   ├─ PyIceberg
 Flink ─┤   atomically   ┌───────────────────┐  scan plan         ├─ agents
        │  ───────────▶  │   M E R I D I A N   │  ◀───────────       │
        │   write path   │  the control point │  read path          │
        └────────────────└─────────┬─────────┘────────────────────┘
                                    │
                          ┌─────────▼─────────┐
                          │  object storage   │   (S3 / GCS / Azure / MinIO / fs)
                          │  Iceberg tables   │   · your data, your bucket
                          └───────────────────┘

Nothing else in the stack touches every change and every access.

Every tool that bolts governance, quality, or lineage on from the outside is approximating that position with connectors, periodic crawls, and query-log scraping, and paying for it in staleness, blind spots, and per-engine integration tax. A catalog does these things natively, transactionally, with zero instrumentation. That isn't a marginal edge; it's a different physics. And the parts compound, because they share one substrate: lineage powers contract impact-analysis powers agent trust; maintenance telemetry powers the savings ledger. Point solutions can't compound; they don't share a substrate.

the
“wait,
it does
THAT?”

things a catalog isn't supposed to do

the circuit breaker

Stop bad data before it lands.

A data contract is enforced at commit time: a schema-violating write is rejected atomically, quarantined to an audit branch, or waved through with an incident. Observability vendors sell faster alarms. Meridian sells the fire not starting.

scan-plan security

Policy applied inside the plan.

Row filters compile into the residual every planning client must apply; an ABAC deny aborts planning with a 403 before a file is returned. It's the mechanism the walled gardens keep for themselves, from a neutral catalog. Column masks strip the column's stats from the plan; what each engine prevents vs detects is stated plainly in the enforcement matrix. No overclaim.

repairs what engines emit

Compaction that keeps every row.

Flink sprays small files; some engines write ugly layouts. Meridian rewrites them to house style through a normal, audited, revertible commit: every row preserved, verified by reading the rewritten Parquet back through real Iceberg manifests and comparing row-for-row.

universal views

One view, every dialect.

Author a view once; every engine reads it in its own SQL, transpiled and labeled verified / best-effort / unsupported. The five-year-old cross-engine view bug, closed.

branch-as-catalog

Branches any engine can use.

A branch mounts as warehouse@branch, so every IRC engine reads and writes zero-copy dev environments on prod data without knowing branching exists. A branch commit leaves main byte-for-byte untouched.

the agent firewall

A real perimeter for AI agents.

Every agent: an identity, a purpose, a budget, a kill switch, and a court-grade audit trail, on columns and rows, not just tool calls. Governed context returns restricted columns absent, so a prompt can't even learn their names.

A → L

the manifest: twelve pillars, one binary

Every pillar below is built and test-backed, with the partial and deferred pieces tracked honestly, per feature, in docs/status.md.

Acore catalogfull IRC surface on an atomic, crash-safe commit path · OIDC · deny-by-default RBAC · hash-chained audit

Bfederationmirror external IRC catalogs as read-only foreign assets · sprawl dashboard (Glue is a planned source)

Ctable opshealth model · built-in compaction & expiry through the commit path · savings ledger

Dgovernancerow filters & Cedar ABAC enforced inside scan planning; column masks withhold stats (see enforcement matrix)

Eobservabilityzero-scan monitors from the commit stream · data contracts · the circuit breaker

Flineagecommit-native lineage · OpenLineage in/out · impact analysis + CI gate

Gsemanticsuniversal views (SQLGlot sidecar) · metrics · glossary · certified data products

Hagent gatewayMCP-native · governed context & query tools · budgets · kill switch · full audit chain

Iai assetsfilesets · model registry · training-run pinning to exact snapshots

Jsharingread-only cross-org shares over a per-share IRC endpoint · instant revocation with STS vending

Kbranchingcatalog branches & tags · branch-as-catalog · contract-gated merges

Lworkbenchgoverned SQL over a small-scan executor · Next.js console · Terraform · catalog-as-code

$ _

run it in 60 seconds

1. clone it, then bring up Postgres + the server (migrations run on start)

git clone https://github.com/meridian-catalog/meridian.git cd meridian # everything in Docker docker compose -f docker-compose.dev.yml up --build curl -s localhost:8181/healthz # {"status":"ok","checks":{"database":"ok"}}

2. create a warehouse (this is the IRC prefix)

curl -s -X POST localhost:8181/api/v2/warehouses \ -H 'content-type: application/json' \ -d '{"name":"demo","storage_root":"file:///tmp/meridian-demo"}'

3. point any Iceberg engine at it

from pyiceberg.catalog.rest import RestCatalog cat = RestCatalog("demo", uri="http://localhost:8181/iceberg", warehouse="demo") cat.create_namespace("analytics") # create tables, append data, time-travel, evolve schema. it's a normal IRC catalog

Running from source, the web console, the S3/MinIO setup, and Spark/Trino/Flink/DuckDB examples are in docs/dev.md and conformance/.

the
proof

engines it's actually been run against

engine	create	write	evolve	read	row-level	views
PyIceberg	✔	✔	✔	✔	-	✔
DuckDB	-	-	-	✔	-	-
Spark 3.5	✔	✔	✔	✔	✔ MoR	±
Trino 482	✔	✔	✔	✔	✔ x-engine	✔
Flink 1.20	✔	✔	-	✔	-	-

✔ passing · ± passing with a documented gap · - not exercised. Spark writes merge-on-read MERGE/DELETE and Trino reads the result back exactly (cross-engine). Full honest matrix & gaps: conformance/engines.

honest
to a
fault

status: what's true, what isn't

We keep the doubts in public. Genuinely not built yet:

GCS and Azure credential vending (those clouds return a clear unsupported error; AWS S3 vending & remote signing work)
the Iceberg REST compatibility kit (RCK) has not been run; conformance rests on the engine matrix today
column-level lineage from SQL-log parsing, classification scanners, SCIM, and a Trino OPA bridge are tracked follow-ups

The full, self-critical feature matrix is docs/status.md, where every “Implemented” is backed by a test. Trust is the product; the docs say what's real.

how
it's
built

architecture

Rust coreone service (axum · sqlx · tokio) serving the IRC + management APIs and hosting every layer
PostgreSQL onlyall catalog state lives in Postgres; no queue, cache, or coordinator to run. Object storage is yours.
single binaryone process for a working catalog; a Helm chart and BYOC topology for later
Python sidecaroptional SQLGlot service for cross-dialect SQL, with an off-by-default, key-gated, always-validated LLM-assist fallback
web consolea Next.js/TypeScript UI for catalog, governance, quality, lineage, and agents

The repo is a Cargo workspace (catalog core, the Iceberg metadata/manifest engine, storage IO, vending, authz, federation, lineage, agents, a small-scan query executor), the sidecar, the console, a Terraform provider, and an engine conformance harness. Decisions live in docs/adr/.