Smart CDC

Write once. Run incrementally forever.

Write normal SQL. OndatraSQL automatically processes only changed data.

No is_incremental(). No manual filters. No extra logic.

Mental Model

Your query, but only for new data. You don’t change your SQL — OndatraSQL changes how it runs.

Before and After

Without Smart CDC:

SELECT * FROM raw.events
WHERE event_time > (SELECT MAX(event_time) FROM target)

Manual logic. Easy to get wrong.

With Smart CDC:

-- @kind: append

SELECT * FROM raw.events

That’s it.

How It Works

OndatraSQL:

1. Finds your source tables (from SQL AST)
2. Checks what changed since last run (DuckLake snapshots)
3. Runs your query only on that data

First run: all rows. Every run after: only changes.

Example

-- @kind: merge
-- @unique_key: customer_id

SELECT * FROM raw.customers

• First run → all rows
• Next runs → only changed rows

Not All Tables Are Treated the Same

• Main table → only changed rows
• Joins → adjusted for correctness

You always get correct results — even with joins and aggregations.

Where It Applies

Smart CDC works with:

• append
• merge
• partition

Other kinds use different strategies:

• scd2 → full comparison (needs complete source)
• table → dependency tracking (skip or rebuild)

Why This Matters

Most tools require writing incremental filters, managing state manually, and debugging edge cases.

Smart CDC removes all of that.

Write once. Run incrementally forever.