Skip to content
R Roesli.
Go back
dbt

A native Fabric analytics stack with dbt, Airflow, Lakehouse, and Direct Lake

Build Jaffle Shop entirely in Microsoft Fabric with a Lakehouse-native dbt Job, Fabric Apache Airflow orchestration, and a Direct Lake semantic model.

The interesting change is not that dbt disappeared. It is that Fabric can now run a Lakehouse dbt Job, orchestrate it with managed Apache Airflow, and serve the Delta output through Direct Lake without an external dbt runner or a Warehouse target.

Jaffle Shop CSV seeds
        |
        v
Fabric dbt Job (dbt-fabricspark)
        |
        v
Fabric Lakehouse / Delta tables
  Raw -> Staging -> Gold
        |
        +----------------------+
        |                      |
        v                      v
Fabric Apache Airflow     Direct Lake semantic model
orchestration             and Power BI

Preview boundary: Lakehouse support in Fabric dbt Jobs is preview. The tested runtime uses dbt-fabricspark 1.12.2, dbt Core 1.11, and Python 3.12. dbt-fabric 1.10.0 targets Fabric Warehouse and is not interchangeable.

What the walkthrough creates

LayerFabric itemPurpose
Storagejaffle_shop_lakehouseDelta tables in OneLake
Transformationjaffle_shop_native_dbtManaged dbt execution
Orchestrationjaffle_shop_airflow_orchestratorDAG that runs the dbt Job
Consumptionjaffle_shop_direct_lakeStar-schema semantic model

The Gold contract is:

Why the adapter choice matters

A Lakehouse SQL analytics endpoint is read-only. The Lakehouse route uses dbt-fabricspark, Spark SQL, and Delta tables written through Fabric Livy.

Warehouse routeLakehouse route
dbt-fabric 1.10.0dbt-fabricspark 1.12.2
T-SQLSpark SQL
Warehouse objectsOneLake Delta objects
SQL target is writableSQL analytics endpoint remains read-only

There is no pip install step in the managed job. Select the Lakehouse profile and keep the adapter-specific project syntax correct.

Prerequisites

You need:

  1. a Fabric capacity or trial supporting the workloads;
  2. Contributor or higher workspace access;
  3. dbt Jobs (preview) enabled;
  4. capacity support for Apache Airflow Jobs;
  5. permission to create the Lakehouse, dbt Job, Airflow Job, and semantic model;
  6. an identity-backed Fabric connection for Airflow.

Free and PPU workspaces do not support Airflow Jobs.

1. Create the workspace and Lakehouse

Create a capacity-backed workspace named DBTAirflow, then:

jaffle_shop_lakehouse

Use logical schemas:

raw
staging
gold

If the tenant exposes only the default schema, keep those logical boundaries in dbt folders and tags and materialize them in the supported target schema.

2. Create the dbt Job

In DBTAirflow:

  1. create a dbt Job named jaffle_shop_native_dbt;
  2. import or create the project;
  3. choose Fabric Lakehouse;
  4. select jaffle_shop_lakehouse;
  5. select the supported target schema;
  6. enable seed data; and
  7. use runtime V1.0.

Project configuration:

name: jaffle_shop
version: 1.0.0
config-version: 2

profile: jaffle_shop

model-paths: ["models"]
seed-paths: ["seeds"]
macro-paths: ["macros"]

models:
  jaffle_shop:
    staging:
      +schema: staging
      +materialized: view
    marts:
      +schema: gold
      +materialized: table

seeds:
  jaffle_shop:
    +schema: raw

Fabric manages the profile. Do not commit a password-bearing profiles.yml.

3. Seed and model the data

Place these under seeds:

raw_customers.csv
raw_orders.csv
raw_payments.csv

Run dbt seed or enable seeds in dbt build.

Payment amounts are cents, so the staging model converts them:

select
    cast(id as bigint) as payment_id,
    cast(order_id as bigint) as order_id,
    cast(payment_method as string) as payment_method,
    cast(amount as decimal(18,2)) / 100 as amount
from {{ ref('raw_payments') }}

The thin staging models are:

stg_customers
stg_orders
stg_payments

They standardize keys, dates, statuses, and payment amounts. Aggregation belongs in Gold.

The Gold model dependency graph is:

raw_customers -> stg_customers -> dim_customers
raw_orders    -> stg_orders    -> dim_customers
                              -> dim_dates
                              -> fct_orders
raw_payments  -> stg_payments  -> fct_orders

The customer dimension uses the tested pattern:

with customer_orders as (
    select
        customer_id,
        min(order_date) as first_order_date,
        max(order_date) as most_recent_order_date,
        count(*) as number_of_orders
    from {{ ref('stg_orders') }}
    group by customer_id
),
customer_payments as (
    select
        o.customer_id,
        sum(p.amount) as lifetime_value
    from {{ ref('stg_orders') }} o
    left join {{ ref('stg_payments') }} p
        on o.order_id = p.order_id
    group by o.customer_id
)

select
    c.customer_id,
    c.first_name,
    c.last_name,
    co.first_order_date,
    co.most_recent_order_date,
    coalesce(co.number_of_orders, 0) as number_of_orders,
    coalesce(cp.lifetime_value, cast(0 as decimal(18,2))) as lifetime_value
from {{ ref('stg_customers') }} c
left join customer_orders co
    on c.customer_id = co.customer_id
left join customer_payments cp
    on c.customer_id = cp.customer_id

The date dimension contains distinct order dates:

select distinct
    order_date as date_day,
    year(order_date) as calendar_year,
    month(order_date) as month_number,
    day(order_date) as day_of_month
from {{ ref('stg_orders') }}

4. Make quality part of the run

Test unique and non-null keys plus customer and date relationships. Use:

dbt build

For this project:

SettingValue
Operationbuild
Threads4
Fail fastenabled during development
Full refreshdisabled for normal runs

5. Orchestrate with Airflow

Add Airflow when dbt is one step in a wider workflow: ingestion, semantic model processing, notifications, SLAs, or cross-item dependencies. If dbt is the whole workflow, its own scheduler is simpler.

Create:

jaffle_shop_airflow_orchestrator

Enable triggerers and attach the workspace-identity Fabric connection. The operator expects the connection GUID, not its display name:

from datetime import datetime, timedelta

from airflow import DAG
from airflow.providers.microsoft.fabric.operators.run_item import (
    MSFabricRunJobOperator,
)

FABRIC_CONN_ID = "<Fabric connection GUID>"
WORKSPACE_ID = "<DBTAirflow workspace ID>"
DBT_JOB_ID = "<jaffle_shop_native_dbt item ID>"

with DAG(
    dag_id="orchestrate_jaffle_shop_dbt",
    schedule=None,
    start_date=datetime(2026, 1, 1),
    catchup=False,
    default_args={
        "owner": "fabric",
        "retries": 1,
        "retry_delay": timedelta(minutes=5),
    },
) as dag:
    run_dbt_build = MSFabricRunJobOperator(
        task_id="run_jaffle_shop_dbt",
        fabric_conn_id=FABRIC_CONN_ID,
        workspace_id=WORKSPACE_ID,
        item_id=DBT_JOB_ID,
        job_type="DataBuildToolJob",
        wait_for_termination=True,
        deferrable=True,
    )

job_type="Execute" is wrong here. Execute is the control-plane operation name, not the Airflow provider value. Use DataBuildToolJob or DBT.

6. Create the Direct Lake model

Create jaffle_shop_direct_lake from the Lakehouse and select only:

Relationships:

FromToCardinality
fct_orders[customer_id]dim_customers[customer_id]Many-to-one
fct_orders[order_date]dim_dates[date_day]Many-to-one

Measures:

Total Orders =
DISTINCTCOUNT(fct_orders[order_id])
Total Revenue =
SUM(fct_orders[amount])
Average Order Value =
DIVIDE([Total Revenue], [Total Orders])

Direct Lake reads Delta data from OneLake. Schema changes still require model management.

7. Validate every contract

Lakehouse checks:

select count(*) as order_count
from gold.fct_orders;
select
    sum(amount) as total_revenue,
    min(order_date) as first_order_date,
    max(order_date) as last_order_date
from gold.fct_orders;
select count(*) as orphan_customers
from gold.fct_orders f
left join gold.dim_customers c
    on f.customer_id = c.customer_id
where c.customer_id is null;

DAX smoke test:

EVALUATE
ROW(
    "Orders", [Total Orders],
    "Revenue", [Total Revenue],
    "Average Order Value", [Average Order Value],
    "Customers", COUNTROWS('dim_customers')
)

The validated deployment returned:

ValidationResult
dbt resources passed23
warnings / errors / skips0 / 0 / 0
Customers10
Dates11
Orders12
Total revenue175.00
Average order value14.5833
Orphan customer keys0
Orphan date keys0
Airflow DAGSuccess
Airflow-triggered dbt runCompleted

The Airflow item, workspace-identity connection, Direct Lake model, enhanced refresh, framing, and DAX smoke test were also validated. No secret is in the DAG or project files.

Constraints

  1. Lakehouse dbt support is preview.
  2. The managed runtime is curated and has no build cache.
  3. Spark SQL is not T-SQL.
  4. Adapter support for packages, hooks, snapshots, and incremental models needs its own validation.
  5. The SQL analytics endpoint is read-only.
  6. Airflow has capacity, networking, and authentication constraints.
  7. Direct Lake treats Gold schemas as contracts.

Final architecture

Use the full stack when each layer adds value. If dbt is the only step, skip Airflow. If the workload is SQL-first, choose a Warehouse. Native does not mean every workload belongs in the same path.

References


Share this post:

Continue exploring

Previous Post
The agent runbook for native Fabric dbt, Airflow, Lakehouse, and Direct Lake
Next Post
Use a managed dbt job in Microsoft Fabric
Community

Join the conversation

Sign in with GitHub to leave a comment.

GitHub

Loading comments…

Sign in with GitHub to comment