Dagster 1.6 New Features — A Field Guide

Dagster's pitch is simple to state and easy to underrate: orchestrate assets, not tasks. Most schedulers, Airflow included, are built around "run this script, then run that script." Dagster is built around "here is the dataset I produce, and here is what it depends on" — and that shift in the core abstraction changes how you write pipelines, debug them, and reason about data quality.

Software-defined assets: the core abstraction

A software-defined asset (SDA) in Dagster is a Python function decorated with @asset that describes a persistent object — a table, a file, a model — and the code that produces it. Instead of defining a DAG of opaque steps and inferring what data they touch, you declare the data itself, and Dagster infers the dependency graph from the function signatures: if orders_cleaned takes raw_orders as an argument, Dagster knows orders_cleaned depends on raw_orders without you wiring an explicit edge.

from dagster import asset

@asset
def raw_orders() -> None:
    ...  # extract from source system

@asset
def orders_cleaned(raw_orders) -> None:
    ...  # transform, Dagster infers this depends on raw_orders

This matters in practice because the dependency graph in a task-based orchestrator is a graph of execution order, not a graph of data lineage. In Dagster the two are the same graph, which is what makes the rest of the model — lineage, freshness, selective materialization — fall out for free instead of being bolted on.

Materialization and the asset graph

Running an asset's function and persisting its output is called materializing it. You can materialize a single asset, a subset of the graph, or everything downstream of a given asset — because Dagster knows exactly what depends on what, it can compute the minimal set of work needed to bring stale assets up to date. This is the practical payoff of modeling data instead of tasks: "rerun everything downstream of the orders table" is a selection query on the asset graph, not a manual list of DAG nodes you have to keep in sync by hand.

Asset checks for data quality

Asset checks let you attach validation directly to an asset — row count thresholds, null checks, referential integrity, schema conformance — and have Dagster run them as part of materialization and surface pass/fail status right next to the asset in the UI. The distinction from a generic test suite is that checks are scoped to the asset they validate and travel with it: anyone looking at the asset graph sees whether the data currently backing an asset is trusted, not just whether the pipeline "succeeded" in some generic sense.

Partitioned assets

Partitioning lets an asset represent a family of materializations sliced by time, category, or another dimension — a daily-partitioned asset has one logical materialization per day rather than one big table rebuilt from scratch every run. This is what makes backfills tractable: instead of a monolithic re-run, you can target specific partitions (a date range, a failed subset) and Dagster tracks materialization status per partition, so you always know which slices are current and which are stale.

Sensors, schedules, and the Dagster UI

Schedules trigger materializations on a cron-like cadence. Sensors trigger them in response to events — a new file landing in object storage, an upstream asset becoming stale, a signal from an external system — which covers the event-driven cases a pure cron schedule can't. Both are visible, along with the full asset graph, materialization history, and check results, in the Dagster UI (historically called Dagit), which is where the asset-centric model earns its keep operationally: you can look at the graph and see which assets are fresh, which are stale, which checks are failing, and what a targeted rerun will actually touch before you kick it off.

How this differs from task-based orchestrators

Airflow's core unit is the task, and a DAG is a graph of task execution order that you wire explicitly with >> operators or dependency lists; what each task actually produces is opaque to the scheduler. Dagster's core unit is the asset, and the execution graph is derived from what each asset declares it depends on. The practical difference shows up in observability and maintenance: in a task-based system, "is this table up to date and trustworthy" requires separate tooling layered on top; in Dagster it's a property the orchestrator already tracks, because the thing being orchestrated is the data, not the script that happens to produce it.

Wrapping up

The asset-first model is Dagster's real differentiator, not any single feature release. If you're evaluating it against Airflow or Prefect, the question worth asking isn't which one has more integrations — it's whether you want your orchestrator to know what data it's producing, or just what scripts it ran. For pipelines where lineage, targeted backfills, and inline data-quality checks matter, that difference compounds fast.