Back to docs
Concepts

Data Quality & Expectations

Dagy lets tasks declare **expectations** about their data and records the results, so bad data is caught (and optionally blocked/quarantined) instead of propagating.

Expectations (SDK)

dagy.expectations (src/dagy/expectations.py) defines the checking model:

  • Each declared check produces a CheckResult with a severity.
  • Results roll up into a QualityReport with ok, passed, failures, warnings, quarantined, and pass_rate.
  • A blocking failure raises DataQualityError, which carries the full report.

severity (via the Severity enum) distinguishes warnings from blocking failures, and results can mark rows/partitions as quarantined.

Recording results (API)

The quality service (src/dagy_api/quality/service.py) persists results:

  • record_check_results(...) is the executor integration hook — call it after evaluating a task's expectations. It accepts either dagy.expectations.CheckResult objects or plain dicts.
  • Results are stored in the table named by DAGY_QUALITY_TABLE. With no table configured, every function no-ops (_table_enabled() guard), so the module is import-safe.

Query API

Method & pathPurpose
GET /quality/catalogavailable check types
GET /quality/runs/{run_id}quality results for a run
GET /quality/summaryrolled-up pass/fail over a time window
GET /quality/checkschecks configured across the org's flows