Concepts

Metadata Catalog & Data Contracts

Dagy provides a lightweight **dataset catalog** for discovery and **data contracts** for enforcing dataset shape and volume.

Catalog (API)

The catalog module (src/dagy_api/catalog/) is an org-scoped dataset registry stored in the table named by DAGY_CATALOG_TABLE (documented default dagy-catalog-{environment}). With no table configured the endpoints degrade gracefully.

Method & path	Purpose
`GET /catalog/datasets`	list registered datasets
`POST /catalog/datasets`	register a dataset
`GET /catalog/datasets/{dataset_key}`	dataset detail
`PUT /catalog/datasets/{dataset_key}`	update metadata
`DELETE /catalog/datasets/{dataset_key}`	remove a dataset
`GET /catalog/search`	search datasets

The catalog pairs naturally with lineage: catalog entries are the nodes, lineage edges are the relationships.

Data Contracts (SDK)

dagy.contracts (src/dagy/contracts.py) defines a contract a dataset must satisfy:

FieldSpec — a field's name, type, and nullability (accepts(value) checks a value against the spec; bools are rejected for numeric fields).
VolumeBounds — expected min/max row counts.
DataContract — the full spec (fields + volume).
Violations surface as ContractViolation / ContractViolationError, with a Severity, and ContractDiff compares two contract versions to detect breaking changes.

Contracts can be checked in CI with the repo helper:

uv run python scripts/check_contracts.py

Because contracts detect breaking schema changes before they ship, they are the "is this a compatible change?" gate for datasets that lineage tells you have downstream consumers.