Data Contract
A formal, machine-readable agreement between a data producer and its consumers that specifies schema, quality SLAs, semantics, and ownership — enforced automatically at pipeline ingestion points.
What is Data Contract?
A data contract is a formal, version-controlled specification that defines what a data producer commits to delivering to its consumers: the schema (field names, types, required/optional status), quality SLAs (acceptable null rates, value range constraints, timeliness), semantic definitions (what 'order_total' means in business terms, what currency unit applies), and ownership (who to contact when the contract is violated).
Data contracts solve the silent failure problem endemic to data pipelines: upstream teams change a field name or data type, downstream dashboards and models break silently, and analysts discover the issue only when a business decision is made on bad data. With contracts enforced at ingestion — using tools like Great Expectations, Soda, or custom validation layers — violations are caught at the pipeline boundary, not in the boardroom.
AiRAT implements data contracts as versioned YAML or JSON schema files stored in the same repository as pipeline code, with CI/CD enforcement that prevents promotion from Bronze to Silver unless the incoming data passes all contract assertions. Breaking contract changes require a deprecation notice and a migration window, governed by the producing team's SLA.
Examples in production
Schema version enforcement
A payment processor adds a new mandatory field to its API response. The data contract version bumps from v1.2 to v2.0, the downstream Silver promotion pipeline fails until the transformation layer handles both versions, and the producing team is notified via automated alerting.
Quality SLA breach
The data contract for customer profile data requires null_rate(email) < 2%. When a CRM export lands with 15% null emails, the ingestion job halts, an incident is opened, and the producing team's on-call is paged — before any corrupted data reaches Silver.
Questions about Data Contract
What format should a data contract use?
Common formats include YAML schema files (human-readable, git-compatible), JSON Schema (tooling-compatible), or domain-specific contract languages like OpenDataContract (ODCS). The format matters less than version control, CI/CD enforcement, and producer/consumer sign-off. Start with a simple YAML file in the producer's repo and enforce it with a validation step in the ingestion pipeline.
Who owns a data contract?
The producing team owns and maintains the contract — they commit to its SLAs. Consuming teams are stakeholders who review and accept changes. A data platform team typically provides the tooling and enforcement layer. Without producer ownership, contracts become wishful documentation that nobody enforces.
Running a system that uses Data Contract?
We build production-grade data engineering systems for regulated enterprises.