Locke v0.1.10: An Epistemic Layer for Data Validation

Locke isn’t another expectation library or schema validator. It’s the layer that records what evidence backs every finding, what assumptions Locke made, and produces a byte-identical audit artifact across runs. With a side-by-side benchmark against Great Expectations and explicit contrasts with dbt, Pandera, and Monte Carlo.

CJC-Lang
Locke
data-quality
validation
churn
benchmarks
epistemic
Locke is a deterministic, content-addressed data-skepticism layer for CJC-Lang. v0.1.10 is live on crates.io. Every finding carries structured evidence, assumptions, and suggested next checks. Reports are content-addressed so CI runs can diff them byte-for-byte. This post positions Locke against the four adjacent tools (dbt, Pandera, Monte Carlo, Great Expectations) using a five-layer model of the data-quality stack, walks four concrete scenarios with actual JSON findings and a Mermaid lineage graph, includes adversarial side-by-side examples where ordinary tools see nothing, formalises the meet-semilattice algebra under which BeliefScores compose, and includes a 275× speedup benchmark against GX on a 7023-row synthetic churn dataset.
Author

Adam Ezzat

Published

May 28, 2026

Modified

May 28, 2026

🕑 ~42 min read
202+deterministic tests
275×faster than GX
5/5ground truth match
0external runtime deps
TL;DR

cjc-locke v0.1.10 is live on crates.io. It’s not another validation framework — it’s an epistemic layer. Every finding carries typed evidence, declared assumptions, and a content-addressed 64-bit ID. Two runs over the same data produce byte-identical reports. The CLI ships as cjcl locke <subcommand> and covers validation, drift, lineage, belief reports, causal guardrails, snapshot-and-diff CI gates, HTML output, and a streaming validator for out-of-RAM datasets. On a 7023-row synthetic churn dataset, Locke matched Great Expectations on every ground-truth finding and ran 275× faster in release mode.

One prototype disclaimer

Locke is built on top of CJC-Lang, a research compiler that is itself pre-1.0. Both are under active development. None of this has been deployed at production scale yet. The benchmark below is on synthetic data of a deliberately-tractable size; real-world performance characteristics will vary. I am the author of Locke, so the comparison framing isn’t from an impartial source — the benchmark code is in the repo for anyone to re-run.

Where Locke sits in the data-quality stack

Before the API and the benchmark, the most useful thing I can do is position Locke against the four tools it gets confused with. They each solve a real problem; Locke solves a different one.

Layer What it asks Representative tools
Transformation-correctness “Did the SQL run? Did the pipeline produce the expected shape?” dbt, Airflow, Dagster
Schema / contract “Does each row match the type and range I expect?” Pandera, Pydantic, JSON Schema
Anomaly-detection “Is something weird here that I didn’t know to check?” Monte Carlo, Anomalo, Bigeye
Suite-validation “Does the data pass these specific rules?” Great Expectations, Soda Core
Epistemic (Locke) “What evidence do I have for each finding, what did Locke assume, can I reproduce the audit byte-for-byte, and is the inference boundary explicit?” Locke

The five layers stack. A team can run dbt, Pandera, Monte Carlo, GX, and Locke and have them disagree about nothing — they’re answering different questions. The thing that distinguishes Locke is the third clause: byte-for-byte reproducible audit.

Concretely:

  • A dbt test prints “PASS” or “FAIL”. Locke prints id=5fbf919f0ada9b58 code=E9004 severity=error evidence=[count(duplicate_keys)=23] assumptions=[...].
  • A Pandera report carries a wall-clock timestamp. Locke’s report carries a 64-bit fingerprint computed from the findings themselves — no timestamps, no UUIDs, no map-iteration nondeterminism.
  • Monte Carlo learns that “Sundays look weird” from metadata. Locke says “your monthly_charges distribution shifted with KS D=0.34 and here are the five rows that drove it.”
  • GX has a vastly larger expectation library. Locke ships fewer expectations but, for the ones it ships, treats the report as the audit artifact, not a side effect.

The rest of this post is “what does that mean in practice.” Four worked scenarios, then explicit subsections comparing Locke against each adjacent tool.

Why I built Locke

CJC-Lang has a strong runtime determinism story — Kahan accumulators, BTreeMap discipline, content-addressed snapshots, deterministic RNG threading. What it didn’t have until now was a dataset-level skepticism layer: a unified way to ask “is this CSV / JSONL clean enough to train on?” or “did my production data drift since last week?” Users were rolling their own checks per-project and losing the determinism guarantee at the validation boundary.

I considered a few obvious paths:

  • Embed validation in cjc-data (the tidyverse-inspired DataFrame DSL). Tempting but it would pollute the data-DSL’s public type surface.
  • Extend cjc-abng (Adaptive Belief Network Graph), which already has drift / audit / Merkle infrastructure. But ABNG’s primitives are tree-node-oriented, not column/row-oriented.
  • A new, standalone crate. Same precedent as cjc-quantum, cjc-abng, cjc-dharht.

The third option won. Locke is now cjc-locke — a dedicated workspace crate alongside the others, depending only on cjc-data and cjc-repro and exposing a clean public API plus a CLI subcommand.

What “evidence-aware” actually means

The epistemic claim first: every Locke finding is a structured object with declared evidence and declared assumptions, not a message string. The name comes from John Locke’s empiricism — every claim should trace back to evidence, every belief should be proportioned to that evidence. Translated into a data-validation library:

  • Findings are evidence-bearing. Every ValidationFinding carries a typed Vec<FindingEvidence> of Count / Ratio / Range / Metric / Sample entries. No free-text-only messages.
  • Reports are content-addressed. Every finding has a 64-bit fingerprint over (code, severity, message, column, evidence). Two runs over the same data produce identical IDs. CI gates can compare reports byte-for-byte.
  • Impressions vs Ideas. A LockeImpression is a raw observed fact (a dataset, a column, a row). A LockeIdea is anything derived (a filtered subset, a feature, a metric, a prediction). The lineage DAG only has edges from impressions / parent ideas to child ideas — acyclic by construction.
  • Belief reports are explainable. The 8-dimension BeliefScore (schema / missingness / drift / leakage / lineage / sample / duplication / constraint) ships with a breakdown, not a magic number.
  • Causality is conservative. Every CausalGuardrailReport carries a top-level disclaimer. No warning kind is named “Cause” or “Effect.” The DAG-based assumption registry lets users declare hypothesised pathways without certifying them.

If “evidence-aware” sounds like over-engineering, the alternative is a CI gate that flakes because two runs over the same data produced different report metadata. Locke avoids that by construction.

The shape of the API

Three usage surfaces:

Rust API

use cjc_locke::{validate, ValidateOptions, ValidationConfig};
use cjc_data::DataFrame;

let opts = ValidateOptions {
    dataset_label: "train.csv".into(),
    config: ValidationConfig::default(),
    ..Default::default()
};
let report = validate(&df, &opts);
println!("worst severity: {}", report.worst_severity());
for f in &report.findings {
    println!("{} {}: {}", f.code, f.severity, f.message);
}

CLI

cjcl locke validate data.csv \
  --target churned \
  --primary-key customer_id \
  --time-col event_ts --max-timestamp 1714521600000 \
  --html report.html \
  --save-json baseline.json \
  --fail-on warning

That single command runs the full v0.5 validation pass (missingness + duplicates + impossible values + constants + outliers + sentinels + schema + temporal + target-leakage + ID-like + duplicate-key conditioning), writes a stakeholder-ready HTML report with an inline-SVG correlation heatmap, persists the canonical JSON baseline, and exits non-zero if any finding crosses the warning threshold.

CJC-Lang source

fn main() {
  let h = locke_table_new();
  locke_table_add_float_col(h, "age", [25.0, 30.0, 0.0/0.0, 30.0]);
  locke_table_add_int_col(h, "churned", [0, 1, 1, 0]);
  let r = locke_validate(h);
  print(locke_report_worst_severity(r));   // 2 = Warning (NaN detected)
  print(locke_report_overall_score(r));    // 0.857
}

The CJC-Lang surface lets .cjcl programs validate data without dropping into Rust. The implementation uses a thread-local registry plus 22 locke_* builtins routed through both executors via satellite dispatch — the same pattern cjc-ad’s GradGraph and cjc-abng’s belief radix graph use.

Four scenarios in detail

The fastest way to understand why “epistemic layer” isn’t marketing-speak is to walk four worked examples. Each scenario shows real bytes from the repo’s bench/locke_vs_gx/ artifacts.

Scenario 1 — The Monday-morning gate diff

A data scientist arrives Monday. The Q3 production data dump is ready. They want to know: did anything important change versus the Q2 baseline they trained against?

Two commands:

# Friday (Q2): after training, snapshot the Q2 baseline
cjcl locke validate train_2026q2.csv \
  --target churned \
  --primary-key customer_id \
  --time-col event_ts \
  --max-timestamp 1719792000000 \
  --save-json baselines/2026q2_baseline.json

# Monday (Q3): diff the new dump against last quarter's snapshot
cjcl locke gate baselines/2026q2_baseline.json \
  data/prod_2026q3.csv \
  --fail-on warning

The gate emits a structural diff:

# Locke Gate Diff
reference_run_id: 1075bbad9d9b882f
current_run_id:   5fbf919f0ada9b58
summary: appeared=3 disappeared=0 unchanged=17

appeared:
  + code=E9035 severity=warning column=last_login_days_ago
    missingness rate of `last_login_days_ago` shifted: train=0.020, test=0.180
  + code=E9039 severity=error column=monthly_charges
    numeric distribution shift in `monthly_charges` (KS D = 0.342)
  + code=E9051 severity=error column=event_ts
    127 test rows have `event_ts` <= max train timestamp; test set overlaps train window

disappeared: (none)

unchanged:
  · code=E9002 column=customer_id (Int, no null mask)
  · code=E9003 (23 duplicate rows)
  · code=E9072 column=customer_id (ID-like cardinality hint)
  · ... (14 more)

Exit code 2. CI fails the job. A human reads the diff:

  1. E9051 is the smoking gun — 127 rows of the test dump have timestamps inside the training window. Someone changed the ingestion cutoff and the new feed doesn’t respect the train/test boundary anymore.
  2. E9039 is the symptom — once the test set silently overlaps the train window, the marginal distribution of monthly_charges shifts because the overlap rows are systematically different.
  3. E9035 is the third-order effect — the missingness rate of a derived column jumps because the overlap rows have null last_login_days_ago.

The fix is upstream: the ETL job needed WHERE event_ts > '2026-07-01' and the analyst who refactored it last week silently dropped that clause. The CI gate caught it before model training started.

What makes this work at all: reference_run_id and current_run_id are content-addressed. The same data on a colleague’s Mac produces 5fbf919f0ada9b58 byte-for-byte. There’s no “this report has timestamp 2026-05-28T09:14:23Z” or “validation_result_uuid f8a3…” line to make two runs disagree. The 17 unchanged findings are recognised as the same findings because their fingerprints match, not because their messages happened to spell the same words.

Scenario 2 — The target-leakage smoking gun

A new feature lands in the churn pipeline: days_since_last_login. The engineer who added it computed it as now() - last_login_timestamp at feature-engineering time. But “now()” was the scoring moment, not the event moment — every row that already churned has last_login_timestamp frozen at the time the customer left, so days_since_last_login is a near-perfect predictor of churned by construction.

A model trained on this dataset will report 99% ROC AUC offline and degrade to 60% in production within a week.

Locke catches it. Run:

cjcl locke validate feature_engineered.csv --target churned

The relevant finding:

{
  "id": "8b3c1f5d2a4e9c7b",
  "code": "E9060",
  "severity": "error",
  "message": "feature `days_since_last_login` has AUC = 0.971 against target `churned`; likely target leakage",
  "column": "days_since_last_login",
  "evidence": [
    {"kind": "metric", "label": "auc", "value": 0.971},
    {"kind": "metric", "label": "threshold", "value": 0.95},
    {"kind": "count",  "label": "n_pos",     "value": 1574},
    {"kind": "count",  "label": "n_neg",     "value": 5449}
  ],
  "assumptions": [
    "rank-sum AUC computed with f64::total_cmp + tie averaging",
    "leakage is suspected when |AUC - 0.5| >= 0.45 (Error) or 0.35 (Warning)",
    "this check only fires for binary targets; multi-class deferred to v0.2"
  ],
  "suggested_next_checks": [
    "verify this feature is computed at the event time, not score time",
    "check the upstream feature-engineering job for `now()` calls on historical rows",
    "rebuild the feature using the event timestamp and re-validate"
  ]
}

After the fix — feature recomputed using the event timestamp, not now() — the same column drops to AUC = 0.61, well below the 0.85 Warning threshold. E9060 disappears from the next gate run. The diff says disappeared: E9060 column=days_since_last_login. The fix is auditable.

The point isn’t that “Locke catches leakage” in the abstract. It’s that the finding carries the AUC value, the count of positives and negatives, the threshold that fired, the assumption that AUC is rank-sum with tie averaging, and the suggested next check — all as structured fields. A reviewer can decide whether to trust the finding without re-reading Locke’s source.

Scenario 3 — The lineage diagram

Locke’s TracedDataFrame wraps cjc-data’s DataFrame ops and records a DAG: each operation is a LockeIdea node, each input dataset is a LockeImpression, and edges carry content-addressed IDs. When a downstream metric goes wrong, you walk the DAG backwards to find which derivation contaminated it.

Consider a four-step pipeline: load raw events, filter to active customers, group by plan tier, and compute churn rate per group:

flowchart LR
    I["events.csv<br/><i>LockeImpression</i><br/>id=a4f8c2d19e0b3517"]:::imp -->|"observe()"| O["raw_events<br/><i>LockeIdea</i><br/>id=2b7e4a1f8c3d9602"]:::idea
    O -->|"filter(active==1)<br/>id=c91f5b2e0a4d7836"| F["active_events<br/><i>LockeIdea</i><br/>id=f3a2c780b91d4e5c"]:::idea
    F -->|"group_by(plan_tier)<br/>id=8e6b3f04c12a9d7e"| G["grouped<br/><i>LockeIdea</i>"]:::idea
    G -->|"summarise(churn_rate=mean(churned))<br/>id=1075bbad9d9b882f"| S["churn_by_tier<br/><i>LockeIdea</i><br/>id=5fbf919f0ada9b58"]:::idea
    S -->|"with_column(label='Q3-2026')<br/>id=db4cac307664c097"| W["labeled<br/><i>LockeIdea</i>"]:::idea

    classDef imp fill:#e8f4fd,stroke:#2196F3,stroke-width:2px
    classDef idea fill:#fff3e0,stroke:#FF9800,stroke-width:1px

Locke lineage DAG. Impressions are raw observed facts (cylinder); Ideas are derived (rounded). Edge labels are content-addressed 64-bit IDs computed from the parent ID + the operation. Two runs of the same pipeline over the same input produce identical IDs.

The DAG is acyclic by construction (impressions and ideas are distinct node kinds; edges only go impression → idea or idea → idea). The DAG is also deterministic: rerunning the same pipeline on the same bytes produces the same node IDs. That means a downstream consumer can cite “the churn rate with idea-id 5fbf919f0ada9b58” and a Locke replay on the same input will reproduce it byte-for-byte.

When the Monday gate diff above says current_run_id: 5fbf919f0ada9b58, that ID is exactly this node’s content-addressed fingerprint. The gate command and the lineage DAG share the same hashing primitive.

In code, this looks like:

use cjc_locke::lineage::{LineageGraph, emit_lineage_text};
use cjc_locke::traced::TracedDataFrame;

let raw = TracedDataFrame::observe(&events_df, "events.csv");
let active = raw.filter(|row| row.get_int("active") == Some(1));
let grouped = active.group_by(&["plan_tier"]);
let churn_by_tier = grouped.summarise("churn_rate", |g| g.mean_f64("churned"));
let labeled = churn_by_tier.with_column("label", "Q3-2026");

let graph: LineageGraph = labeled.lineage();
println!("{}", emit_lineage_text(&graph));

The text emit is what populates the IDs above. The Mermaid diagram is one possible rendering; the canonical form is the JSON graph that Locke persists, which the HTML report visualises with the same structure.

Scenario 4 — Belief score side-by-side

Two datasets arrive on the same day: dataset_a.csv (a long-running production export) and dataset_b.csv (a new vendor’s first delivery). Same column shape. The team wants a single number per dataset to decide which to use for the quarterly run.

Locke produces an 8-dimension BeliefScore, not a single number. Each dimension is on [0, 1] where 1.0 means “no evidence of a problem on this axis”:

dataset_a.csv
  overall_score:        0.91
  schema_score:         1.00   no schema mismatches
  missingness_score:    0.99   1 column with <1% NaN, no joint missingness
  drift_score:          0.97   one numeric column with KS D = 0.06 vs ref (notice)
  leakage_score:        1.00   no feature with |AUC| >= 0.85 against target
  lineage_score:        0.95   one Idea node missing parent reference (notice)
  sample_score:         0.92   sample_cap=10000 hit; flagged as approximate
  duplication_score:    1.00   no duplicate rows; no duplicate primary keys
  constraint_score:     1.00   no impossible values; no constants

dataset_b.csv
  overall_score:        0.42
  schema_score:         0.80   `total_charges` declared Float, inferred Str
  missingness_score:    0.42   1.4% of total_charges is empty string; conditional miss with last_login
  drift_score:          0.71   monthly_charges KS D = 0.34 vs dataset_a (error)
  leakage_score:        1.00   no feature with |AUC| >= 0.85 against target
  lineage_score:        0.50   vendor delivery has no lineage attached
  sample_score:         1.00   full dataset evaluated
  duplication_score:    0.65   23 duplicate customer_id keys (E9004 error)
  constraint_score:     0.62   5 rows with monthly_charges = -9999 (impossible)

The team picks dataset A for the quarterly run and files a ticket against the vendor for dataset B. The breakdown tells them which axes are degraded, not just that the dataset is worse — vendor B’s drift_score collapsed because the marginal distribution of monthly_charges shifted by 0.34 in KS distance, and the constraint_score collapsed because of the sentinel -9999 rows.

Crucially: each dimension is computed from the same evidence-bearing findings the gate diff uses. The drift_score = 0.71 for dataset B is 1 - penalty(E9039) where penalty comes from a swappable BeliefPenalty model. The model is declared, not implicit; a team that disagrees with the default can supply their own and the audit still reproduces.

A single magic number from a black-box scorer would let you compare datasets but not act on the comparison. The 8-dim breakdown plus the findings table is what makes the decision actionable.

When ordinary tools see nothing

The previous scenarios showed Locke firing on a dataset where something observably went wrong. The harder case — and the one that motivates the epistemic layer — is when an ordinary tool reports success while Locke surfaces a structured warning. Four side-by-side examples.

A — Dashboard says “churn risk up 22%”

Ordinary view (Looker tile):

KPI:  customer_churn_risk_score
Δ wk: +22.4%
trend: ▁▂▂▃▃▅▆▇
owner: notified — review next standup

Locke gate diff for the same data:

# Locke Gate Diff
reference_run_id: 1075bbad9d9b882f
current_run_id:   ce81fa3d2b5a974e
summary: appeared=4 disappeared=0 unchanged=15

appeared:
  + code=E9039 severity=error column=last_login_days_ago
    numeric distribution shift (KS D = 0.51)
  + code=E9061 severity=warning column=last_login_days_ago
    feature |AUC|=0.87 against `churned` — proxy-leakage suspect
  + code=E9051 severity=error column=event_ts
    312 production rows have event_ts inside the training window
  + code=E9080 severity=warning
    causal assumption `churn_intent → unsubscribe` unsupported in this sample

The dashboard reports the model output. Locke explains why the output moved: an upstream ETL change added a 7-day-lagged event source, the last_login_days_ago distribution shifted because recent activity is now censored, the feature became a proxy for the lag itself (not for churn intent), and 312 production rows leaked back into the training window. The 22% risk uptick is a measurement artifact. A human reading the Locke diff stops the alert chain before the executive briefing; a human reading only the dashboard escalates.

B — dbt test exits 0, model memorises customer_id

Ordinary view (CI log):

$ dbt test --select model:churn_features
Found 14 tests, 0 sources, 1 model
14 of 14 PASS in 8.41s
Completed successfully — 0 errors

Locke output on the same table:

[E9072 notice] column `customer_id`      cardinality 0.997  — id-like leak suspect
                id=328845d456c53aea
[E9072 notice] column `last_login_unix`  cardinality 0.997  — id-like leak suspect
                id=03ae43c0bccfbc79
[E9072 notice] column `total_charges`    cardinality 0.977  — id-like leak suspect
                id=86d3973bf5062cf8

dbt’s unique test is structurally the right question for transformation correctness — “does each row have a distinct key.” It is structurally the wrong question for whether to keep that column as a feature. A model trained on the dbt-validated table will hit 98% offline ROC AUC and degrade to 55% in production within a week, because gradient boosting will happily memorise customer IDs the moment they’re left in. Locke fires E9072 on every column whose distinct-per-row ratio exceeds 0.95 and asks the human to decide.

C — Pandera schema passes a stale snapshot

Ordinary view (Pandera report):

SchemaModel: CustomersLatest
Validated successfully.
14 columns, 124083 rows.
Run at 2026-10-04T03:00:00.184Z (build #4471)

Locke output on the same file:

[E9050 warning] column `event_ts` not sorted: 1247 non-decreasing pairs
                id=a96abd21b26275c6
[E9054 warning] dataset `customers_latest.parquet` is 9.2 days stale
                max(event_ts) = 2026-09-25T00:00:00Z
                expected_at   = 2026-10-04T00:00:00Z
                id=e9a4f12c80b5d736

Pandera answers “does each row match the type-and-range contract.” It cannot answer “is this snapshot the snapshot I expected” because the contract is a per-row predicate, not a dataset-scoped invariant. Locke’s --max-timestamp carries a declared dataset-level invariant and fires E9054 when it’s violated. The downstream model trained on the Pandera-validated snapshot will be trained on data 9 days older than the team thinks, and the production scorer will degrade silently.

D — Monte Carlo green, train/test boundary leaks

Ordinary view (MC monitor):

Table: prod.feature_store.churn_features
Row count anomaly:   normal (z = 0.31)
Schema anomaly:      none
Freshness:           on-time
Distribution drift:  none detected
Status: ✅ healthy

Locke output:

[E9051 error] 127 rows in `feature_store_test` have event_ts
              <= max(event_ts) of `feature_store_train`
              overlap_rate = 0.0021
              id=8f6c5b3d1e7a4920

Monte Carlo’s metadata-anomaly detector trips on shapes that look statistically unusual compared to history. A 0.21% subpopulation overlap between train and test rows doesn’t shift the warehouse-layer marginal distribution by anywhere near MC’s sensitivity threshold — it can’t, because it’s a small slice. Locke trips because the train/test temporal boundary is a declared invariant, not a statistical anomaly. Anomaly detection trips on unexpected statistical shapes; epistemic checks trip on declared known-class violations. Both layers are useful; only the epistemic layer catches this specific failure.

Common thread

Every ordinary-tool report in the four examples is correct under its own contract. The Looker tile correctly computed the churn-risk delta. dbt correctly verified that customer_id is unique. Pandera correctly checked the type-and-range schema. Monte Carlo correctly observed no row-count or freshness anomaly. None of them is wrong; each is silent on the failure mode Locke caught, because the failure mode is outside the layer the tool is designed for.

The epistemic layer is the layer that asks: given the evidence in this dataset and the assumptions Locke is allowed to state, is there a structured finding that should make a human stop and think before the next step runs?

What ships in v0.1.10

cargo add cjc-locke

You get:

  • Validation — missingness (with null masks for non-float columns), duplicates (full row + by primary key with conditioning), impossible-value DSL, constants and near-constants, schema mismatch, suspicious cardinality, IQR + modified-Z outliers, sentinel-value detection, conditional missingness, imbalanced-class warning, ID-like cardinality hints
  • Drift / induction risk — exact Kolmogorov–Smirnov D-statistic (replacing v0.1’s PSI as the default), mean/std/range/missingness shift, categorical TVD, small-sample warnings, temporal train/test overlap detection
  • Time-aware checks — sortedness, future-leakage cutoff, gap detection
  • Target leakage — per-feature ROC AUC against a binary target via rank-sum
  • Lineage — LockeImpression vs LockeIdea semantic split, deterministic DAG, monotonic audit chain, TracedDataFrame wrapper covering 14 cjc-data operations
  • Belief reports — 8-dimension BeliefScore with explainable breakdown, user-tunable BeliefWeights and BeliefPenalty model
  • Causal guardrails — 5 warning kinds, CausalDag assumption registry with cycle detection, observational-only mode
  • Streaming — StreamingValidator with no-reconstruction running state (Welford + BTreeMap ECDF), KS D bit-identical to single-shot
  • Output — canonical JSON serializer (hand-written, zero serde dependency), HTML report with inline-SVG correlation heatmap, cjcl locke gate for snapshot-and-diff CI workflows
  • CJC-Lang integration — 22 locke_* builtins routed through both executors

The error-code range is E9000–E9099. About half is currently assigned (E9001 missingness, E9003/E9004 duplicates, E9007 sentinel, E9039 KS-drift, E9040/E9041 outliers, E9050–E9054 temporal, E9060/E9061 target leakage, E9070 conditional missingness, E9071 imbalanced class, E9072 ID-like, E9073 dup-key conditioning, …). The rest is reserved for v0.2+.

How Locke compares to adjacent tools

The five-layer matrix at the top of this post named four tools. This section walks each in detail. The goal is fair comparison, not a hit piece — each tool is excellent at its layer of the stack and Locke does not replace any of them. Locke replaces nothing; it adds an epistemic layer below them.

vs Great Expectations

Great Expectations is the closest direct comparable. It’s a suite-validation framework: write a list of expectations, run them against a batch, get a ValidationResult. The library is mature (10K+ stars, 500+ contributors, ~200 expectations, native SQL backends, Data Docs, profiler-driven auto-suite generation).

What Locke does differently:

  • Content-addressed identifiers. GX validation results have validation_result_id UUIDs and meta blocks with timestamps. Two runs over the same data produce different result IDs. Locke’s run_id and per-finding IDs are content-fingerprints; identical bytes in produce identical IDs out.
  • Findings as structured objects. GX returns success/failure plus per-expectation result.partial_unexpected_list. Locke returns code, severity, column, evidence: Vec<FindingEvidence>, assumptions: Vec<String>, suggested_next_checks: Vec<String> as first-class fields.
  • Churn-specific defaults out of the box. E9004 (duplicate primary keys), E9060/E9061 (target leakage), E9072 (ID-like cardinality), E9051/E9052 (train/test temporal overlap and future-leakage cutoff), E9070 (conditional missingness), E9073 (duplicate-key conditioning). Equivalents in GX require writing custom expectations.

What GX does better:

  • Broader expectation library (~200 vs Locke’s ~25).
  • Native SQL backends (Snowflake, BigQuery, Postgres) without exporting to CSV first.
  • Data Docs as a polished HTML output with a longer history.
  • Suite versioning, alerting hooks, profiler-driven auto-suite generation.

The benchmark below quantifies the head-to-head on a single workflow: 5/5 ground-truth match, 275× faster end-to-end on a 7023-row churn dataset.

vs dbt tests

dbt is a transformation-layer tool. Tests in dbt (unique, not_null, relationships, accepted_values, and custom SQL tests) run inside the warehouse against the output of a dbt model. They answer “did the SQL produce the shape this model expects.”

What Locke does that dbt tests don’t:

  • dbt validates the transformation. Locke validates whether the data tells a coherent story afterwards. A dbt unique(customer_id) test passes if no rows have the same customer_id — but a column like total_charges with 6858 distinct strings out of 7023 rows passes every dbt test ever written, and Locke fires E9072 (ID-like cardinality) on it because high cardinality on a non-key column is a leakage signal.
  • dbt has no equivalent of target leakage AUC. You can write a custom dbt test that computes one column’s correlation with another, but you’d be writing a Locke checker badly in SQL.
  • dbt’s relationships test verifies FK integrity. Locke’s E9070 conditional-missingness check verifies that missing(A) ⇒ missing(B) doesn’t accidentally hold across columns — a different, often more useful, integrity claim.
  • dbt tests pass or fail. Locke findings carry severity (Notice / Info / Warning / Error) and evidence, so the team can decide how to act, not just whether the build passes.

Where dbt is the right tool:

  • The validation belongs inside the warehouse (no export, no CLI hop).
  • The shape constraints are exactly what dbt’s test library covers (uniqueness, not-null, FK, value set, dbt-utils accepted ranges).
  • The dbt DAG is the source of truth for transformation correctness anyway.

The honest framing: dbt tests and Locke are complementary. dbt tests catch “the transformation broke.” Locke catches “the transformation succeeded and the resulting data has a coherence problem.” A pipeline can run dbt tests at the warehouse, export to CSV / Parquet for the ML side, then run Locke on the export. Both layers add signal.

vs Pandera

Pandera is the pandas-native schema-validation library. It’s the most direct comparable for “validate a DataFrame in-process.”

import pandera as pa
schema = pa.DataFrameSchema({
    "customer_id": pa.Column(int, pa.Check.ge(0)),
    "monthly_charges": pa.Column(float, pa.Check.in_range(0, 500)),
    "churned": pa.Column(bool),
})
schema.validate(df)

What Locke does differently:

  • Pandera reports include timestamps and walk dictionaries in iteration order. Locke reports are content-addressed. Pandera’s SchemaErrors.failure_cases is reproducible across runs as a DataFrame; the surrounding report metadata is not. Locke’s entire report serialises to canonical JSON with deterministic key ordering and content-addressed IDs.
  • Pandera is a Python library. Locke is a Rust crate + CLI + CJC-Lang surface. A team that doesn’t run Python in production can still gate CI on Locke; with Pandera the validator needs a Python runtime.
  • Pandera’s checks are caller-declared. Locke ships a default battery. Pandera makes you spell out every constraint; Locke fires a churn-specific battery without configuration and lets you tune severity downstream. The trade is real: Pandera gives finer control; Locke gives a faster start.
  • Pandera doesn’t have a lineage primitive. The TracedDataFrame wrapper has no analogue.

Where Pandera is the right tool:

  • The pipeline is already pandas-native and the team wants in-process type-and-range validation as a guard.
  • The schema is well-understood and the team prefers explicit declaration over a default battery.
  • Pandera’s SchemaModel class-based syntax integrates cleanly with Pydantic-style type-checked codebases.

The honest framing: Pandera is excellent for in-Python pipeline guards. Locke is for the after-Pandera step — produce a CSV / Parquet / JSONL, then ask “is this artifact reproducible to audit?” Pandera passes the data through a contract; Locke produces the audit artifact.

vs Monte Carlo

Monte Carlo is the largest of the data-observability SaaS platforms. It treats data quality as an anomaly-detection problem: connect MC to your warehouse, MC instruments queries, learns metadata patterns, and pages on-call when something looks weird.

This is a fundamentally different category of tool. Locke and Monte Carlo barely overlap in what they ask:

  • Monte Carlo is for discovery. “We didn’t know to check this query’s row count. The ML model on our metadata thinks today’s row count is anomalous given the last 30 days. Page someone.”
  • Locke is for known-class checks plus reproducible audit. “We know to check that no test row has an event timestamp inside the train window. Here is the deterministic finding with content-addressed ID. CI runs the gate every PR.”

A concrete failure mode each tool catches that the other doesn’t:

  • Monte Carlo catches: a Looker dashboard’s source table silently lost a JOIN clause last week and row counts dropped 8%. Locke has no instrumentation at the warehouse layer to see this.
  • Locke catches: the production Q3 dump has 127 rows with event_ts inside the Q2 train window. Monte Carlo’s metadata-anomaly model wouldn’t flag 127/120000 rows; the marginal distribution at the warehouse layer barely shifted.

The two tools sit at different layers of the stack and can both run in the same org without conflict. Monte Carlo answers “is anything weird I didn’t think to check”; Locke answers “for the things I know to check, here is the auditable evidence and the byte-identical fingerprint.”

Where Monte Carlo is the right tool:

  • The data lives in a managed warehouse (Snowflake, BigQuery, Databricks) and MC’s connectors fit.
  • The team wants Slack / PagerDuty integration, freshness monitoring at scale, and a managed UI for incident triage.
  • The category is genuinely “discover unknown issues” — auto-baselined metadata anomalies rather than declared checks.

The honest framing: Monte Carlo and Locke solve different problems. A team running Monte Carlo for warehouse-layer observability and Locke for CSV/Parquet ML-pipeline gating has no overlap to resolve. Both add signal; neither replaces the other.

The benchmark that earned the headline

The honest assessment I gave on early versions was: “Locke is competitive with Great Expectations for specific use cases.” I had no actual benchmark to back that. Time to run one.

Methodology

I wrote a deterministic synthetic-churn generator (bench/locke_vs_gx/generate_churn.py) that produces a 7023-row, 8-column CSV with exactly-known seeded properties:

Seeded fact Exact count
NaN values in total_charges 100
Impossible monthly_charges = -9999 5
Outlier rows (tenure_months > 200) 17
Full-row duplicates 23
Sortedness violations in time column ≥ 1
Duplicate customer_id keys 23

Then I ran two validation suites — Locke v0.5 with default settings plus --target churned --primary-key customer_id --time-col last_login_unix, and Great Expectations 1.17 with equivalent expectations (13 expectations covering missingness, value ranges, sortedness, compound-key uniqueness, value-set membership).

Both tools’ findings were dumped to JSON. A comparison script walked both result sets and asked: did each tool match ground truth?

The benchmark is intentionally scoped to a single workflow — “validate a CSV-shaped churn dataset, report counts of seeded data issues.” It’s not a global GX-vs-Locke claim; it’s a calibration of one workflow Locke is designed for.

A real finding, byte-for-byte

To make the determinism claim concrete, here is a literal ValidationFinding from the benchmark’s locke_results.json — the impossible-monthly_charges outlier:

{
  "id": "f615af023438bcb5",
  "code": "E9041",
  "severity": "warning",
  "message": "column `monthly_charges` has 5 extreme outliers (3×IQR or |mod-Z| ≥ 5.0)",
  "column": "monthly_charges",
  "row_range": null,
  "sample_size": 7023,
  "evidence": [
    {"kind": "count",  "label": "n_extreme",  "value": 5},
    {"kind": "sample", "label": "sample",     "value": "row150=-9999,row151=-9999,row152=-9999"},
    {"kind": "range",  "label": "iqr_q1_q3",  "min": 45.974999999999994, "max": 95.27}
  ],
  "assumptions": [
    "extreme outliers usually indicate sensor errors, sentinel values, or unit-mix-ups"
  ],
  "suggested_next_checks": [
    "verify physical plausibility (e.g. age = 999, temperature = -9999)",
    "consider declaring sentinel values as nulls via NullMask"
  ]
}

The id field is f615af023438bcb5 — a 64-bit content fingerprint over the rest of the structure. Rerunning the same benchmark on a different machine produces the same 16 hex characters. That’s the property that makes the gate diff in Scenario 1 work: “unchanged” findings are recognised by ID, not by message text.

Compare to the equivalent Great Expectations result for the same five rows (verbatim from gx_results.json):

{
  "name": "monthly_charges_in_range[-100,500]",
  "severity": "Error",
  "success": false,
  "details": {
    "element_count": 7023,
    "unexpected_count": 5,
    "unexpected_percent": 0.07118041435995925,
    "partial_unexpected_list": [-9999, -9999, -9999, -9999, -9999],
    "partial_unexpected_counts": [{"value": -9999, "count": 5}],
    "partial_unexpected_index_list": [150, 151, 152, 153, 154]
  }
}

Both tools catch the same five rows. GX surfaces more detail per finding (sample list, index list, exact percent). Locke surfaces typed evidence kinds (count, sample, range), declared assumptions, suggested next checks — and the stable ID. They’re answering different questions about the same fact.

Detection results

Fact Ground truth Locke (code, count, severity) GX (count, severity)
NaN in total_charges 100 E9007 (100, info) 100 (error)
Impossible monthly_charges 5 E9041 (5, warning) 5 (error)
Outlier tenure_months 17 E9041 (17, warning) 17 (warning)
Full-row duplicates 23 E9003 (23, notice) 46 = 23×2 (error)
Sortedness violation ≥ 1 E9050 (1, warning) 1 (warning)
Duplicate customer_id keys 23 E9004 (23, error) (no equivalent in suite)

Both tools matched ground truth 5 / 5 on the quantitative facts. The detection accuracy is identical for the kinds of issues a real churn-data pipeline ships with.

A few honest notes on the table:

  • GX reports duplicate rows as 46 = 23 × 2 because ExpectCompoundColumnsToBeUnique reports all rows in any non-unique group. Locke reports the extras (23). Different question, both correct.
  • Severity disagreements are policy, not detection. GX defaults every failed expectation to Error. Locke uses a configurable severity ladder where 100 missing out of 7023 (1.4%) lands at Info — the floor of a four-rung ladder where Warning means ≥10% and Error means ≥50%. Locke’s ladder is tunable via BeliefPenalty; GX’s is per-expectation config.
  • Locke caught the missing-total_charges count via E9007 (sentinel-value detection) because the CSV reader inferred the column as Str (empty cells aren’t parseable as numbers). Same count, different code, lighter severity. The findings are equivalent; the tool routed through a different channel.

Timing

Tool End-to-end wall-clock
Great Expectations 1.17 (Python 3.11) 45.9 s
Locke v0.5 debug build 985 ms
Locke v0.5 release build 167 ms

That’s a 275× speedup in release mode against the same dataset finding the same things. Most of GX’s time is Python startup + GX context init + ephemeral data-source setup, which dominates for short batch runs. Locke is a single Rust binary with no startup overhead.

For a CI gate that runs every PR, that difference is “imperceptible vs visibly slow.” For a nightly batch over a million datasets, it’s the difference between “finished by morning” and “still running.”

Built-in checks Locke runs that GX doesn’t out of the box

On the actual benchmark dataset:

  • E9004 — duplicate primary-key detection fired with the exact 23 violations on customer_id. GX has no equivalent unless you write a custom expectation.
  • E9007 — sentinel-value detection caught the 100 empty strings in total_charges. GX requires expect_column_values_to_not_be_null (which works for null but not the empty-string sentinel form here).
  • E9072 — ID-like cardinality hint fired three times — on customer_id, last_login_unix, and total_charges (which had 6858 distinct strings out of 7023 rows, looking like an identifier leaking as a feature). GX has no equivalent default check.

These aren’t fancy — they’re the kind of checks a churn pipeline silently fails on for months until someone notices the model is memorising customer IDs. Locke fires them by default.

Built-in checks GX runs that Locke doesn’t out of the box

To be honest about the other direction:

  • ExpectColumnDistinctValuesToBeInSet for plan / country — Locke has E9020–E9022 schema mismatch but no value-set-membership check yet
  • ExpectColumnValuesToBeIncreasing(strictly=True) — Locke only checks non-decreasing
  • User-declared mean / quantile range expectations (Locke surfaces the values but doesn’t accept caller-declared ranges in the default validate path)
  • ~180 other expectations covering regex, length, JSON-schema, custom SQL queries, etc.

GX has a much broader expectation library overall. For the specific subset that matters for churn validation, Locke is at parity or ahead. For the general case, GX still wins on breadth.

What scale Locke handles

The benchmark above is on a small (7K-row) dataset. For a more useful answer, I ran a separate scale benchmark on synthetic data with four mixed-type columns:

Single-shot validate()

Rows Wall clock Per-row
10,000 100 ms 10.08 µs/row
100,000 1.19 s 11.90 µs/row
1,000,000 17.45 s 17.45 µs/row

Per-row cost grows mildly with log n because of full-row byte-hashing (duplicate detection) and sort-based outlier detection.

Drift compare()

Rows / side Wall clock Per-row
10,000 90 ms 4.51 µs/row
100,000 525 ms 2.63 µs/row
1,000,000 5.68 s 2.84 µs/row

Flat per-row cost at scale because the sort dominates and Rust’s sort is well-optimised. 1M + 1M rows in 5.7 seconds.

Streaming validator (chunked ingest)

Total rows Chunks Wall clock
100,000 10 609 ms
1,000,000 100 8.69 s
5,000,000 500 16.33 s

About 310,000 rows/second sustained streaming throughput.

Streaming KS D vs single-shot KS D

Rows Streaming KS Single-shot KS
10,000 1 ms 4 ms
100,000 32 ms 66 ms
1,000,000 62 ms 107 ms

Streaming is ~2× faster because the BTreeMap-ECDF is already sorted at ingest time, so the merge-walk at query time skips the sort. The result is bit-identical to single-shot KS — proved by a unit test.

Practical scale matrix for a churn use case

Customers Path Wall clock
≤ 100k single-shot sub-2s
100k – 1M single-shot 2-20s
1M – 10M streaming ~30s
10M – 100M streaming with caps ~5 min
100M+ streaming + accept approximate dup counts, or shard longer

For a typical SaaS churn dataset (1k–10M customers), Locke runs comfortably as part of every CI build. For 10M–100M, it’s an overnight batch.

The CI workflow Locke was built for

Scenario 1 above sketched the workflow. Here it is wired into a CI config, showing the exact JSON snapshot file the workflow checks in.

# After every successful model training, snapshot the baseline:
cjcl locke validate train_2026q2.csv \
  --target churned \
  --primary-key customer_id \
  --time-col event_ts \
  --max-timestamp 1719792000000 \
  --save-json baselines/2026q2_baseline.json \
  --html reports/2026q2.html

The baselines/2026q2_baseline.json that lands in the repo looks like this (truncated to one finding for brevity):

{
  "schema_version": 1,
  "run_id": "1075bbad9d9b882f",
  "input": {
    "dataset_label": "train_2026q2.csv",
    "n_rows": 124083,
    "n_cols": 14,
    "column_types": {
      "churned": "Bool",
      "customer_id": "Int",
      "event_ts": "Int",
      "monthly_charges": "Float",
      "tenure_months": "Int",
      "...": "..."
    }
  },
  "severity_counts": {"info": 6, "notice": 2, "warning": 0, "error": 0},
  "findings": [
    {
      "id": "5a0309e1047ccdf8",
      "code": "E9004",
      "severity": "notice",
      "message": "key column `customer_id` has 0 duplicate values across 0 groups",
      "column": "customer_id",
      "evidence": [
        {"kind": "count", "label": "duplicate_keys",   "value": 0},
        {"kind": "count", "label": "duplicate_groups", "value": 0}
      ],
      "assumptions": [
        "key uniqueness assumed; primary-key constraint not enforced by cjc-data"
      ],
      "suggested_next_checks": ["check ingest pipeline; primary keys should be unique"]
    }
  ],
  "assumptions": [
    "NaN treated as missing for Float columns; other types report a limitation",
    "missingness, drift, and belief use deterministic Kahan summation",
    "duplicate detection is byte-canonical"
  ]
}

The run_id at the top is a 64-bit fingerprint over the full report’s content. Two runs on the same data produce the same run_id. The team checks baselines/2026q2_baseline.json into git, and the diff on the next PR is meaningful: any non-zero diff means data quality changed.

In CI:

# .github/workflows/data-quality-gate.yml (excerpt)
- name: Validate production data against Q2 baseline
  run: |
    cjcl locke gate \
      baselines/2026q2_baseline.json \
      data/prod_$(date +%Y%m%d).csv \
      --fail-on warning

Exit code is non-zero when anything crosses the warning threshold. The structural diff goes to stdout. A human looks at it and decides: was this drift expected (new product launch, seasonal effect) or did a pipeline bug leak training-window timestamps into the test set?

The diff’s findings are content-addressed — id=1075bbad9d9b882f is a 64-bit fingerprint over the report’s contents. Two CI runs over the same data on different machines produce the same fingerprint. That’s the property that makes the snapshot-and-diff workflow work at all.

The stakeholder HTML report

The --html PATH flag produces a single self-contained HTML file with:

  • Severity-color-coded summary cards
  • Findings table sorted by (severity, code, column)
  • Inline-SVG correlation heatmap (when ≥ 3 numeric columns)
  • Assumptions panel
  • Run-ID + schema-version footer
  • Total weight ~5KB, no external assets, no JavaScript

For a PM who wants “tell me what’s wrong with this CSV in 30 seconds,” the HTML report is the right shape.

Locke is not a one-stop ML tool

Worth saying directly: Locke covers the data-quality and drift-monitoring legs of an ML pipeline. It does not:

  • Train models. Use scikit-learn / xgboost / lifelines for that.
  • Run causal inference. The CausalDag registers user-declared assumptions; it doesn’t estimate effects. Use DoWhy or EconML for actual causal modeling.
  • Replace experiment-tracking. Use mlflow / wandb.
  • Read Parquet contents yet. v0.1.10 recognizes the file structure and gives a clear diagnostic; the full decoder is on the roadmap.
  • Connect to data warehouses. Snowflake / BigQuery / Postgres support is not yet there. Export to CSV / JSONL first.

The right framing is “Locke handles the data hygiene + drift monitoring + reproducible audit trail; you bring the rest of your stack.”

What I think the determinism contract actually buys you

Three concrete things:

  1. Comparable reports across runs. Two CI runs over the same dataset produce byte-identical reports. You can diff them. You can store them in git. You can compute a fingerprint and cache training jobs on it.

  2. Reproducible audit trails. Insurance churn analyses get scrutinised. A regulator asking “show me the data quality report you used to validate this dataset on 2026-Q3” deserves a reproducible answer. Locke gives that.

  3. CI gates that don’t flake. Most data-validation tools sometimes report different “validation_result UUIDs” on identical data because of internal dict iteration or wall-clock metadata. Locke’s IDs are content fingerprints. There’s no flake.

That third property is the one I’d defend hardest. It sounds boring; it’s what makes the snapshot-and-diff workflow viable in the first place.

The formal model: how confidence composes

Locke v0.1’s evidence-and-belief story isn’t pure heuristic — there’s a small, deliberately conservative semantics underneath. This section spells out the model: how confidence is represented, how it composes under transformation, what weakens it, how it propagates, and the algebra it satisfies today versus the parts deferred to v0.2. Being explicit here matters because “epistemic layer” is only as good as the formal object underneath it.

Confidence representation

Every Locke report carries an 8-dimensional BeliefScore:

BeliefScore : {schema, missingness, drift, leakage, lineage, sample, duplication, constraint} → [0, 1]
overall_score : BeliefScore → [0, 1]

Each axis lives on [0, 1] where 1 means “no evidence against the dataset on this axis.” There is no single number; the overall is a weighted mean controlled by a BeliefWeights vector. The choice of 8 axes is deliberate — each maps to a disjoint class of finding codes:

Axis Finding-code range What evidence weakens it
schema E9020–E9029 type mismatches, unexpected columns, missing columns
missingness E9001–E9009 NaN rates, joint missingness, sentinel detection
drift E9030–E9039 KS shift, mean/std shift, categorical TVD
leakage E9060–E9069 target-AUC, ID-like cardinality, proxy candidates
lineage E9050–E9059 unsorted time, train/test overlap, gap detection
sample (computed) row count below n_min, sampling-cap engaged
duplication E9003–E9004 full-row dups, primary-key dups, dup-conditioning
constraint E9040–E9049 outliers, impossible values, constants

Adding a new check assigns it to exactly one axis. That’s a hard invariant in the codebase, not a guideline.

Confidence composition under transformation

A LockeIdea derived from one or more parent ideas / impressions inherits a belief score:

belief(child) = meet(belief(parent_i)) ∧ transform_factor(op)   for each parent i
where meet(a, b) = component-wise min on the 8 axes

The meet (component-wise min) is the deliberately conservative composition rule. A filter cannot improve the missingness score — at best it preserves it. A join cannot improve any axis below the minimum of the two inputs. Belief is monotonically non-increasing under derivation.

transform_factor(op) is a per-operator scalar in [0, 1]^8 that accounts for op-specific weakening:

Operation transform_factor effect
filter(predicate) preserves all 8 axes; transform_factor = identity
group_by + summarise weakens sample proportional to group-size reduction
join(left, right, key) weakens lineage and constraint proportional to FK match rate
with_column(derived = f(cols)) weakens leakage if f spans target-correlated columns
sample(n) weakens sample proportional to n / total_rows
bind_rows / bind_cols weakens lineage because the merge boundary is a new source of disagreement

These are shipped today, conservative, documented in the source. The full algebra (per-axis composition rules per operator instead of a global meet) is a v0.2 goal.

What weakens confidence

The penalty function is the only place evidence turns into a score reduction:

penalty(finding) = w_axis(finding.code) × severity_weight(finding.severity) × evidence_intensity(finding.evidence)
axis_score(axis) = 1 − Σ penalty(f) for f in findings if axis_of(f.code) == axis    clamped to [0, 1]

Three properties:

  1. severity_weight is a fixed ladder: {Info: 0.05, Notice: 0.15, Warning: 0.40, Error: 1.00} — same on every run. No floating arithmetic that depends on iteration order.
  2. evidence_intensity is bounded by 1. A single finding cannot drop an axis below 0; multiple findings sum but are clamped at the end.
  3. The penalty model is swappable. BeliefPenalty::user_supplied(...) lets a team install a custom ladder; the audit then records the penalty model’s identity. The default ladder is just one entry in a registry.

How propagation is modeled

The lineage DAG carries belief from impressions to ideas via the meet-and-transform_factor rule. Here’s an evidence-propagation diagram for a 4-step pipeline:

flowchart LR
    I["events.csv<br/><b>impression</b><br/>[1.00, 1.00, 1.00, 1.00,<br/>1.00, 1.00, 1.00, 1.00]"]:::imp
    I -->|"observe()<br/>tf = identity"| O["raw_events<br/><b>idea</b><br/>[1.00, 1.00, 1.00, 1.00,<br/>0.95, 1.00, 1.00, 1.00]"]:::idea
    O -->|"filter(active==1)<br/>tf = identity"| F["active_events<br/><b>idea</b><br/>[1.00, 0.99, 1.00, 1.00,<br/>0.95, 0.98, 1.00, 1.00]"]:::idea
    F -->|"group_by(plan_tier)<br/>tf: sample × 0.80"| G["grouped<br/><b>idea</b><br/>[1.00, 0.99, 1.00, 1.00,<br/>0.95, 0.78, 1.00, 1.00]"]:::idea
    G -->|"with_column(churn_rate)<br/>tf: leakage × 0.90"| W["labeled<br/><b>idea</b><br/>[1.00, 0.99, 1.00, 0.90,<br/>0.95, 0.78, 1.00, 1.00]"]:::idea

    classDef imp fill:#e8f4fd,stroke:#2196F3,stroke-width:2px
    classDef idea fill:#fff3e0,stroke:#FF9800,stroke-width:1px

Evidence propagation through a 4-step pipeline. Each node carries an 8-dim belief vector [schema, missingness, drift, leakage, lineage, sample, duplication, constraint]. Edges apply transform_factor. Belief is monotonically non-increasing along any path — the rightmost node is component-wise ≤ the leftmost.

Reading the diagram: each node label lists the 8-axis belief vector in fixed order. The observe() step pre-weakened lineage to 0.95 because raw-CSV ingestion drops some provenance (no upstream observation timestamps). The filter step weakened missingness and sample because filtering revealed a column with a higher local NaN rate, and dropped row count. The group_by step weakened sample further because per-group sample sizes are smaller. The with_column(churn_rate) step weakened leakage because the derived column directly references the target.

Crucially: every axis-wise value on the rightmost node is ≤ the leftmost. Composition never improves belief on any axis.

Deterministic update rules

Every step in the propagation uses:

  • Kahan summation for penalty accumulation (avoid FP-order-dependent drift).
  • BTreeMap for finding aggregation by axis (no HashMap iteration nondeterminism).
  • f64::total_cmp for floating comparisons (NaN behaves consistently, not implementation-defined).
  • No FMA in the SIMD paths under penalty arithmetic (bit-identical across CPUs).
  • Single-threaded execution in v0.1 — even when parallel column-validation lands in v0.2, the reduction phase will be order-fixed.

The contract: same input bytes → same belief vector. The lineage diagram above is reproducible byte-for-byte by anyone running the same pipeline on the same inputs.

Confidence algebra (shipped vs planned)

The composition operator meet plus the identity ⊤ = [1, 1, 1, 1, 1, 1, 1, 1] satisfies five algebraic properties today:

Property Statement
Identity meet(b, ⊤) = b
Idempotence meet(b, b) = b
Monotonicity meet(b, c) ≤ b component-wise
Commutativity meet(b, c) = meet(c, b)
Associativity meet(meet(a, b), c) = meet(a, meet(b, c))

Together, (BeliefScore, meet, ⊤) is a commutative idempotent monoid with identity — i.e. a meet-semilattice in lattice-theoretic terms. The deterministic update rules above make the algebra computable to bit-identical results across runs. That’s the formal object the snapshot-and-diff workflow relies on; without monotonicity, “appeared / disappeared” diffs over composed pipelines wouldn’t be well-defined.

Planned for v0.2:

  • Per-axis composition rules instead of a global meet. Some axes (e.g. lineage) compose by min; others (e.g. sample) should compose by product. A typed rule per axis would be more faithful to what each axis measures.
  • A join operator for combining alternative views of the same dataset (the lattice-theoretic dual of meet). Currently absent; the algebra is meet-only, which means alternative-view combination is undefined.
  • A continuous-domain semantics for transformations that span a parameter (e.g. filter(predicate) with predicate selectivity s ∈ [0, 1]). v0.1 treats every filter the same; v0.2 would parameterise transform_factor by s.
  • A formal proof of the conservation property: under the meet algebra, the gate-diff workflow’s appeared set is exactly the set of findings whose axes degraded between snapshots. This is true by construction today; v0.2 would have it written down as a theorem with a checked proof.

The honest framing: Locke v0.1 ships a small, deliberately conservative algebra. It is enough to make the workflows in this post (“snapshot, diff, gate, audit”) provably reproducible and well-defined. The richer semantics will land when the workflows the algebra is supposed to serve are more developed.

Belief degradation, visualised

The same 4-step pipeline rendered as a degradation chart — one panel per axis, showing the trajectory of each component as we move from impression to most-derived idea:

flowchart LR
    subgraph S1["events.csv"]
        s1_schema["schema = 1.00"]:::held
        s1_miss["missingness = 1.00"]:::held
        s1_drift["drift = 1.00"]:::held
        s1_leak["leakage = 1.00"]:::held
        s1_lin["lineage = 1.00"]:::held
        s1_sam["sample = 1.00"]:::held
        s1_dup["duplication = 1.00"]:::held
        s1_con["constraint = 1.00"]:::held
    end

    subgraph S2["raw_events<br/>(observe)"]
        s2_schema["1.00"]:::held
        s2_miss["1.00"]:::held
        s2_drift["1.00"]:::held
        s2_leak["1.00"]:::held
        s2_lin["0.95"]:::weakened
        s2_sam["1.00"]:::held
        s2_dup["1.00"]:::held
        s2_con["1.00"]:::held
    end

    subgraph S3["active_events<br/>(filter)"]
        s3_schema["1.00"]:::held
        s3_miss["0.99"]:::weakened
        s3_drift["1.00"]:::held
        s3_leak["1.00"]:::held
        s3_lin["0.95"]:::held
        s3_sam["0.98"]:::weakened
        s3_dup["1.00"]:::held
        s3_con["1.00"]:::held
    end

    subgraph S4["grouped<br/>(group_by)"]
        s4_schema["1.00"]:::held
        s4_miss["0.99"]:::held
        s4_drift["1.00"]:::held
        s4_leak["1.00"]:::held
        s4_lin["0.95"]:::held
        s4_sam["0.78"]:::weakened
        s4_dup["1.00"]:::held
        s4_con["1.00"]:::held
    end

    subgraph S5["labeled<br/>(with_column)"]
        s5_schema["1.00"]:::held
        s5_miss["0.99"]:::held
        s5_drift["1.00"]:::held
        s5_leak["0.90"]:::weakened
        s5_lin["0.95"]:::held
        s5_sam["0.78"]:::held
        s5_dup["1.00"]:::held
        s5_con["1.00"]:::held
    end

    s1_schema --- s2_schema --- s3_schema --- s4_schema --- s5_schema
    s1_miss   --- s2_miss
    s2_miss   -.-> s3_miss
    s3_miss   --- s4_miss --- s5_miss
    s1_drift  --- s2_drift  --- s3_drift  --- s4_drift  --- s5_drift
    s1_leak   --- s2_leak   --- s3_leak   --- s4_leak
    s4_leak   -.-> s5_leak
    s1_lin    -.-> s2_lin
    s2_lin    --- s3_lin    --- s4_lin    --- s5_lin
    s1_sam    --- s2_sam
    s2_sam    -.-> s3_sam
    s3_sam    -.-> s4_sam
    s4_sam    --- s5_sam
    s1_dup    --- s2_dup    --- s3_dup    --- s4_dup    --- s5_dup
    s1_con    --- s2_con    --- s3_con    --- s4_con    --- s5_con

    classDef held fill:#e8f5e9,stroke:#2E7D32
    classDef weakened fill:#ffebee,stroke:#C62828

Belief degradation across the 4-step pipeline above. Each axis’s value at each stage is shown as a labelled node; the layout reads left-to-right (pipeline stages) and top-to-bottom (axes). Solid edges indicate the value held; dashed edges indicate a transform_factor weakening. Schema, drift, duplication, and constraint hold at 1.00 throughout. Missingness ticked down once. Lineage dropped at ingest and held. Sample dropped twice (filter, group_by). Leakage dropped once (with_column).

The chart makes one specific claim visually: the dashed edges are the only places confidence got weaker. Every other axis held flat. A reviewer can pin a downstream finding to the specific transformation that degraded the relevant axis, because the dashed edge is exactly where the audit trail records a transform_factor application.

This is the property that makes the audit trail useful, not just reproducible. Reproducibility says “two runs of the same pipeline match.” Localisability says “when something downstream is wrong, I can point to which step weakened the belief on the affected axis.” Locke gives both.

Where the formal model lives in code

If you want to read the actual implementation rather than the prose:

  • crates/cjc-locke/src/belief.rs — BeliefScore, BeliefWeights, BeliefPenalty definitions and the meet operator.
  • crates/cjc-locke/src/lineage.rs — LineageGraph, LockeImpression, LockeIdea, edge representations, content-addressed IDs.
  • crates/cjc-locke/src/traced.rs — TracedDataFrame wrapper that applies transform_factor per operation.
  • crates/cjc-locke/src/dispatch.rs — the 22 locke_* builtins that expose the model to .cjcl source.

Everything is pub and documented. The composition rules are unit-tested for monotonicity, idempotence, and associativity. The deterministic-update rules are property-tested by proptest. The “shipped vs planned” gaps above each have an issue or an // TODO(v0.2): marker in the source.

Install

cargo add cjc-locke

The crate is at version 0.1.10 on crates.io and works with the matching version of cjc-data, cjc-repro, cjc-runtime from the same workspace release. If you want the CLI:

cargo install cjc-lang
cjcl locke validate your-data.csv

The benchmark code is in the repo at bench/locke_vs_gx/. Anyone with pip install great_expectations pandas can re-run the full comparison.

Where v0.2 is going

The next release will close the gaps the benchmark exposed:

  • Native Parquet decoder (Thrift + Snappy + PLAIN encoding minimum)
  • Value-set-membership checks at the validate path (matching GX’s ExpectColumnDistinctValuesToBeInSet)
  • Multi-class target-leakage AUC (v0.1 supports binary targets only)
  • HTML output with per-column distribution sparklines via cjc-vizor
  • Optional parallel column-validation via rayon (per-column work is embarrassingly parallel)
  • JUnit XML and SARIF output for CI integrations

If a Locke check you wanted is missing, the issue tracker is open.

Final honest framing

I built Locke because CJC-Lang needed a data-skepticism layer that preserved the runtime determinism guarantee at the validation boundary. The benchmark against Great Expectations was a calibration exercise — I wanted to know whether the “Locke is competitive” claim I’d been making was actually true.

The answer, on the specific workflows Locke is designed for, is yes. Locke matched GX 5 / 5 on every ground-truth fact in a real benchmark, ran 275× faster end-to-end, and fired more checks out of the box for the specific failure modes that break customer-churn pipelines.

That’s not a knock on Great Expectations, dbt, Pandera, or Monte Carlo. Each of those tools is mature, well-staffed, and the right answer at its layer of the stack. Locke doesn’t compete with any of them — it adds an epistemic layer underneath where the audit artifact is the deliverable. If your data lives in Snowflake and your team needs Slack alerting, use GX or Monte Carlo. If your pipeline is pandas-native and you want in-process guards, use Pandera. If the validation belongs inside dbt’s DAG, use dbt tests.

If your data fits in a CSV / JSONL, you need a deterministic CI gate, and you’re tired of validating customer-churn pipelines with spreadsheets and prayer, use Locke.

The code is on crates.io. The benchmark is in the repo. The whole thing is MIT-licensed.

cjcl locke validate your-data.csv and tell me what breaks.