The handoff said X. The data said Y.

I’d told the handoff E9061 (suspicious target leakage, |AUC| in [0.85, 0.95]) would fire on out_prncp and total_pymnt_inv — partial signals of the payment streams. E9060 (the strong-leakage code, |AUC| ≥ 0.95) would fire on the payment streams proper.

What actually fired:

E9060 — the “near-deterministic” leakage code at the |AUC| ≥ 0.95 threshold — fired zero times. Every post-origination column the handoff named landed below the 0.85 threshold and produced no finding at all (E9061 is continue in the loop, not an Info-severity ack).

The top two signals were last_fico_range_high and last_fico_range_low — the borrower’s last credit-bureau refresh, recorded after the loan exists. The handoff hadn’t predicted those columns. It had predicted total payment amounts.

The retrospective interpretation is sensible: payment amounts are dominated by loan size and term duration; their distributional spread under both terminal outcomes overlaps significantly, dragging AUC down to the sub-threshold band. The borrower’s post-loan FICO score, by contrast, collapses sharply when the loan is charged off (delinquency reports update the bureau record), so the FICO refresh column carries a much cleaner separation signal.

Locke disagreed with the analyst, and Locke was right. This is the first finding worth banking: not “Locke caught what the analyst predicted” but “Locke surfaced something the analyst’s reasoning missed.” A tool that updates the analyst’s priors is more valuable than one that confirms them.

Other useful surprises in the same run

• E9072 (ID-like cardinality, distinct / n_rows ≥ 0.95) fired on id (the loan key, expected) AND on url (unexpected, but the LC URL embeds the loan ID — so it inherits the cardinality). member_id did not fire because the same member can hold multiple loans.
• E9001 (high missingness) fired on the joint-application column set (annual_inc_joint, dti_joint, verification_status_joint) at ~85% missingness, matching the empirical ratio of individual-to-joint loans.
• E9008 (auto-detected string sentinel) fired on emp_length for "n/a" and on the joint columns for "".

But it also under-fired in two ways the handoff had anticipated, and getting those right matters for the rest of the post:

• E9008 did not fire on term despite its leading-space " 36 months" format. The detector matches against BUILTIN_STRING_SENTINELS exactly ("", "?", "NA", etc.) — leading whitespace on a domain-meaningful value isn’t a sentinel by Locke’s definition. The handoff’s prediction here conflated “sentinel” with “formatting artifact.”
• E9070 (conditional missingness on (A, B) pairs) did not fire on the joint columns — a real gap that would later turn out to be two gaps. We come back to this in §5.

The empirical fixture I committed (expected_findings.json) was calibrated against measured reality, not the handoff’s predictions. The doc-vs-data divergence is itself the artifact: a future contributor reading the fixture sees the predictions that didn’t pan out and the empirical replacement.

What was available in cjc-runtime

This is one of the moments where having the underlying language stack pays off. The four primitives needed already existed:

// From cjc-runtime
pub fn logistic_regression(x_flat: &[f64], y: &[f64], n: usize, p: usize) -> Result<LogisticResult, String>;
pub fn auc_roc(scores: &[f64], labels: &[bool]) -> Result<f64, String>;
pub fn train_test_split(n: usize, test_fraction: f64, seed: u64) -> (Vec<usize>, Vec<usize>);
pub fn z_score(data: &[f64]) -> Result<Vec<f64>, String>;

logistic_regression is IRLS (Iteratively Reweighted Least Squares) with the intercept auto-added. auc_roc is the trapezoidal ROC integration with f64::total_cmp tie-breaking — deterministic. train_test_split uses a seeded Fisher-Yates shuffle. z_score does Kahan-summed mean + standard deviation.

That covers everything needed for a deterministic tabular-ML evaluator. The demo’s honest_model binary is ~300 LOC of Rust orchestrating these primitives:

// Pseudocode, not the actual source.
let (raw_df, _) = load_csv_gz(input, Some(load_cap))?;
let df = binarize_loan_status(raw_df)?;
let sample_indices: Vec<usize> = (0..actual_sample).collect();
let (train_local, test_local) = train_test_split(sample_indices.len(), test_fraction, seed);

for (label, exclusion_set) in [
    ("pre-Locke (naive)", vec![]),                                  // 87 cols
    ("Locke-filtered",     LOCKE_E9061_COLUMNS.to_vec()),           // 84 cols (–3)
    ("domain-honest",      DOMAIN_POST_ORIGINATION_COLUMNS.to_vec()), // 75 cols (–12)
] {
    let (names, columns) = select_numeric_columns(&df, &full_excludes);
    let (names, columns) = drop_useless_columns(names, columns);
    // Per-column: fit mean/std on train only; apply to train + test.
    // No information leak across the split.
    let logreg = logistic_regression(&x_train_flat, &y_train, n_train, p)?;
    let eta = score_logistic(&logreg.coefficients, &x_test_flat, n_test, p);
    let auc = auc_roc(&eta, &y_test_bool)?;
    // ...
}

The deliberate constraints: 200K-row stratified subsample (because IRLS at full n × p² is slow), seed = 42 fixed, 70/30 train/test split, z-score normalization fit on train only.

The three-way comparison

=== Summary ===
variant                     |AUC|    n_train     n_test          p
pre-Locke (naive)          0.9993     140000      60000         87
Locke-filtered             0.9995     140000      60000         84
domain-honest              0.7394     140000      60000         75

Pre-Locke (the naive model). 87 numeric columns; only the target column itself and the two ID columns excluded. Test |AUC| = 0.9993. IRLS hits the iteration cap of 100 without converging. The model is catastrophically over-fit via leakage — but more interestingly, the exact form of catastrophe is the post-origination columns moving together as a multicollinear cluster. The X^T W X matrix in IRLS becomes near-singular; Newton steps stop converging.

Locke-filtered (E9061 only). Same setup minus last_fico_range_high, last_fico_range_low, total_rec_prncp — the three columns Locke flagged at |AUC| ≥ 0.85. Test |AUC| = 0.9995. Essentially unchanged from naive. IRLS still hits the iteration cap.

Domain-honest (data dictionary). Same setup minus the full handoff §3.3 post-origination list: 12 columns covering payment streams, recoveries, outstanding principal, and the last-refreshed FICO. Test |AUC| = 0.7394. IRLS converges cleanly in 14 iterations.

The headline three-way table prints in the binary’s CLI output and lands in cross_validate.md §3.1.

The finding

Locke’s |AUC| ≥ 0.85 heuristic catches the strongest leakage but not all of it. The columns that survive the Locke filter (total_pymnt, out_prncp, total_pymnt_inv, last_pymnt_amnt, recoveries, etc.) carry enough leakage signal jointly to keep the model’s AUC inflated. Removing them — via the domain-knowledge list — is what actually collapses AUC into the honest band.

This is not a bug in Locke. Locke’s per-feature ROC AUC computation is correct. The post-origination columns are leaky; their individual AUCs are merely in [0.5, 0.85) because each carries a noisy signal that happens to align with a sibling column. The detector’s threshold is doing what it was designed to do — fire on individually-strong signals. The architectural question is whether Locke should also detect jointly-strong-but-individually-weak leakage. v0.8 doesn’t change that.

What v0.8 does change is the framing of Locke’s value claim.

The naive framing is “Locke catches leakage.” That doesn’t survive contact with the LC follow-up. The Locke-filtered AUC stays at 0.9995. A reviewer who tried this would conclude the tool failed.

The honest framing is “Locke shows you where to look; the analyst applies domain triage.” That survives the LC follow-up because the data shows exactly that. last_fico_range_high jumping out at |AUC| 0.94 is the analyst’s cue to ask: is this column recorded before or after the loan? The answer (after) implies a fuller exclusion list — the kind of list DOMAIN_POST_ORIGINATION_COLUMNS encodes. Without Locke, the analyst might never have realized the last-refreshed FICO range was post-origination at all.

But that framing implies a missing piece: the analyst’s domain triage has nowhere to live inside Locke. It lives in their own code, as a separate exclusion list maintained per project. The triage doesn’t compose with the belief algebra, doesn’t get fingerprinted, doesn’t show up in the canonical JSON, doesn’t get caught by regression gates if the domain knowledge changes.

That gap is what ADR-0041 closes.

5.1 Code namespace E9500..=E9999

Locke’s built-in codes occupy E9001..=E9112. The custom range starts safely above them and leaves 500 slots, with the upper 4000 reserved for future framework use. Reading a report, you can always tell which findings came from Locke’s opinionated baseline (E9xxx with xxx < 9500) versus a team’s custom rules (xxx >= 9500).

The validator validate_custom_code(&str) -> Result<u32, CustomDetectorError> is the single point of truth. Both the Rust-side registration and the Python bridge call it, so error messages are uniform. Codes outside the namespace are rejected at registration time (not at run time, which would be a footgun) with a clear CustomDetectorError::CodeOutOfNamespace { code, reason }. The result type is exposed so callers can inspect rejection reasons without trying to parse error strings.

5.2 FindingSink as the only emission surface

The sink is the restricted finding-emission surface. emit() is the only mutation method:

pub fn emit(
    &mut self,
    severity: FindingSeverity,
    message: impl Into<String>,
    column: Option<String>,
    row_range: Option<(usize, usize)>,
    evidence: Vec<FindingEvidence>,
    sample_size: u64,
) -> usize;

All canonicalization (fingerprint composition, byte ordering) happens inside emit. The detector cannot construct findings directly. This is what makes determinism a structural guarantee rather than a trust contract — you can’t write a non-deterministic custom detector that produces non-deterministic output, even if you try. The sink also rejects two cases:

• Empty message. A finding without a message is unhelpful; rejected with CustomDetectorError::EmptyMessage.
• Non-Info severity from an axes-empty detector. A detector that declared BeliefAxisSet::NONE produces findings that affect no belief score (advisory-only). Allowing it to emit Warning- or Error-severity findings would be a self-contradiction — the finding would appear in the report but couldn’t drag the score. Rejected with CustomDetectorError::NonAdvisoryAxesEmpty.

Both errors are recorded, not raised. The detector’s run() continues; the framework drains the errors after run() returns and surfaces them in LockeReport.assumptions.

5.3 Belief-axis routing via BeliefAxisSet

The 8 Locke belief axes (schema, missingness, drift, leakage, lineage, sample, duplication, constraint) are represented as a u16 bitset:

pub struct BeliefAxisSet(pub u16);

impl BeliefAxisSet {
    pub const SCHEMA: Self      = Self(1 << 0);
    pub const MISSINGNESS: Self = Self(1 << 1);
    pub const DRIFT: Self       = Self(1 << 2);
    pub const LEAKAGE: Self     = Self(1 << 3);
    // ...
    pub const NONE: Self        = Self(0);
}

When the framework runs a custom detector, it records the detector’s declared axes in a new field on LockeReport:

pub struct LockeReport {
    // ... existing fields ...
    pub custom_axis_assignments: BTreeMap<String, BeliefAxisSet>,
}

The belief composition (the meet-semilattice algebra over (BeliefScore, meet, ⊤=[1,...,1]) from ADR-0036) gets one new closure:

let custom_contains = |code: &str, axis: BeliefAxisSet| -> bool {
    report.custom_axis_assignments
        .get(code)
        .map(|axes| axes.contains(axis))
        .unwrap_or(false)
};

Then each per-axis filter ORs the existing built-in code matchers with a custom_contains(code, AXIS) check. Built-in codes still match by name; custom codes match by their declared axes. No restructuring of the algebra was needed — the closure-based filter pattern in penalty_from_findings_with_model extends cleanly.

The empty-map default is byte-identical to pre-v0.8 belief composition when no custom detectors are registered.

5.4 The Python bridge: structural determinism across the FFI boundary

The most architecturally interesting piece is the Python side. The naive PyO3 approach — give Python a mutable reference to the Rust FindingSink — runs into lifetime restrictions (#[pyclass] can’t hold borrowed references that might outlive the Python object). The path I took is a two-staged collector:

#[pyclass(name = "CustomDetectorSink", module = "cjc_locke")]
struct PyDetectorSink {
    pending: Vec<PyEmittedFinding>,
    code: String,
    axes_empty: bool,
    last_error: Option<String>,
}

The Python detector’s run(df, sink) receives a PyDetectorSink it can call emit(...) on; the emit pushes into pending but does NOT yet construct a canonical ValidationFinding. After Python’s run() returns, the Rust adapter drains pending into the real Rust FindingSink:

impl CustomDetector for PyCustomDetectorAdapter {
    fn run(&self, df: &DataFrame, sink: &mut FindingSink) {
        Python::with_gil(|py| {
            let view = Py::new(py, PyDetectorDataFrame { inner: Arc::new(df.clone()) })?;
            let py_sink = Py::new(py, PyDetectorSink { /* ... */ })?;
            self.instance.bind(py).call_method1("run", (view, py_sink.clone_ref(py)))?;
            // Drain pending emissions into the Rust sink. Canonicalization
            // happens at the Rust-side emit call.
            for emission in py_sink.bind(py).borrow().pending.clone() {
                sink.emit(emission.severity, emission.message, emission.column, /* ... */);
            }
        });
    }
}

This is the architectural sleight-of-hand: the framework canonicalizes the Python detector’s findings on the Rust side after Python returns. The user-supplied Python code can be as non-deterministic as it wants — random.shuffle, set iteration, dict order from PYTHONHASHSEED — and the framework still produces byte-identical reports across runs as long as the set of findings is the same.

This is provable. The pytest in python/tests/test_custom_detectors.py:

class _ShuffleEmissionDetector(cjc_locke.CustomDetector):
    """Emits findings in a randomised order each run."""
    def __init__(self, seed: int):
        self._rng = random.Random(seed)

    def code(self): return "E9502"
    def belief_axes(self): return ["leakage"]

    def run(self, df, sink):
        names = list(df.column_names())
        self._rng.shuffle(names)            # explicit non-determinism
        for n in names:
            sink.emit("warning", f"shuffle finding on `{n}`",
                      column=n, sample_size=df.n_rows)


def test_determinism_under_emission_shuffle(lc_like_data):
    d1 = _ShuffleEmissionDetector(seed=1)
    d2 = _ShuffleEmissionDetector(seed=2)   # different shuffle seed
    r1 = cjc_locke.validate(lc_like_data, custom_detectors=[d1])
    r2 = cjc_locke.validate(lc_like_data, custom_detectors=[d2])
    assert r1.to_json() == r2.to_json()     # byte-identical

This is the credibility-anchor test for the whole Python bridge. The detector emits findings in a random.Random-shuffled order each call — two different shuffle seeds in two different runs — and the resulting JSON output is byte-identical. The framework’s canonical sort on the Rust side masks Python’s emission order entirely.

Without this property, “Locke determinism survives the Python boundary” would be a brittle promise. With it, it’s a structural guarantee.

The data path that makes that work:

sequenceDiagram
    autonumber
    participant Py as Python user code
    participant Sink as PyDetectorSink<br/>(collector)
    participant Adapter as PyCustomDetectorAdapter<br/>(implements Rust trait)
    participant Rust as Rust FindingSink
    participant Report as LockeReport JSON

    Note over Py: Free to use random.shuffle,<br/>set iter, dict order
    Py->>Sink: sink.emit("warning", msg₁, ...)
    Py->>Sink: sink.emit("warning", msg₃, ...)
    Py->>Sink: sink.emit("warning", msg₂, ...)
    Note over Sink: pending = [f₁, f₃, f₂]<br/>non-canonical order
    Py-->>Adapter: run() returns
    Adapter->>Rust: drain pending → emit each
    Note over Rust: canonical fingerprint<br/>per finding
    Rust->>Report: findings.sort_by(sort_key)
    Note over Report: byte-identical across runs,<br/>regardless of Python order

The “non-canonical order” → “canonical sort” transition is the structural fix. The user is free to emit in whatever order they like; the framework reorders before the bytes leave Rust.

5.5 The Python ABC

The Python facade is intentionally small. Three methods, no metaclass magic:

class CustomDetector:
    """Base class for user-defined Locke detectors (ADR-0041)."""

    def code(self) -> str:
        raise NotImplementedError(...)

    def belief_axes(self) -> List[str]:
        # Names from {schema, missingness, drift, leakage, lineage,
        # sample, duplication, constraint}. Empty list = advisory-only.
        raise NotImplementedError(...)

    def run(self, df, sink) -> None:
        raise NotImplementedError(...)

A working detector subclass is ~10 lines:

class PostOriginationByName(cjc_locke.CustomDetector):
    def code(self) -> str:
        return "E9500"

    def belief_axes(self) -> list[str]:
        return ["leakage"]

    def run(self, df, sink) -> None:
        for col in df.column_names():
            if col.startswith(("total_", "last_pymnt_", "recoveries")):
                sink.emit("error",
                          f"`{col}` matches a post-origination naming pattern",
                          column=col, sample_size=df.n_rows)

Wiring it into a validate call:

report = cjc_locke.validate(
    data,
    label="lc",
    custom_detectors=[PostOriginationByName()],
)

Axis names cross the boundary as strings. Unknown names (e.g. "nonsense") raise ValueError at registration — the underlying BeliefAxisSet::from_names returns a structured Err which the bridge converts to a PyValueError.

5.6 The 4-way LC re-measurement

The reason the layer exists is to close the LC gap. The Rust-side reference implementation is in the demo crate itself (demos/lendingclub/src/lib.rs):

#[derive(Debug, Default)]
pub struct PostOriginationByNameDetector;

impl PostOriginationByNameDetector {
    pub fn is_post_origination(name: &str) -> bool {
        const PREFIX_PATTERNS: &[&str] = &[
            "total_", "last_pymnt_", "last_fico_range_",
            "out_prncp", "debt_settlement_",
            "hardship_", "settlement_",
        ];
        const EXACT_PATTERNS: &[&str] = &[
            "recoveries", "collection_recovery_fee", "debt_settlement_flag",
        ];
        EXACT_PATTERNS.iter().any(|p| *p == name)
            || PREFIX_PATTERNS.iter().any(|p| name.starts_with(p))
    }
}

impl cjc_locke::custom_detector::CustomDetector for PostOriginationByNameDetector {
    fn code(&self) -> &'static str { "E9500" }
    fn belief_axes(&self) -> BeliefAxisSet { BeliefAxisSet::LEAKAGE }
    fn run(&self, df: &DataFrame, sink: &mut FindingSink) {
        for (name, _col) in &df.columns {
            if Self::is_post_origination(name) {
                sink.emit(
                    FindingSeverity::Error,
                    format!("column `{}` matches a known post-origination naming pattern", name),
                    Some(name.clone()),
                    None,
                    vec![],
                    df.nrows() as u64,
                );
            }
        }
    }
}

The CLI gets a --use-custom-detectors flag that wires this detector into ValidateOptions::custom_detectors. On the full LC dataset:

E9500 × 39 — the custom detector fires 39 times, flagging 39 columns by name.

The honest-model harness gets a --from-report <path> flag that reads the report JSON, extracts every column with an E9500 or E9061 finding, and uses that as the exclusion set for a fourth model variant.

The four-way result, same 200K-row sample, same seed=42:

=== Summary ===
variant                     |AUC|    n_train     n_test          p   iter
pre-Locke (naive)          0.9993     140000      60000         87    100
Locke-filtered             0.9995     140000      60000         84    100
Locke + custom det.        0.7388     140000      60000         68     17  ← closes the gap inside Locke
domain-honest              0.7394     140000      60000         75     14

Locke + custom det. at |AUC| = 0.7388 lands within 0.0006 of the hand-curated domain-honest baseline at 0.7394. The custom detector recovered honest credit-risk performance entirely inside Locke configuration — using a ~30-LOC detector to encode the same domain knowledge that previously lived as a separate hard-coded exclusion list in the modeling code.

Two further notes on this table:

• The custom-detector variant uses FEWER features (68 vs 75). It’s more aggressive about exclusion (39 columns flagged vs the handoff’s 12, because the custom detector catches sec_app_*, hardship_*, settlement_* etc. that the handoff’s list didn’t anticipate). And yet it achieves essentially the same AUC. That’s empirical evidence the extra exclusions weren’t carrying useful signal — the custom detector is at least as conservative as expert curation.
• IRLS convergence speed is a real diagnostic. The leaky models hit the 100-iter cap because the multicollinear post-origination cluster makes X^T W X near-singular. The honest variants both converge in 14–17 iterations because removing that cluster conditions the matrix well. Convergence speed is a proxy for “how well did the exclusion set capture the multicollinear leakage subspace” — and Locke + custom converged in 17 iters, comparable to domain-honest’s 14.

5.7 Tests, in three columns

The integration test file leans on a NameMatchDetector helper plus ProptestDetector plus ShuffleEmissionDetector — three detector subclasses that exercise different aspects of the trait. Three of the integration tests are bolero fuzz tests that exercise the registration namespace validator (validate_custom_code) and the emit input validation under arbitrary byte sequences. None of the fuzz inputs panic; either the input is valid (no error recorded) or it’s invalid (error recorded, no finding constructed). Determinism is also exercised in the Rust proptest: random axes + emit count + arbitrary DataFrame → byte-identical JSON across runs.

Workspace test totals after ADR-0041 alone: cjc-locke lib passes 423 (was 408, +15); tests/locke integration passes 291 (was 277, +14); zero regressions in pre-existing tests.

6.1 Gap one: type inference from the first data row

cjc-data::CsvReader types each column from its first data row only. For annual_inc_joint, the first row’s value is "" (empty string — the loan was an individual application, not joint). infer_type("") returns InferredType::Str. Once the column is Str, every subsequent numeric value ("100000", "50000.0", etc.) gets stored as a string. Downstream Float-only detectors skip the column.

The fix lives in a new module crates/cjc-locke/src/auto_promote.rs. Each Str column is scanned and each value classified as:

• Sentinel — matches the existing BUILTIN_STRING_SENTINELS ("", "?", "NA", "N/A", "NULL", "null", "nan", "NaN", "None", "-") or cfg.additional_sentinels.
• Parseable — str::parse::<f64>() succeeds. Accepts integer and float literals, including scientific notation (1.5e-3).
• Other — text, malformed numbers, anything else.

A column is promoted when:

• non_sentinel_rows >= cfg.min_non_sentinel_rows_for_promotion (default 10), AND
• n_parseable / non_sentinel >= cfg.min_parseable_fraction_for_promotion (default 0.80).

Promoted columns are rebuilt as Column::Float where sentinels and unparseable values become NaN. Each emits one E9009 (Info severity) finding with full evidence (counts, fraction).

pub fn auto_promote_str_columns(
    df: &DataFrame,
    cfg: &ValidationConfig,
) -> (Option<DataFrame>, Vec<ValidationFinding>);

The Option<DataFrame> return is the zero-allocation no-op path: if no column qualifies, returns None and the caller uses the original DataFrame without any clone cost. The wiring in api::validate is the working_df pattern:

let (maybe_promoted_df, promo_findings) =
    crate::auto_promote::auto_promote_str_columns(df, &opts.config);
let working_df: &DataFrame = maybe_promoted_df.as_ref().unwrap_or(df);

let mut findings = validate_dataframe(working_df, ...);
findings.extend(promo_findings);
// All downstream detectors + column reports + custom detectors use working_df.

The report’s input.column_types reflects post-promotion types, so JSON consumers see the right view. The default auto_promote_str_to_float = true was the deliberate choice — a flag that’s off by default doesn’t fix the gap for users who don’t know it exists.

6.2 Gap two: E9070 was defined but never invoked

Here’s where the investigation got interesting. After adding the auto-promotion, I ran the LC demo with promotion enabled. E9070 (conditional missingness) didn’t fire. Promotion was working — column types changed, joint columns were Float, NaN positions were correct — but E9070 stayed at zero.

The cause was simpler and more embarrassing than the typing fix:

$ grep -n "detect_conditional_missingness\b" crates/cjc-locke/src/validation.rs
1425:pub fn detect_conditional_missingness(
2398:    let r = detect_conditional_missingness(&df, ...);  # inside a unit test
2420:    let r = detect_conditional_missingness(&df, ...);  # inside a unit test
2771:    let r = detect_conditional_missingness(            # inside a unit test

detect_conditional_missingness was defined, exported, unit-tested. But the call site inside validate_dataframe — the function that orchestrates the actual validation pipeline — never invoked it. The detector had been silently zero-firing for everyone, on every input, since v0.6.4. The unit tests that exercised it caught the function’s own behavior; they didn’t catch the missing wire.

The fix is one out.extend(...) line:

out.extend(detect_missingness(df, cfg, &effective_masks));
out.extend(detect_conditional_missingness(   // <-- new
    df,
    &ConditionalMissingnessConfig::default(),
    &effective_masks,
));
out.extend(detect_duplicates_full_row(df, cfg));

The detector honors effective_masks (the merged caller + auto-sentinel mask map), so it works for both Float-NaN columns and Str-with-mask columns. The opt-out path — disable auto-promotion via ValidationConfig::auto_promote_str_to_float = false — still fires E9070 via the sentinel-mask path. Both code paths produce findings; the auto-promotion makes them more numerous and the per-column missingness rates cleaner.

6.3 The real LC measurement post-fix

Re-running lendingclub_demo on the full 2.26M-row CSV with v0.8 enabled:

The 22 columns the auto-promotion fires on:

• sec_app_* (12 columns) — secondary applicant fields: fico_range_high/low, mort_acc, num_rev_accts, open_acc, open_act_il, inq_last_6mths, chargeoff_within_12_mths, collections_12_mths_ex_med, mths_since_last_major_derog, revol_util
• hardship_* (5 columns) — workout plan fields: amount, dpd, length, payoff_balance_amount, last_payment_amount
• settlement_* (3 columns) — debt settlement fields: amount, percentage, term
• revol_bal_joint, orig_projected_additional_accrued_interest, mths_since_last_record

E9070’s 1879 firings span ~54 unique columns. Most pairs are within the joint/secondary/hardship/settlement set: every co-missing pair fires twice (a→b and b→a are separate findings). For users who want a smaller set, raising ConditionalMissingnessConfig::implication_threshold above 0.95 or post-hoc filtering by column-pair is the right knob.

6.4 Gap three: CSV quoted-string handling (deferred)

annual_inc_joint, dti_joint, verification_status_joint — the three columns the handoff originally named — are NOT in the v0.8 promotion set. Investigation:

Sampled 50000 rows of annual_inc_joint:
  empty:     49290
  parseable: 248
  other:     462
  non-parseable samples:
    'Individual'
    'Individual'
    'Individual'
    'Individual'
    'Individual'

The non-sentinel rows split 248 parseable vs 462 “Other” — parseable_fraction = 0.349, well below the 0.80 threshold. So promotion correctly declines.

The “Other” values are 'Individual' — the string from the LC dataset’s application_type column. They’re bleeding into annual_inc_joint because cjc-data::CsvReader does NOT support RFC-4180 quoted strings. LC’s desc (free-text loan description) column contains embedded commas; without quote-handling, every cell after desc in such a row gets shifted left by however many commas the description contained. application_type (which sits after desc in column order) leaks into annual_inc_joint’s position.

This is a real third gap, but it lives in cjc-data, not cjc-locke. ADR-0042 documents it as a known limitation and bounds it: the parseable-fraction guard correctly refuses to promote a column polluted by column-shift, which is the right defensive behavior. The upstream CSV fix is tracked separately. The right user workaround until then is to pre-clean the CSV with a parser that understands quoting (Python’s csv module, pandas) and pass the cleaned dict to Locke via the Python bridge or via DataFrame::from_columns.

6.5 Discovery cascade, visualized

flowchart TD
    A[Handoff: Project A = LC demo<br/>predicted top leakage = payment streams] -->|build demo| B[E9061 fires on last_fico_range_*<br/>not on payment streams]
    B -->|honest-model AUC follow-up| C[Locke-filtered AUC = 0.9995<br/>Locke flags alone don't recover honesty]
    C -->|architectural response| D[ADR-0041:<br/>custom detector extension layer]
    D -->|LC re-measurement| E[Locke + custom AUC = 0.7388<br/>within 0.0006 of domain-honest 0.7394]
    E -->|prepare for Project B| F[Str-with-sentinels gap blocks<br/>NYC Taxi drift detection]
    F -->|investigate| G[Gap 1: CSV type inference from first row]
    F -->|investigate| H[Gap 2: E9070 was never invoked]
    G --> I[ADR-0042: auto_promote_str_columns]
    H --> I
    I -->|re-run LC demo| J[E9009 × 22, E9070 × 1879<br/>+ Gap 3 discovered: CSV quoted strings]
    J -->|out of scope for v0.8| K[Tracked as upstream cjc-data fix]
    style B fill:#fff4d6
    style C fill:#fff4d6
    style F fill:#fff4d6
    style J fill:#fff4d6

Each yellow node is a discovery that drove the next change. The brief at the top would have predicted A → I directly. The path through B, C, F is what actually happened.

Quick-start (Python)

pip install cjc-locke[numpy]

import cjc_locke

class PostOriginationByName(cjc_locke.CustomDetector):
    def code(self) -> str:
        return "E9500"

    def belief_axes(self) -> list[str]:
        return ["leakage"]

    def run(self, df, sink) -> None:
        for col in df.column_names():
            if col.startswith(("total_", "last_pymnt_", "recoveries")):
                sink.emit("error",
                          f"`{col}` matches a post-origination naming pattern",
                          column=col, sample_size=df.n_rows)

report = cjc_locke.validate(
    {"loan_amnt": [...], "total_pymnt": [...], "dti": [...]},
    label="my-loans",
    custom_detectors=[PostOriginationByName()],
)

print(report.severity_counts)   # {'info': ..., 'notice': ..., 'warning': ..., 'error': ...}
print(report.finding_codes())   # ['E9001', 'E9500', ...]
print(report.to_json())         # canonical, byte-deterministic

Known follow-ups for v0.1.3 / v0.9

Two cosmetic Python issues caught during post-publish verification:

1. Stale __version__ constant. python/src/lib.rs hardcodes m.add("__version__", "0.1.0")? — cjc_locke.__version__ reports "0.1.0" regardless of the actual wheel version. The __rust_crate_version__ field uses env!("CARGO_PKG_VERSION") and reports correctly. One-line fix.
2. NumPy probe panic when numpy isn’t installed. PyO3’s value.extract::<PyReadonlyArray1<f64>>() invokes numpy’s lazy init even for non-numpy inputs; the bridge needs either a hard dep on numpy or a py.import("numpy").is_ok() guard. The [numpy] extra in pyproject.toml works around this for typical use.

And two cjc-data follow-ups that this session surfaced but didn’t fix:

3. RFC-4180 quoted-string support in cjc-data::CsvReader. The third gap from §6.4 lives here.
4. A pre-existing tests/test_cjc_v0_1_hardening.rs path bug on Linux/macOS — inline mod unit { #[path = "../..."] mod foo; } resolves the relative path against the module name, not the file location. Master CI has been red on Linux/macOS Test Suite for some time as a result. Unrelated to v0.8 but worth fixing.

Total session footprint

7.1 The ? sentinel detector — E9007 + E9008

E9007 (Info) flags the occurrence of a known-sentinel string. Locke’s built-in list is {"?", "NA", "N/A", "NULL", "null", "none", "None", "nan", "NaN", "-", ""}. If any of those appears at ≥1% of a Str column’s rows, E9007 fires for that column.

E9008 is the consequence: Locke folds those positions into an effective null mask for the downstream missingness check. That’s why E9001 then fires on the same three columns — the engineer didn’t tell Locke that ? meant missing, but Locke worked it out and now the missingness count is accurate.

This is one of the most practically important detectors. Every data engineer has been bitten by a sentinel that wasn’t NaN — 999 for missing age, 1900-01-01 for missing timestamp, [redacted] for missing PII. Locke catches the string class for free. The numeric ones (E9007 numeric, -9999/99999) fire on the same pattern.

# What the finding's structured evidence looks like:
{
  "code": "E9007",
  "severity": "info",
  "column": "workclass",
  "message": "column `workclass` has 275 occurrences of \"?\" (5.5%); may be a sentinel missing value",
  "evidence": [
    {"kind": "count", "label": "n_sentinel_occurrences", "value": 275},
    {"kind": "ratio", "label": "sentinel_share", "value": 0.055},
    {"kind": "sample", "label": "sentinel_value", "value": "?"}
  ],
  "assumptions": ["the literal string `?` is one of Locke's built-in sentinel strings"],
  "suggested_next_checks": [
    "if `?` is intentional, suppress E9007 for this column",
    "if `?` means missing, the auto-inferred null mask (E9008) is already applied downstream"
  ]
}

7.2 The label-encoded-nominal detector — E9023

E9023 (Notice) is one of Locke’s quieter wins. The finding text is:

column education-num has 16 distinct Int values densely packed in [1, 16] — looks like a label-encoded nominal, not an ordinal numeric

The semantic: an Int column with distinct_count == max - min + 1 (i.e., densely packed across its full range, no gaps) and a relatively small distinct count is probably a relabelling of a categorical column, not a true ordinal. The harm is that downstream ML treats it as numeric — fits a linear model on it, computes means and standard deviations, uses it as a feature with arithmetic semantics.

The synthetic Adult clone has education-num exactly as the integer-encoded version of education — 16 levels packed into 1..16 with no gaps. So E9023 fires correctly.

Pandera has no concept of “this Int column is a label-encoded categorical.” GE doesn’t either by default. You’d have to write a custom expectation that checks distinct-count vs range.

7.3 Distribution-shape detector — E9024

capital-gain has skewness 8.05 and excess kurtosis 63.6 — the long-tail sparse-zero distribution we baked in. E9024 (Notice) fires whenever skewness > ±2 and excess kurtosis > 7 simultaneously. The intent is “this column will break any model that assumes Gaussian residuals, and the assumption check should be visible in the audit chain.”

The structured evidence carries the actual skew and kurtosis numbers, so a CI gate can pin them and detect when a downstream filter accidentally drops the long tail.

7.4 Rare-category detector — E9016

workclass has Without-pay and Never-worked at <0.1% each — fewer than 5 rows in our 5,000-row sample. E9016 (Notice) flags categories with count < 5 because they’ll dominate variance in any per-class metric and will likely vanish from a 70/30 split.

7.5 Edit-distance near-duplicate detector — E9082

education has 16 levels including 10th, 11th, 12th, which are all within edit distance ≤ 2 of one another. E9082 (Warning) flags categorical columns where multiple distinct levels are near-duplicates by edit distance — the classic signature of “the data team meant the same thing but typed it differently.” Here the levels are intentionally distinct (10th grade vs 11th grade), but the warning is still useful as an “are you sure?” signal that you can suppress per-column once verified.

income also fires E9082 — >50K and <=50K are 4 characters apart. Marking them as near-duplicates is technically correct but operationally noise; in real use you’d apply_policy to suppress E9082 on income.

7.6 Outlier detectors — E9040 + E9041

Two-tier IQR-based outlier detection. E9040 (Notice) fires when ≥1% of values are mild outliers (1.5×IQR or |modZ| ≥ 3.5). E9041 (Warning) fires when ≥0.5% are extreme outliers (3×IQR or |modZ| ≥ 5).

education-num is the interesting case here — fires both because the densely-packed 1..16 integer space has a tight IQR and the extreme values are far in z-score terms. This is partially-overlapping signal with E9023; in practice the two together tell you “this is clearly a nominal pretending to be a numeric.”

age and hours-per-week are mildly skewed and have natural outliers.

7.7 What I deliberately didn’t put in

Two findings I’d hoped to surface but didn’t, for honest reporting:

• E9063 (multi-class one-vs-rest AUC ≥ 0.95) — would catch education-num predicting education, but it needs the target encoded as an Int and the feature to be Float. Our education is Str. The same dataset with education integer-encoded would fire it.
• E9064 (per-level deterministic prediction P(class | level) ≥ 0.99) — would catch the Preschool → <=50K signature. It needs the target to be Bool or binary Int. Our income is Str. With the binary recoding (last section of run_locke.py) E9064 still doesn’t fire on this synthetic because the support per level isn’t high enough — only 73 Preschool rows in 5,000, and the conditional-threshold default is 0.99 which would need ≥73 P(>50K|Preschool) = 0 to clear support. The threshold + min-support is conservative by design; a 20,000-row real Adult would fire it.

These omissions are real, and I want them documented rather than hidden. The point of the post isn’t “Locke catches everything.” It’s “Locke catches an honest set of real things, with default config.”

8.1 The detector zoo (60+ finding codes)

validate() runs roughly six detector clusters:

compare_drift() adds:

Each finding is a structured object with typed evidence, declared assumptions, and a suggested-next-check list. The shape is in section 7.1 above.

8.2 The belief score — an 8-axis composable score

Most data-quality tools give you pass/fail or a single number. Locke gives you an 8-vector:

belief = cjc_locke.belief_report(report)
belief.score_dict()
# {'overall': 0.9685, 'schema': 0.7800, 'missingness': 0.9883,
#  'drift': 1.0000, 'leakage': 1.0000, 'lineage': 1.0000,
#  'sample': 1.0000, 'duplication': 1.0000, 'constraint': 0.9800}

On the Adult clone the schema axis is the lowest (0.78) because the 12 E9002 informational findings plus the 4 E9023/E9024/E9040/E9041 distributional notices dock the schema-confidence axis. The other axes are near-perfect because there’s no drift, leakage, lineage, sample-size, or duplication issue in this dataset.

The score isn’t a magic-number aggregate — it’s a meet-semilattice element. The composition operator compose(b1, b2, rules) takes per-axis minimums under default rules (Min per axis). The proof-checked properties under the Min rule are:

• identity: compose(b, top()) = b where top() = (1,1,1,1,1,1,1,1)
• idempotence: compose(b, b) = b
• commutativity: compose(b1, b2) = compose(b2, b1)
• associativity: compose(compose(b1, b2), b3) = compose(b1, compose(b2, b3))
• monotonicity: compose(b1, b2) ≤ b1 componentwise

These are proptest-locked in crates/cjc-locke/src/algebra.rs. The practical consequence: cascading multiple drift reports against the same train belief gives a monotonically floor-ing drift axis — the score never recovers above its current value, by algebraic construction. That’s the v0.7+ B5.4 fix surfaced as a contract.

8.3 Lineage + audit chain with content-addressed IDs

If you want to track which transforms produced which findings, Locke gives you a deterministic lineage graph:

b = cjc_locke.LineageBuilder("daily-pipeline")
src = b.add_impression("train.csv", kind="dataset", n_rows=10_000, columns=["x", "y"])
filt = b.add_idea(name="filter_active",
                  op_id="filter",
                  parents=[src],
                  params={"expr": "is_active == True"})
agg = b.add_idea(name="aggregate",
                 op_id="groupby",
                 parents=[filt],
                 params={"by": "x"})
graph = b.finish()

assert graph.is_acyclic()
graph.validate_audit_monotonic()          # raises ValueError on violation
print(graph.emit_mermaid())               # Quarto-renderable diagram

Every node’s FingerprintId is a SHA-256-derived 64-bit content-addressed hash over the node’s logical content (impression source / idea name + transform-record-fingerprint + sorted parents). Re-running the pipeline produces identical fingerprints — so you can store fingerprints in a manifest and verify them on replay.

Binary ops (join, concat, bind_cols) encode parent order in the transform’s params so A.join(B) ≠ B.join(A) at the identity level. That was the v0.7+ B5.1 fix — LockeIdea::new sorts parents (commutative-safe), so binary asymmetric ops have to carry the ordering as a param.

The audit chain (Vec<AuditEvent>) is monotonically increasing in seq within a run_label. Externally-constructed events can be validated via graph.validate_audit_monotonic().

8.4 Streaming validator — out-of-core

If your data doesn’t fit in RAM, the streaming validator runs the same detectors over chunks:

sv = cjc_locke.StreamingValidator("stream", config={"sample_cap": 100_000})
for chunk in chunked_csv_reader("huge.csv", chunk_size=50_000):
    sv.ingest_chunk(chunk)               # chunk is a dict[str, ndarray|list]
final = sv.into_report()

Running summaries (streaming_summaries()) maintain bit-exact Welford + ECDF state, lossless across chunk boundaries. into_report() reconstructs a sampled DataFrame and runs the standard validate(). There is a documented caveat (B5.6) that into_report() is byte-identical to a single-shot validate(full_df) only when no column exceeded sample_cap — past that point, the report sees the truncated sample. Running summaries are unaffected.

8.5 Policy gates

For CI/CD use, the apply_policy() interface lets you express suppression rules, owner attribution, and pass/fail requirements:

result = cjc_locke.apply_policy(report, policy={
    "suppressions": [
        # Pin a finding as acknowledged; doesn't fail the gate.
        {"code": "E9082", "column": "income", "reason": "binary target by design"},
    ],
    "owners": [
        # Routing: any remaining E9072 falls to the data-quality team.
        {"team": "data-quality", "code": "E9072"},
    ],
    "requirements": [
        # The gate condition: no E9001 must remain.
        {"code": "E9001", "operator": "eq_zero", "threshold": 0},
    ],
})

if result.gate_fails():
    raise SystemExit("policy gate failed")

gate_fails() returns True when any requirement is unsatisfied AND there’s at least one un-suppressed error-severity finding remaining. Owner attribution flows to first-match-wins (the v0.7+ B5.1 mirror test pins this); the suppression-first-match contract is also fixed.

Suppression rules can be exact-column ("column": "phone") or glob ("column": "diag_*") — ColumnMatcher::from_pattern auto-routes based on the presence of *. Empty-pattern column = "" only matches an empty-string column name (v0.7+ B6 / Polish P6 documented; use any_column for code-only suppression).

8.6 Causal guardrail

Locke ships a conservative causal layer:

cg = cjc_locke.causal_guardrail(
    data,
    target_column="income",
    causal_config={"mode": "observational_only"},
)
print(cg.n_warnings, cg.warning_kinds())

This computes pairwise Pearson correlations, flags strong ones, identifies confounder candidates (a Z correlated with both feature and target), and scans any user-supplied label text for causal-language keywords. Every report carries a top-level disclaimer; no warning kind is named Cause or Effect. In observational_only mode every correlation finding is upgraded to a warning (“treat as non-causal until intervention”).

The CausalDag lets users pre-register hypothesised causal pathways. Pairs reachable in the DAG have their warning severity downgraded (not removed) — making “I expected this; don’t bother me” explicit.

8.7 JSON, HTML, Mermaid emit

# Canonical byte-identical JSON for CI diffing
js = report.to_json()
assert js == cjc_locke.validate(df, label="adult-synth").to_json()  # always True

# Reparse back to a structured object
restored = cjc_locke.parse_report_json(js)

# Dict-shaped form for ad-hoc Python work
d = report.to_dict()
for f in d["findings"]:
    print(f["code"], f["severity"], f["message"], f["evidence"])

# Mermaid lineage diagram (Quarto-renderable, see section 8.3)
print(graph.emit_mermaid())

The JSON path uses a hand-rolled deterministic emit (zero external deps; the schema is documented at crates/cjc-locke/src/json_emit.rs). The v0.7+ deep dive hardened it: parse_int → parse_u64 for unsigned fields (catches negative-wrap attacks), surrogate-pair handling in \u escapes, a thread-local interner so repeated parses don’t leak.

There’s also an HTML emit (emit_locke_report_html, accessed from the CLI side, not yet wired through the Python wrapper) that renders an inline-SVG Pearson heatmap when ≥3 numeric columns are present.

8.8 The determinism contract

The thing that distinguishes Locke from the alternatives, repeated in this post a few times because it matters: every Locke artefact is byte-identical across runs.

• Finding IDs are content-addressed (SHA-256-derived 64-bit fingerprint over the finding’s logical content).
• JSON output is canonical (BTreeMap-sorted keys, no timestamps, no UUIDs, no map-iteration noise).
• Numeric reductions use Kahan summation (no parallel-order-dependence on floating-point sums).
• BTreeMap and BTreeSet are used everywhere; no HashMap iteration order leaks.
• The RNG is SplitMix64 with explicit seed threading.
• No SIMD path uses FMA (preserves bit-equality across CPU vendors).

The deep-dive audit (Locke v0.7+) ran four parallel agent-clusters over ~26K LOC and found exactly four silent-correctness bugs over the lifetime of the codebase — CategoricalAdaptive::cell_bytes writing the row index instead of dereferencing the dictionary; distinct_count returning row count instead of dictionary length; top_value_freq returning 0; ks_d_statistic over-counting D on tied clusters. All four fixed; all four covered by regression tests. The audit-chain canaries at 629/629 unmoved across the entire fix series. That’s the determinism cost-of-doing-business.

13.1 The philosophy is currently stronger than the implementation story

This post says “Locke detects drift.” It doesn’t show you a train distribution next to a shifted production distribution next to the confidence degradation next to the causal warning next to the lineage trace next to the resulting belief report. Every one of those pieces exists in the code; none of them is in a single narrative flow.

The next batch of work needs concrete demos: an end-to-end Jupyter notebook on a real sklearn-shipped dataset (California Housing or similar) walking the full pipeline, with the belief score visibly degrading axis-by-axis as deliberate corruptions are introduced, and the lineage graph rendering inline. The more tangible the demos, the more impressive the system looks. The roadmap is in docs/locke/POST_LAUNCH_ADOPTION_HANDOFF.md under deliverables D2.1 through D2.4.

13.2 Python ergonomics matter enormously

The architecture is right — Rust crate, Python bindings via PyO3 — but the current Python wrapper is utilitarian: repr(report) returns a one-line tag, there’s no rich _repr_html_ for Jupyter, no Arrow or Parquet zero-copy path, no sklearn.Pipeline integration, no pandas Styler highlighting flagged rows. If Python usage feels awkward people will admire the architecture without using it.

The roadmap entries D3.1 through D3.5 cover this:

• _repr_html_ on every output class (8-axis SVG for BeliefReport, severity-chip row for LockeReport, inline Mermaid for LineageGraph). Pure Python facade, no Rust changes.
• pyarrow.Table / pyarrow.RecordBatch zero-copy bridge so 10 GB Parquet files don’t have to round-trip through pandas.
• LockeCheck(BaseEstimator, TransformerMixin) so Locke fits into a scikit-learn Pipeline as the first stage (fit raises on policy gate failure; transform passes through).
• style_with_locke(df, report) returning a pandas Styler that highlights rows referenced in any finding’s row_range.

None of these require Rust changes. They’re a couple of days of Python facade work each, and they unlock notebook fluency.

13.3 Avoid sounding philosophical without grounding

Words like “epistemic layer” and “evidence-bearing” and “meet-semilattice” are accurate technical terms but they risk reading as branding — “AI philosophy layer,” academic aesthetic — unless every use is immediately anchored to a code-visible fact.

For example, “belief reports are explainable” is currently asserted but the only “explanation” shown is the 8 axes themselves. A reader can’t see why a particular axis got a particular score without manually walking the findings list. The fix is to pair every philosophical claim with:

• A measurable system capability (schema_axis: 0.78 because 12 E9002 findings and 4 E9023/E9024/E9040/E9041 notices fired against the schema-relevant detector codes).
• A benchmark (a CI job that prints the actual byte-diff of two successive runs, demonstrating zero-byte drift).
• A reproducible demo (an axis_demos/ folder with one tiny script per axis, runnable in 5 seconds).
• A failure case (what happens when this contract breaks — show two reports diverging when BTreeMap is swapped for HashMap).

That last one is rhetorically the strongest. Readers care about non-determinism only when they see it silently break a system. A “what if Locke had a bug?” demo would do more for adoption than three paragraphs about determinism.

13.4 Why this is in the post rather than elsewhere

The deep-dive audit at v0.7+ found four silent-correctness bugs that unit tests didn’t catch — integration tests caught them. The same principle applies here: the substrate is healthy under unit-test scrutiny, but the adoption layer hasn’t been integration-tested against real users yet. The three areas above are the integration findings. Naming them inline keeps the story honest.

The next post in this series will be the end-to-end notebook (D2.1 above) with the belief score visibly degrading across stages, the lineage rendering inline, and the determinism contract demonstrated on a real dataset. That’ll be the “this is what infrastructure actually looks like in use” piece, and it’s the one that needs to exist before Locke can claim “ready for adoption.”

What the change did at 20K

The 20K trial (same stratified sub-sample as the first post, seed 42, 80/20 train/test) moved as expected:

Leaf utilisation is much better too: 12/16 populated (vs 45/64 = 70% before), and the smallest populated leaf has 428 rows (vs 1 — a single-row leaf where a BLR has essentially no information). The mean rises from 355 to 1,333 — every leaf now has enough data to train a meaningful posterior.

time_in_hospital and num_inpatient are the MI-selected routing features at 20K. We’ll come back to that in §5 — they don’t stay the winners at full scale.

4.1 E9004 — the leakage hole the first post missed

code=E9004 severity=error column=patient_nbr
  message: key column `patient_nbr` has 30248 duplicate values
           across 16773 groups
  evidence:
    - duplicate_keys=30248
    - duplicate_groups=16773

30,248 duplicate patient_nbr values across 16,773 groups — i.e. 16,773 patients had multiple hospital encounters in the dataset. This is correct medical data (real patients return), but it creates a leakage path: if the train/test split is row-level (encounter-level), the same patient can appear in both splits. The model can then learn features about that patient and “look up” their outcome in the test set.

The first post used row-level splitting. So did this Round Two. A truly leakage-free protocol would split by patient_nbr, which would reduce effective training data (more conservative splits typically do) but would tighten the leakage story.

Locke surfaced this without running a single model. AUC numbers wouldn’t have shown it. Whether the encounter-level result is “real signal” or “partial patient-memorisation” can only be decided by re-running with a patient-level split — recorded as future work in §10.

4.2 E9063 — the predicted leakage finding that didn’t fire

The handoff predicted that E9063 (multi-class target leakage) would fire on discharge_disposition_id. The reasoning: codes 11, 13, 14, 19, 20, 21 in that column correspond to death and hospice outcomes. A dead patient can’t be readmitted by construction; those rows are deterministically readmitted = NO. Leakage.

E9063 did not fire. Looking at the detector’s logic (crates/cjc-locke/src/leakage.rs:229), the reason becomes clear: E9063 uses per-feature ROC-AUC (max one-vs-rest) against the multi-class target. Death codes (11, 13, 14, 19, 20, 21) sit interspersed in the numeric range with non-death codes (12, 15, 16, 17, 18). So the column-wide rank statistic doesn’t separate cleanly — most non-death codes also predict NO (89% base rate), so the AUC stays below the 0.85 warning threshold.

The leakage exists at the level — “code 11 → NO with probability ~1.0” — not at the column. ROC-AUC can’t see level-by-level conditional probability.

This is itself a publishable Locke result. A concrete detector gap, pinned to a concrete real-world example:

4.3 We applied the prune anyway — and AUC dropped

Domain knowledge still says drop the death-discharge rows. The detector missed; we know better; filter and retry.

Added filter_out_death_discharges in the harness — drops rows where discharge_disposition_id ∈ {11, 13, 14, 19, 20, 21}. That’s 2,423 of 101,766 rows (2.38%), all readmitted = NO. After the filter, the positive-class proportion rises slightly (11.16% → 11.43%).

Re-ran the 20K trial at the same config (K=2, prior=0.5, seed 42):

Removing leakage rows hurt the model. That looks paradoxical until you read the bottom of the table: the MI-selected routing columns changed. Before the prune, the top-K=2 routing features were [17, 9] = num_inpatient + time_in_hospital. After the prune, they shifted to [17, 16] = num_inpatient + number_emergency. time_in_hospital was a stronger routing feature than number_emergency at 20K — when the prune removes the correlation between time_in_hospital and the target (death patients had specific hospital-stay durations), the MI selector picks the next best feature, which happens to be less useful.

This is a confounded experiment. The §4.B intervention affects both (i) leakage row removal and (ii) downstream routing-feature selection. To isolate (i), you’d need to force the same routing columns across both configs — override the MI selector. That’s not in §4.B’s scope; this is now a documented finding for the next experiment campaign.

But there’s a more interesting insight in the routing change: [17, 16] is the same routing pair the model picks at full 101K scale (§5 below). Removing the leakage rows at 20K shifts MI in the same direction scaling up shifts it. The §4.B negative result is, in part, MI selection drift that was about to happen anyway.

5.1 Single seed at full scale

One call to the existing diabetes130_full_run test (which uses budget = usize::MAX, i.e. the entire dataset). Wall clock 213.84 s (≈ 3.6 min). Seed 42.

AUC 0.6621 — in the strong-published-model band (0.64–0.68). Without any architectural change. Without calibration. Still on the linear BLR head.

The calibration metrics (Brier 0.0953, NLL 0.3349, ECE 0.0044) are all better than the first post’s calibrated values (0.098 / 0.344 / 0.010) — and that comparison is against post-Platt numbers, while these are raw. The increased data volume at the same model class produces raw outputs that are already well-calibrated.

The first post predicted: “Calibration is a monotonic transform, so it cannot change the ranking, and it didn’t… A nonlinear MLP-leaf-head variant is the next experiment — it may lift AUC, or may need a lot of tuning.”

The implication was that AUC ≥ 0.64 required the MLP. It did not. Data scale was enough.

5.2 Routing columns flipped at full scale

The MI-selected routing features at full 101K are [17, 16] — num_inpatient + number_emergency. Not [17, 9] (which won at 20K).

The flip happens because time_in_hospital’s MI advantage over number_emergency is small at 20K and disappears with more data:

This is the architecture’s MI selector doing real work. It picked a different feature pair at a different scale, automatically, no human intervention. Whether time_in_hospital would re-emerge at, say, 50K is an interesting probe — the threshold at which the two cross over is measurable.

5.3 Three seeds — putting an error bar on the headline

A single-seed result is dangerous. §4.F ran the same full-101K pipeline at seeds 42, 43, 44. Wall clock 610.77 s for three runs (~10 min total).

The variance is tiny — 0.0018 = 0.27% of the AUC mean. The seed-42 single-seed result (0.6621) was actually below the 3-seed mean (0.6645). Without the multi-seed pass, the single-seed headline understated the model.

All three seeds picked the same routing pair ([17, 16]). Structural decisions are pinned by scale; the per-seed noise is purely the held-out-test stratified sample.

Each seed’s chain head is reproducible bit-for-bit:

seed 42: chain = 56af19614b4dfff6 df97c53949668765 acdc045e316e4fed 087a8a5d9b3233b6
seed 43: chain = 590c3fb936cec418 91576128b82e7243 dc399d6ca49a845c 87bb7392ddb45f7d
seed 44: chain = 89e02f56dadbb651 39418e0ecd3536c0 096b4851ae099694 f3736912ecde2bb2

5.4 The honest comparison vs the first post

The new numbers beat the first post on every probabilistic axis without Platt calibration. The +0.054 AUC lift is the main story; it puts ABNG in the strong-published-model band on its linear-only configuration.

The first post said:

A nonlinear MLP-leaf-head variant is the next experiment — it may lift AUC, or may need a lot of tuning.

It turns out the bigger lever was data scale, not architecture. We’ll come back to that in §9.

6.1 Locke didn’t see ? as missing

Running the existing per-leaf belief experiment (tests/abng/per_leaf_belief_diabetes130.rs) produced a per-leaf BeliefScore CSV where every leaf had missingness_score = 1.0000:

leaf_id,n_rows,overall,schema,missingness,drift,leakage,lineage,sample,duplication,constraint
1,9617,0.902500,0.340000,1.000000,1.000000,1.000000,1.000000,1.000000,1.000000,0.880000
2,6215,0.905000,0.360000,1.000000,1.000000,1.000000,1.000000,1.000000,1.000000,0.880000
3,2545,0.902500,0.360000,1.000000,1.000000,1.000000,1.000000,1.000000,1.000000,0.860000
4,1623,0.902500,0.360000,1.000000,1.000000,1.000000,1.000000,1.000000,1.000000,0.860000

Locke is saying: no missingness detected on any leaf. On a dataset where weight is 96.9% ?.

The cause is in Locke’s default ValidationConfig: it treats only f64::NAN as missing for Float columns and produces info-only diagnostics for Str columns (they get an E9002 no null mask supplied notice). Diabetes-130 stores its missing values as the literal string ? — Locke doesn’t see them without explicit configuration.

6.2 The fix — NullMaskMap plumbing

§4.D Part 1 added a harness-side helper that scans every Str column for ? and builds a NullMaskMap:

fn build_question_mark_null_masks(df: &DataFrame) -> NullMaskMap {
    let mut out = NullMaskMap::new();
    for (name, col) in df.columns.iter() {
        if let Column::Str(values) = col {
            let null_rows: Vec<usize> = values.iter().enumerate()
                .filter_map(|(i, v)| if v == "?" { Some(i) } else { None })
                .collect();
            if !null_rows.is_empty() {
                out.insert(name.clone(), NullMask::from_indices(null_rows));
            }
        }
    }
    out
}

Then thread it through ValidateOptions.null_masks. Re-ran the per-leaf belief:

The belief vector is now informative. Seven columns are correctly flagged as ?-bearing, including weight at 96.9% — matching the first post’s “weight is ~97% missing” observation.

6.3 The deeper problem — leaves clustered at uniform belief

With ?-handling fixed, the per-leaf belief vector is informative per axis, but all four leaves cluster at near-identical missingness_score:

Spread: 0.001. All four leaves are within one-tenth of a percent. That’s because the routing column (time_in_hospital, in this simplified 1-D codebook) doesn’t correlate with data-quality variation. The 97% ? rate in weight is roughly uniform across hospital-stay durations.

For §4.D Part 2 (the actual belief-weighted prior experiment) to do anything, the leaves need to differ on belief. They don’t, on this routing. The directional hypothesis from ABNG_PER_LEAF_BELIEF.md is operationally a no-op when all leaves get the same belief multiplier.

6.4 Part 2 — deferred for a real reason

§4.D Part 2 (apply the belief-weighted prior) requires changes to ABNG’s set_blr_prior:

// Current API — graph-wide, one-shot, must be called before any add_node.
pub fn set_blr_prior(
    &mut self,
    precision: f64,
    a: f64,
    b: f64,
) -> Result<(), GraphError> { ... }

There’s no set_blr_prior_for_node(node_id, precision, a, b). Adding one would touch:

1. Public API surface (new method on AdaptiveBeliefGraph).
2. Lifecycle relaxation (must work after add_node, not before).
3. A new AuditKind::BlrLeafPriorOverride variant for the audit chain.
4. New SHA-256 canary locks for that event type.

Multi-session work, at the same scope as §4.C MLP heads. Deferred — see §10.

The §4.D Part 1 work is still a real Locke contribution: it makes the per-leaf belief informative on ?-sentinel datasets. The pre-blocker is resolved. The actual experiment (Part 2) is waiting on ABNG architecture work plus the routing-vs-belief alignment question being settled.

8.1 The φ-vs-routing feature split

ABNG’s categorical transform produces two outputs per row: x (low-dim integer buckets for routing) and phi (high-dim one-hot for the BLR head). The schema’s CategoricalPhiOnly role marks columns as “contribute to φ but never to routing”.

On diabetes-130, that’s what makes the 23 medication columns and 3 ICD-9 codes (700+ levels each) usable. They participate in prediction (φ width stays ≤ 266 across all configs) without exploding the routing tree (which would otherwise need 4^700 leaves). Trees and forests don’t cleanly separate “what splits” from “what predicts” — every feature is a split candidate or excluded. ABNG’s split is at the schema layer, before routing.

8.2 Pre-fixed MI-selected routing

Greedy tree learners pick splits locally — each split optimises against the current data subset. ABNG picks routes globally, once, before any training: score every column’s mutual information against the target, take the top K, build a fixed codebook. The structure is then frozen; training only fills in the per-leaf heads.

This session showed this lever doing real work: at 20K it picked [num_inpatient, time_in_hospital], at 101K it picked [num_inpatient, number_emergency] — automatically, no human tuning. And at 101K all three §4.F seeds picked the same pair: structural decisions are pinned by scale, with seed-to-seed noise concentrated entirely in the per-leaf parameters.

8.3 Per-leaf Bayesian linear regression with conjugate updates

Each leaf is a full Bayesian regression: precision matrix, posterior mean, predictive variance — all updated in closed form per training row. No SGD. No learning rate. No early stopping. O(d²) per step, where d ≈ 266 here.

The root + leaf ensemble (predicted prob = 0.5 × leaf_posterior_mean + 0.5 × root_posterior_mean) is the model that produces 0.6645. Bayesian forests and BART share parts of this idea; the combination of conjugate per-leaf BLRs with frozen pre-MI routing is, as far as I know, ABNG-specific.

8.4 SHA-256 audit chain over training events

Every train_step appends an event to a hash chain. End-state chain head is byte-reproducible. Merkle root verifies. This is the property no commodity ML system carries.

In this session: five commits, each carrying a chain head. Aggressive config changes throughout — constants flipped, new tests added, a new filter applied, multi-seed runs. The 28 SHA-256 determinism canaries didn’t move once, because nothing they cover was touched. The work was safe to be exploratory in a way most ML harnesses don’t permit.

8.5 Bit-identical replay

The first post’s reproducibility evidence (“Run 1 / Run 2 — every metric digit identical, 3× wall-clock difference”) held throughout Round Two. The §4.F std of 0.0018 is direct evidence: variance comes entirely from seed-driven sub-sample / split / training-order permutations, not from numerical noise. Kahan summation, BTreeMap iteration (not HashMap), no FMA in SIMD kernels, deterministic SplitMix64 RNG — the determinism contract is a system-wide invariant.

The five chain heads from this session — 8897c2dd… (before), 57463728… (§4.A after), 78a63f56… (§4.B pruned), 56af1961… (§4.E full), and the three §4.F seeds — are durable artifacts. Anyone with the codebase, the seed, and the dataset can rerun and confirm bit-identity.

Net assessment

Locke did help — but on the systems / determinism / detector-roadmap axis, not directly on the AUC-bump axis. The brief implicitly assumed Locke would surface a leakage pattern that, when filtered, would lift AUC. What actually happened: Locke surfaced detector limits and structural issues that will drive future Phase 0.11 work but didn’t move this session’s number. The §4.B negative result is the most valuable Locke output of the session — it pins a concrete failure mode (E9063 misses per-level outcomes) and points at a concrete fix (E9064 conditional-probability detector).

ABNG, by contrast, was the lever. The K=2 / prior=0.5 config + full data + 3-seed variance is what produces AUC 0.6645 ± 0.0018. ABNG’s audit chain, BLR conjugate update, deterministic stratified sub-sample, and MI-based routing selection all carried this session. The cost in code: two constants and three new test functions.

Figures

Every chart in this post was rendered from the actual session data via generate_figures.py (Plotly + kaleido). Two themes are saved alongside:

• Traditional (used in the body above): figures/traditional/<name>.png
• Cyberpunk: figures/cyberpunk/<name>.png

Interactive HTML versions (hover for exact values, zoom, pan) live under figures/interactive/<theme>/<name>.html.

The Locke Pearson |r| heatmap (figures/locke/pearson_heatmap_actual.svg) is extracted verbatim from the actual cjcl locke validate HTML report via extract_locke_heatmap.py — no recreation, no theme override. It’s the literal Locke artifact, content-addressed and deterministic.

Rust API

use cjc_locke::{validate, ValidateOptions, ValidationConfig};
use cjc_data::DataFrame;

let opts = ValidateOptions {
    dataset_label: "train.csv".into(),
    config: ValidationConfig::default(),
    ..Default::default()
};
let report = validate(&df, &opts);
println!("worst severity: {}", report.worst_severity());
for f in &report.findings {
    println!("{} {}: {}", f.code, f.severity, f.message);
}

CLI

cjcl locke validate data.csv \
  --target churned \
  --primary-key customer_id \
  --time-col event_ts --max-timestamp 1714521600000 \
  --html report.html \
  --save-json baseline.json \
  --fail-on warning

That single command runs the full v0.5 validation pass (missingness + duplicates + impossible values + constants + outliers + sentinels + schema + temporal + target-leakage + ID-like + duplicate-key conditioning), writes a stakeholder-ready HTML report with an inline-SVG correlation heatmap, persists the canonical JSON baseline, and exits non-zero if any finding crosses the warning threshold.

CJC-Lang source

fn main() {
  let h = locke_table_new();
  locke_table_add_float_col(h, "age", [25.0, 30.0, 0.0/0.0, 30.0]);
  locke_table_add_int_col(h, "churned", [0, 1, 1, 0]);
  let r = locke_validate(h);
  print(locke_report_worst_severity(r));   // 2 = Warning (NaN detected)
  print(locke_report_overall_score(r));    // 0.857
}

The CJC-Lang surface lets .cjcl programs validate data without dropping into Rust. The implementation uses a thread-local registry plus 22 locke_* builtins routed through both executors via satellite dispatch — the same pattern cjc-ad’s GradGraph and cjc-abng’s belief radix graph use.

Scenario 1 — The Monday-morning gate diff

A data scientist arrives Monday. The Q3 production data dump is ready. They want to know: did anything important change versus the Q2 baseline they trained against?

Two commands:

# Friday (Q2): after training, snapshot the Q2 baseline
cjcl locke validate train_2026q2.csv \
  --target churned \
  --primary-key customer_id \
  --time-col event_ts \
  --max-timestamp 1719792000000 \
  --save-json baselines/2026q2_baseline.json

# Monday (Q3): diff the new dump against last quarter's snapshot
cjcl locke gate baselines/2026q2_baseline.json \
  data/prod_2026q3.csv \
  --fail-on warning

The gate emits a structural diff:

# Locke Gate Diff
reference_run_id: 1075bbad9d9b882f
current_run_id:   5fbf919f0ada9b58
summary: appeared=3 disappeared=0 unchanged=17

appeared:
  + code=E9035 severity=warning column=last_login_days_ago
    missingness rate of `last_login_days_ago` shifted: train=0.020, test=0.180
  + code=E9039 severity=error column=monthly_charges
    numeric distribution shift in `monthly_charges` (KS D = 0.342)
  + code=E9051 severity=error column=event_ts
    127 test rows have `event_ts` <= max train timestamp; test set overlaps train window

disappeared: (none)

unchanged:
  · code=E9002 column=customer_id (Int, no null mask)
  · code=E9003 (23 duplicate rows)
  · code=E9072 column=customer_id (ID-like cardinality hint)
  · ... (14 more)

Exit code 2. CI fails the job. A human reads the diff:

1. E9051 is the smoking gun — 127 rows of the test dump have timestamps inside the training window. Someone changed the ingestion cutoff and the new feed doesn’t respect the train/test boundary anymore.
2. E9039 is the symptom — once the test set silently overlaps the train window, the marginal distribution of monthly_charges shifts because the overlap rows are systematically different.
3. E9035 is the third-order effect — the missingness rate of a derived column jumps because the overlap rows have null last_login_days_ago.

The fix is upstream: the ETL job needed WHERE event_ts > '2026-07-01' and the analyst who refactored it last week silently dropped that clause. The CI gate caught it before model training started.

What makes this work at all: reference_run_id and current_run_id are content-addressed. The same data on a colleague’s Mac produces 5fbf919f0ada9b58 byte-for-byte. There’s no “this report has timestamp 2026-05-28T09:14:23Z” or “validation_result_uuid f8a3…” line to make two runs disagree. The 17 unchanged findings are recognised as the same findings because their fingerprints match, not because their messages happened to spell the same words.

Scenario 2 — The target-leakage smoking gun

A new feature lands in the churn pipeline: days_since_last_login. The engineer who added it computed it as now() - last_login_timestamp at feature-engineering time. But “now()” was the scoring moment, not the event moment — every row that already churned has last_login_timestamp frozen at the time the customer left, so days_since_last_login is a near-perfect predictor of churned by construction.

A model trained on this dataset will report 99% ROC AUC offline and degrade to 60% in production within a week.

Locke catches it. Run:

cjcl locke validate feature_engineered.csv --target churned

The relevant finding:

{
  "id": "8b3c1f5d2a4e9c7b",
  "code": "E9060",
  "severity": "error",
  "message": "feature `days_since_last_login` has AUC = 0.971 against target `churned`; likely target leakage",
  "column": "days_since_last_login",
  "evidence": [
    {"kind": "metric", "label": "auc", "value": 0.971},
    {"kind": "metric", "label": "threshold", "value": 0.95},
    {"kind": "count",  "label": "n_pos",     "value": 1574},
    {"kind": "count",  "label": "n_neg",     "value": 5449}
  ],
  "assumptions": [
    "rank-sum AUC computed with f64::total_cmp + tie averaging",
    "leakage is suspected when |AUC - 0.5| >= 0.45 (Error) or 0.35 (Warning)",
    "this check only fires for binary targets; multi-class deferred to v0.2"
  ],
  "suggested_next_checks": [
    "verify this feature is computed at the event time, not score time",
    "check the upstream feature-engineering job for `now()` calls on historical rows",
    "rebuild the feature using the event timestamp and re-validate"
  ]
}

After the fix — feature recomputed using the event timestamp, not now() — the same column drops to AUC = 0.61, well below the 0.85 Warning threshold. E9060 disappears from the next gate run. The diff says disappeared: E9060 column=days_since_last_login. The fix is auditable.

The point isn’t that “Locke catches leakage” in the abstract. It’s that the finding carries the AUC value, the count of positives and negatives, the threshold that fired, the assumption that AUC is rank-sum with tie averaging, and the suggested next check — all as structured fields. A reviewer can decide whether to trust the finding without re-reading Locke’s source.

Scenario 3 — The lineage diagram

Locke’s TracedDataFrame wraps cjc-data’s DataFrame ops and records a DAG: each operation is a LockeIdea node, each input dataset is a LockeImpression, and edges carry content-addressed IDs. When a downstream metric goes wrong, you walk the DAG backwards to find which derivation contaminated it.

Consider a four-step pipeline: load raw events, filter to active customers, group by plan tier, and compute churn rate per group:

flowchart LR
    I["events.csv<br/><i>LockeImpression</i><br/>id=a4f8c2d19e0b3517"]:::imp -->|"observe()"| O["raw_events<br/><i>LockeIdea</i><br/>id=2b7e4a1f8c3d9602"]:::idea
    O -->|"filter(active==1)<br/>id=c91f5b2e0a4d7836"| F["active_events<br/><i>LockeIdea</i><br/>id=f3a2c780b91d4e5c"]:::idea
    F -->|"group_by(plan_tier)<br/>id=8e6b3f04c12a9d7e"| G["grouped<br/><i>LockeIdea</i>"]:::idea
    G -->|"summarise(churn_rate=mean(churned))<br/>id=1075bbad9d9b882f"| S["churn_by_tier<br/><i>LockeIdea</i><br/>id=5fbf919f0ada9b58"]:::idea
    S -->|"with_column(label='Q3-2026')<br/>id=db4cac307664c097"| W["labeled<br/><i>LockeIdea</i>"]:::idea

    classDef imp fill:#e8f4fd,stroke:#2196F3,stroke-width:2px
    classDef idea fill:#fff3e0,stroke:#FF9800,stroke-width:1px

The DAG is acyclic by construction (impressions and ideas are distinct node kinds; edges only go impression → idea or idea → idea). The DAG is also deterministic: rerunning the same pipeline on the same bytes produces the same node IDs. That means a downstream consumer can cite “the churn rate with idea-id 5fbf919f0ada9b58” and a Locke replay on the same input will reproduce it byte-for-byte.

When the Monday gate diff above says current_run_id: 5fbf919f0ada9b58, that ID is exactly this node’s content-addressed fingerprint. The gate command and the lineage DAG share the same hashing primitive.

In code, this looks like:

use cjc_locke::lineage::{LineageGraph, emit_lineage_text};
use cjc_locke::traced::TracedDataFrame;

let raw = TracedDataFrame::observe(&events_df, "events.csv");
let active = raw.filter(|row| row.get_int("active") == Some(1));
let grouped = active.group_by(&["plan_tier"]);
let churn_by_tier = grouped.summarise("churn_rate", |g| g.mean_f64("churned"));
let labeled = churn_by_tier.with_column("label", "Q3-2026");

let graph: LineageGraph = labeled.lineage();
println!("{}", emit_lineage_text(&graph));

The text emit is what populates the IDs above. The Mermaid diagram is one possible rendering; the canonical form is the JSON graph that Locke persists, which the HTML report visualises with the same structure.

Scenario 4 — Belief score side-by-side

Two datasets arrive on the same day: dataset_a.csv (a long-running production export) and dataset_b.csv (a new vendor’s first delivery). Same column shape. The team wants a single number per dataset to decide which to use for the quarterly run.

Locke produces an 8-dimension BeliefScore, not a single number. Each dimension is on [0, 1] where 1.0 means “no evidence of a problem on this axis”:

dataset_a.csv
  overall_score:        0.91
  schema_score:         1.00   no schema mismatches
  missingness_score:    0.99   1 column with <1% NaN, no joint missingness
  drift_score:          0.97   one numeric column with KS D = 0.06 vs ref (notice)
  leakage_score:        1.00   no feature with |AUC| >= 0.85 against target
  lineage_score:        0.95   one Idea node missing parent reference (notice)
  sample_score:         0.92   sample_cap=10000 hit; flagged as approximate
  duplication_score:    1.00   no duplicate rows; no duplicate primary keys
  constraint_score:     1.00   no impossible values; no constants

dataset_b.csv
  overall_score:        0.42
  schema_score:         0.80   `total_charges` declared Float, inferred Str
  missingness_score:    0.42   1.4% of total_charges is empty string; conditional miss with last_login
  drift_score:          0.71   monthly_charges KS D = 0.34 vs dataset_a (error)
  leakage_score:        1.00   no feature with |AUC| >= 0.85 against target
  lineage_score:        0.50   vendor delivery has no lineage attached
  sample_score:         1.00   full dataset evaluated
  duplication_score:    0.65   23 duplicate customer_id keys (E9004 error)
  constraint_score:     0.62   5 rows with monthly_charges = -9999 (impossible)

The team picks dataset A for the quarterly run and files a ticket against the vendor for dataset B. The breakdown tells them which axes are degraded, not just that the dataset is worse — vendor B’s drift_score collapsed because the marginal distribution of monthly_charges shifted by 0.34 in KS distance, and the constraint_score collapsed because of the sentinel -9999 rows.

Crucially: each dimension is computed from the same evidence-bearing findings the gate diff uses. The drift_score = 0.71 for dataset B is 1 - penalty(E9039) where penalty comes from a swappable BeliefPenalty model. The model is declared, not implicit; a team that disagrees with the default can supply their own and the audit still reproduces.

A single magic number from a black-box scorer would let you compare datasets but not act on the comparison. The 8-dim breakdown plus the findings table is what makes the decision actionable.

A — Dashboard says “churn risk up 22%”

Ordinary view (Looker tile):

KPI:  customer_churn_risk_score
Δ wk: +22.4%
trend: ▁▂▂▃▃▅▆▇
owner: notified — review next standup

Locke gate diff for the same data:

# Locke Gate Diff
reference_run_id: 1075bbad9d9b882f
current_run_id:   ce81fa3d2b5a974e
summary: appeared=4 disappeared=0 unchanged=15

appeared:
  + code=E9039 severity=error column=last_login_days_ago
    numeric distribution shift (KS D = 0.51)
  + code=E9061 severity=warning column=last_login_days_ago
    feature |AUC|=0.87 against `churned` — proxy-leakage suspect
  + code=E9051 severity=error column=event_ts
    312 production rows have event_ts inside the training window
  + code=E9080 severity=warning
    causal assumption `churn_intent → unsubscribe` unsupported in this sample

The dashboard reports the model output. Locke explains why the output moved: an upstream ETL change added a 7-day-lagged event source, the last_login_days_ago distribution shifted because recent activity is now censored, the feature became a proxy for the lag itself (not for churn intent), and 312 production rows leaked back into the training window. The 22% risk uptick is a measurement artifact. A human reading the Locke diff stops the alert chain before the executive briefing; a human reading only the dashboard escalates.

B — dbt test exits 0, model memorises customer_id

Ordinary view (CI log):

$ dbt test --select model:churn_features
Found 14 tests, 0 sources, 1 model
14 of 14 PASS in 8.41s
Completed successfully — 0 errors

Locke output on the same table:

[E9072 notice] column `customer_id`      cardinality 0.997  — id-like leak suspect
                id=328845d456c53aea
[E9072 notice] column `last_login_unix`  cardinality 0.997  — id-like leak suspect
                id=03ae43c0bccfbc79
[E9072 notice] column `total_charges`    cardinality 0.977  — id-like leak suspect
                id=86d3973bf5062cf8

dbt’s unique test is structurally the right question for transformation correctness — “does each row have a distinct key.” It is structurally the wrong question for whether to keep that column as a feature. A model trained on the dbt-validated table will hit 98% offline ROC AUC and degrade to 55% in production within a week, because gradient boosting will happily memorise customer IDs the moment they’re left in. Locke fires E9072 on every column whose distinct-per-row ratio exceeds 0.95 and asks the human to decide.

C — Pandera schema passes a stale snapshot

Ordinary view (Pandera report):

SchemaModel: CustomersLatest
Validated successfully.
14 columns, 124083 rows.
Run at 2026-10-04T03:00:00.184Z (build #4471)

Locke output on the same file:

[E9050 warning] column `event_ts` not sorted: 1247 non-decreasing pairs
                id=a96abd21b26275c6
[E9054 warning] dataset `customers_latest.parquet` is 9.2 days stale
                max(event_ts) = 2026-09-25T00:00:00Z
                expected_at   = 2026-10-04T00:00:00Z
                id=e9a4f12c80b5d736

Pandera answers “does each row match the type-and-range contract.” It cannot answer “is this snapshot the snapshot I expected” because the contract is a per-row predicate, not a dataset-scoped invariant. Locke’s --max-timestamp carries a declared dataset-level invariant and fires E9054 when it’s violated. The downstream model trained on the Pandera-validated snapshot will be trained on data 9 days older than the team thinks, and the production scorer will degrade silently.

D — Monte Carlo green, train/test boundary leaks

Ordinary view (MC monitor):

Table: prod.feature_store.churn_features
Row count anomaly:   normal (z = 0.31)
Schema anomaly:      none
Freshness:           on-time
Distribution drift:  none detected
Status: ✅ healthy

Locke output:

[E9051 error] 127 rows in `feature_store_test` have event_ts
              <= max(event_ts) of `feature_store_train`
              overlap_rate = 0.0021
              id=8f6c5b3d1e7a4920

Monte Carlo’s metadata-anomaly detector trips on shapes that look statistically unusual compared to history. A 0.21% subpopulation overlap between train and test rows doesn’t shift the warehouse-layer marginal distribution by anywhere near MC’s sensitivity threshold — it can’t, because it’s a small slice. Locke trips because the train/test temporal boundary is a declared invariant, not a statistical anomaly. Anomaly detection trips on unexpected statistical shapes; epistemic checks trip on declared known-class violations. Both layers are useful; only the epistemic layer catches this specific failure.

Common thread

Every ordinary-tool report in the four examples is correct under its own contract. The Looker tile correctly computed the churn-risk delta. dbt correctly verified that customer_id is unique. Pandera correctly checked the type-and-range schema. Monte Carlo correctly observed no row-count or freshness anomaly. None of them is wrong; each is silent on the failure mode Locke caught, because the failure mode is outside the layer the tool is designed for.

The epistemic layer is the layer that asks: given the evidence in this dataset and the assumptions Locke is allowed to state, is there a structured finding that should make a human stop and think before the next step runs?

vs Great Expectations

Great Expectations is the closest direct comparable. It’s a suite-validation framework: write a list of expectations, run them against a batch, get a ValidationResult. The library is mature (10K+ stars, 500+ contributors, ~200 expectations, native SQL backends, Data Docs, profiler-driven auto-suite generation).

What Locke does differently:

• Content-addressed identifiers. GX validation results have validation_result_id UUIDs and meta blocks with timestamps. Two runs over the same data produce different result IDs. Locke’s run_id and per-finding IDs are content-fingerprints; identical bytes in produce identical IDs out.
• Findings as structured objects. GX returns success/failure plus per-expectation result.partial_unexpected_list. Locke returns code, severity, column, evidence: Vec<FindingEvidence>, assumptions: Vec<String>, suggested_next_checks: Vec<String> as first-class fields.
• Churn-specific defaults out of the box. E9004 (duplicate primary keys), E9060/E9061 (target leakage), E9072 (ID-like cardinality), E9051/E9052 (train/test temporal overlap and future-leakage cutoff), E9070 (conditional missingness), E9073 (duplicate-key conditioning). Equivalents in GX require writing custom expectations.

What GX does better:

• Broader expectation library (~200 vs Locke’s ~25).
• Native SQL backends (Snowflake, BigQuery, Postgres) without exporting to CSV first.
• Data Docs as a polished HTML output with a longer history.
• Suite versioning, alerting hooks, profiler-driven auto-suite generation.

The benchmark below quantifies the head-to-head on a single workflow: 5/5 ground-truth match, 275× faster end-to-end on a 7023-row churn dataset.

vs dbt tests

dbt is a transformation-layer tool. Tests in dbt (unique, not_null, relationships, accepted_values, and custom SQL tests) run inside the warehouse against the output of a dbt model. They answer “did the SQL produce the shape this model expects.”

What Locke does that dbt tests don’t:

• dbt validates the transformation. Locke validates whether the data tells a coherent story afterwards. A dbt unique(customer_id) test passes if no rows have the same customer_id — but a column like total_charges with 6858 distinct strings out of 7023 rows passes every dbt test ever written, and Locke fires E9072 (ID-like cardinality) on it because high cardinality on a non-key column is a leakage signal.
• dbt has no equivalent of target leakage AUC. You can write a custom dbt test that computes one column’s correlation with another, but you’d be writing a Locke checker badly in SQL.
• dbt’s relationships test verifies FK integrity. Locke’s E9070 conditional-missingness check verifies that missing(A) ⇒ missing(B) doesn’t accidentally hold across columns — a different, often more useful, integrity claim.
• dbt tests pass or fail. Locke findings carry severity (Notice / Info / Warning / Error) and evidence, so the team can decide how to act, not just whether the build passes.

Where dbt is the right tool:

• The validation belongs inside the warehouse (no export, no CLI hop).
• The shape constraints are exactly what dbt’s test library covers (uniqueness, not-null, FK, value set, dbt-utils accepted ranges).
• The dbt DAG is the source of truth for transformation correctness anyway.

The honest framing: dbt tests and Locke are complementary. dbt tests catch “the transformation broke.” Locke catches “the transformation succeeded and the resulting data has a coherence problem.” A pipeline can run dbt tests at the warehouse, export to CSV / Parquet for the ML side, then run Locke on the export. Both layers add signal.

vs Pandera

Pandera is the pandas-native schema-validation library. It’s the most direct comparable for “validate a DataFrame in-process.”

import pandera as pa
schema = pa.DataFrameSchema({
    "customer_id": pa.Column(int, pa.Check.ge(0)),
    "monthly_charges": pa.Column(float, pa.Check.in_range(0, 500)),
    "churned": pa.Column(bool),
})
schema.validate(df)

What Locke does differently:

• Pandera reports include timestamps and walk dictionaries in iteration order. Locke reports are content-addressed. Pandera’s SchemaErrors.failure_cases is reproducible across runs as a DataFrame; the surrounding report metadata is not. Locke’s entire report serialises to canonical JSON with deterministic key ordering and content-addressed IDs.
• Pandera is a Python library. Locke is a Rust crate + CLI + CJC-Lang surface. A team that doesn’t run Python in production can still gate CI on Locke; with Pandera the validator needs a Python runtime.
• Pandera’s checks are caller-declared. Locke ships a default battery. Pandera makes you spell out every constraint; Locke fires a churn-specific battery without configuration and lets you tune severity downstream. The trade is real: Pandera gives finer control; Locke gives a faster start.
• Pandera doesn’t have a lineage primitive. The TracedDataFrame wrapper has no analogue.

Where Pandera is the right tool:

• The pipeline is already pandas-native and the team wants in-process type-and-range validation as a guard.
• The schema is well-understood and the team prefers explicit declaration over a default battery.
• Pandera’s SchemaModel class-based syntax integrates cleanly with Pydantic-style type-checked codebases.

The honest framing: Pandera is excellent for in-Python pipeline guards. Locke is for the after-Pandera step — produce a CSV / Parquet / JSONL, then ask “is this artifact reproducible to audit?” Pandera passes the data through a contract; Locke produces the audit artifact.

vs Monte Carlo

Monte Carlo is the largest of the data-observability SaaS platforms. It treats data quality as an anomaly-detection problem: connect MC to your warehouse, MC instruments queries, learns metadata patterns, and pages on-call when something looks weird.

This is a fundamentally different category of tool. Locke and Monte Carlo barely overlap in what they ask:

• Monte Carlo is for discovery. “We didn’t know to check this query’s row count. The ML model on our metadata thinks today’s row count is anomalous given the last 30 days. Page someone.”
• Locke is for known-class checks plus reproducible audit. “We know to check that no test row has an event timestamp inside the train window. Here is the deterministic finding with content-addressed ID. CI runs the gate every PR.”

A concrete failure mode each tool catches that the other doesn’t:

• Monte Carlo catches: a Looker dashboard’s source table silently lost a JOIN clause last week and row counts dropped 8%. Locke has no instrumentation at the warehouse layer to see this.
• Locke catches: the production Q3 dump has 127 rows with event_ts inside the Q2 train window. Monte Carlo’s metadata-anomaly model wouldn’t flag 127/120000 rows; the marginal distribution at the warehouse layer barely shifted.

The two tools sit at different layers of the stack and can both run in the same org without conflict. Monte Carlo answers “is anything weird I didn’t think to check”; Locke answers “for the things I know to check, here is the auditable evidence and the byte-identical fingerprint.”

Where Monte Carlo is the right tool:

• The data lives in a managed warehouse (Snowflake, BigQuery, Databricks) and MC’s connectors fit.
• The team wants Slack / PagerDuty integration, freshness monitoring at scale, and a managed UI for incident triage.
• The category is genuinely “discover unknown issues” — auto-baselined metadata anomalies rather than declared checks.

The honest framing: Monte Carlo and Locke solve different problems. A team running Monte Carlo for warehouse-layer observability and Locke for CSV/Parquet ML-pipeline gating has no overlap to resolve. Both add signal; neither replaces the other.

Methodology

I wrote a deterministic synthetic-churn generator (bench/locke_vs_gx/generate_churn.py) that produces a 7023-row, 8-column CSV with exactly-known seeded properties:

Then I ran two validation suites — Locke v0.5 with default settings plus --target churned --primary-key customer_id --time-col last_login_unix, and Great Expectations 1.17 with equivalent expectations (13 expectations covering missingness, value ranges, sortedness, compound-key uniqueness, value-set membership).

Both tools’ findings were dumped to JSON. A comparison script walked both result sets and asked: did each tool match ground truth?

The benchmark is intentionally scoped to a single workflow — “validate a CSV-shaped churn dataset, report counts of seeded data issues.” It’s not a global GX-vs-Locke claim; it’s a calibration of one workflow Locke is designed for.

A real finding, byte-for-byte

To make the determinism claim concrete, here is a literal ValidationFinding from the benchmark’s locke_results.json — the impossible-monthly_charges outlier:

{
  "id": "f615af023438bcb5",
  "code": "E9041",
  "severity": "warning",
  "message": "column `monthly_charges` has 5 extreme outliers (3×IQR or |mod-Z| ≥ 5.0)",
  "column": "monthly_charges",
  "row_range": null,
  "sample_size": 7023,
  "evidence": [
    {"kind": "count",  "label": "n_extreme",  "value": 5},
    {"kind": "sample", "label": "sample",     "value": "row150=-9999,row151=-9999,row152=-9999"},
    {"kind": "range",  "label": "iqr_q1_q3",  "min": 45.974999999999994, "max": 95.27}
  ],
  "assumptions": [
    "extreme outliers usually indicate sensor errors, sentinel values, or unit-mix-ups"
  ],
  "suggested_next_checks": [
    "verify physical plausibility (e.g. age = 999, temperature = -9999)",
    "consider declaring sentinel values as nulls via NullMask"
  ]
}

The id field is f615af023438bcb5 — a 64-bit content fingerprint over the rest of the structure. Rerunning the same benchmark on a different machine produces the same 16 hex characters. That’s the property that makes the gate diff in Scenario 1 work: “unchanged” findings are recognised by ID, not by message text.

Compare to the equivalent Great Expectations result for the same five rows (verbatim from gx_results.json):

{
  "name": "monthly_charges_in_range[-100,500]",
  "severity": "Error",
  "success": false,
  "details": {
    "element_count": 7023,
    "unexpected_count": 5,
    "unexpected_percent": 0.07118041435995925,
    "partial_unexpected_list": [-9999, -9999, -9999, -9999, -9999],
    "partial_unexpected_counts": [{"value": -9999, "count": 5}],
    "partial_unexpected_index_list": [150, 151, 152, 153, 154]
  }
}

Both tools catch the same five rows. GX surfaces more detail per finding (sample list, index list, exact percent). Locke surfaces typed evidence kinds (count, sample, range), declared assumptions, suggested next checks — and the stable ID. They’re answering different questions about the same fact.

Detection results

Both tools matched ground truth 5 / 5 on the quantitative facts. The detection accuracy is identical for the kinds of issues a real churn-data pipeline ships with.

A few honest notes on the table:

• GX reports duplicate rows as 46 = 23 × 2 because ExpectCompoundColumnsToBeUnique reports all rows in any non-unique group. Locke reports the extras (23). Different question, both correct.
• Severity disagreements are policy, not detection. GX defaults every failed expectation to Error. Locke uses a configurable severity ladder where 100 missing out of 7023 (1.4%) lands at Info — the floor of a four-rung ladder where Warning means ≥10% and Error means ≥50%. Locke’s ladder is tunable via BeliefPenalty; GX’s is per-expectation config.
• Locke caught the missing-total_charges count via E9007 (sentinel-value detection) because the CSV reader inferred the column as Str (empty cells aren’t parseable as numbers). Same count, different code, lighter severity. The findings are equivalent; the tool routed through a different channel.

Timing

That’s a 275× speedup in release mode against the same dataset finding the same things. Most of GX’s time is Python startup + GX context init + ephemeral data-source setup, which dominates for short batch runs. Locke is a single Rust binary with no startup overhead.

For a CI gate that runs every PR, that difference is “imperceptible vs visibly slow.” For a nightly batch over a million datasets, it’s the difference between “finished by morning” and “still running.”

Built-in checks Locke runs that GX doesn’t out of the box

On the actual benchmark dataset:

• E9004 — duplicate primary-key detection fired with the exact 23 violations on customer_id. GX has no equivalent unless you write a custom expectation.
• E9007 — sentinel-value detection caught the 100 empty strings in total_charges. GX requires expect_column_values_to_not_be_null (which works for null but not the empty-string sentinel form here).
• E9072 — ID-like cardinality hint fired three times — on customer_id, last_login_unix, and total_charges (which had 6858 distinct strings out of 7023 rows, looking like an identifier leaking as a feature). GX has no equivalent default check.

These aren’t fancy — they’re the kind of checks a churn pipeline silently fails on for months until someone notices the model is memorising customer IDs. Locke fires them by default.

Built-in checks GX runs that Locke doesn’t out of the box

To be honest about the other direction:

• ExpectColumnDistinctValuesToBeInSet for plan / country — Locke has E9020–E9022 schema mismatch but no value-set-membership check yet
• ExpectColumnValuesToBeIncreasing(strictly=True) — Locke only checks non-decreasing
• User-declared mean / quantile range expectations (Locke surfaces the values but doesn’t accept caller-declared ranges in the default validate path)
• ~180 other expectations covering regex, length, JSON-schema, custom SQL queries, etc.

GX has a much broader expectation library overall. For the specific subset that matters for churn validation, Locke is at parity or ahead. For the general case, GX still wins on breadth.

Single-shot validate()

Per-row cost grows mildly with log n because of full-row byte-hashing (duplicate detection) and sort-based outlier detection.

Drift compare()

Flat per-row cost at scale because the sort dominates and Rust’s sort is well-optimised. 1M + 1M rows in 5.7 seconds.

Streaming validator (chunked ingest)

About 310,000 rows/second sustained streaming throughput.

Streaming KS D vs single-shot KS D

Streaming is ~2× faster because the BTreeMap-ECDF is already sorted at ingest time, so the merge-walk at query time skips the sort. The result is bit-identical to single-shot KS — proved by a unit test.

Practical scale matrix for a churn use case

For a typical SaaS churn dataset (1k–10M customers), Locke runs comfortably as part of every CI build. For 10M–100M, it’s an overnight batch.

Confidence representation

Every Locke report carries an 8-dimensional BeliefScore:

BeliefScore : {schema, missingness, drift, leakage, lineage, sample, duplication, constraint} → [0, 1]
overall_score : BeliefScore → [0, 1]

Each axis lives on [0, 1] where 1 means “no evidence against the dataset on this axis.” There is no single number; the overall is a weighted mean controlled by a BeliefWeights vector. The choice of 8 axes is deliberate — each maps to a disjoint class of finding codes:

Adding a new check assigns it to exactly one axis. That’s a hard invariant in the codebase, not a guideline.

Confidence composition under transformation

A LockeIdea derived from one or more parent ideas / impressions inherits a belief score:

belief(child) = meet(belief(parent_i)) ∧ transform_factor(op)   for each parent i
where meet(a, b) = component-wise min on the 8 axes

The meet (component-wise min) is the deliberately conservative composition rule. A filter cannot improve the missingness score — at best it preserves it. A join cannot improve any axis below the minimum of the two inputs. Belief is monotonically non-increasing under derivation.

transform_factor(op) is a per-operator scalar in [0, 1]^8 that accounts for op-specific weakening:

These are shipped today, conservative, documented in the source. The full algebra (per-axis composition rules per operator instead of a global meet) is a v0.2 goal.

What weakens confidence

The penalty function is the only place evidence turns into a score reduction:

penalty(finding) = w_axis(finding.code) × severity_weight(finding.severity) × evidence_intensity(finding.evidence)
axis_score(axis) = 1 − Σ penalty(f) for f in findings if axis_of(f.code) == axis    clamped to [0, 1]

Three properties:

1. severity_weight is a fixed ladder: {Info: 0.05, Notice: 0.15, Warning: 0.40, Error: 1.00} — same on every run. No floating arithmetic that depends on iteration order.
2. evidence_intensity is bounded by 1. A single finding cannot drop an axis below 0; multiple findings sum but are clamped at the end.
3. The penalty model is swappable. BeliefPenalty::user_supplied(...) lets a team install a custom ladder; the audit then records the penalty model’s identity. The default ladder is just one entry in a registry.

How propagation is modeled

The lineage DAG carries belief from impressions to ideas via the meet-and-transform_factor rule. Here’s an evidence-propagation diagram for a 4-step pipeline:

flowchart LR
    I["events.csv<br/><b>impression</b><br/>[1.00, 1.00, 1.00, 1.00,<br/>1.00, 1.00, 1.00, 1.00]"]:::imp
    I -->|"observe()<br/>tf = identity"| O["raw_events<br/><b>idea</b><br/>[1.00, 1.00, 1.00, 1.00,<br/>0.95, 1.00, 1.00, 1.00]"]:::idea
    O -->|"filter(active==1)<br/>tf = identity"| F["active_events<br/><b>idea</b><br/>[1.00, 0.99, 1.00, 1.00,<br/>0.95, 0.98, 1.00, 1.00]"]:::idea
    F -->|"group_by(plan_tier)<br/>tf: sample × 0.80"| G["grouped<br/><b>idea</b><br/>[1.00, 0.99, 1.00, 1.00,<br/>0.95, 0.78, 1.00, 1.00]"]:::idea
    G -->|"with_column(churn_rate)<br/>tf: leakage × 0.90"| W["labeled<br/><b>idea</b><br/>[1.00, 0.99, 1.00, 0.90,<br/>0.95, 0.78, 1.00, 1.00]"]:::idea

    classDef imp fill:#e8f4fd,stroke:#2196F3,stroke-width:2px
    classDef idea fill:#fff3e0,stroke:#FF9800,stroke-width:1px

Reading the diagram: each node label lists the 8-axis belief vector in fixed order. The observe() step pre-weakened lineage to 0.95 because raw-CSV ingestion drops some provenance (no upstream observation timestamps). The filter step weakened missingness and sample because filtering revealed a column with a higher local NaN rate, and dropped row count. The group_by step weakened sample further because per-group sample sizes are smaller. The with_column(churn_rate) step weakened leakage because the derived column directly references the target.

Crucially: every axis-wise value on the rightmost node is ≤ the leftmost. Composition never improves belief on any axis.

Deterministic update rules

Every step in the propagation uses:

• Kahan summation for penalty accumulation (avoid FP-order-dependent drift).
• BTreeMap for finding aggregation by axis (no HashMap iteration nondeterminism).
• f64::total_cmp for floating comparisons (NaN behaves consistently, not implementation-defined).
• No FMA in the SIMD paths under penalty arithmetic (bit-identical across CPUs).
• Single-threaded execution in v0.1 — even when parallel column-validation lands in v0.2, the reduction phase will be order-fixed.

The contract: same input bytes → same belief vector. The lineage diagram above is reproducible byte-for-byte by anyone running the same pipeline on the same inputs.

Confidence algebra (shipped vs planned)

The composition operator meet plus the identity ⊤ = [1, 1, 1, 1, 1, 1, 1, 1] satisfies five algebraic properties today:

Together, (BeliefScore, meet, ⊤) is a commutative idempotent monoid with identity — i.e. a meet-semilattice in lattice-theoretic terms. The deterministic update rules above make the algebra computable to bit-identical results across runs. That’s the formal object the snapshot-and-diff workflow relies on; without monotonicity, “appeared / disappeared” diffs over composed pipelines wouldn’t be well-defined.

Planned for v0.2:

• Per-axis composition rules instead of a global meet. Some axes (e.g. lineage) compose by min; others (e.g. sample) should compose by product. A typed rule per axis would be more faithful to what each axis measures.
• A join operator for combining alternative views of the same dataset (the lattice-theoretic dual of meet). Currently absent; the algebra is meet-only, which means alternative-view combination is undefined.
• A continuous-domain semantics for transformations that span a parameter (e.g. filter(predicate) with predicate selectivity s ∈ [0, 1]). v0.1 treats every filter the same; v0.2 would parameterise transform_factor by s.
• A formal proof of the conservation property: under the meet algebra, the gate-diff workflow’s appeared set is exactly the set of findings whose axes degraded between snapshots. This is true by construction today; v0.2 would have it written down as a theorem with a checked proof.

The honest framing: Locke v0.1 ships a small, deliberately conservative algebra. It is enough to make the workflows in this post (“snapshot, diff, gate, audit”) provably reproducible and well-defined. The richer semantics will land when the workflows the algebra is supposed to serve are more developed.

Belief degradation, visualised

The same 4-step pipeline rendered as a degradation chart — one panel per axis, showing the trajectory of each component as we move from impression to most-derived idea:

flowchart LR
    subgraph S1["events.csv"]
        s1_schema["schema = 1.00"]:::held
        s1_miss["missingness = 1.00"]:::held
        s1_drift["drift = 1.00"]:::held
        s1_leak["leakage = 1.00"]:::held
        s1_lin["lineage = 1.00"]:::held
        s1_sam["sample = 1.00"]:::held
        s1_dup["duplication = 1.00"]:::held
        s1_con["constraint = 1.00"]:::held
    end

    subgraph S2["raw_events<br/>(observe)"]
        s2_schema["1.00"]:::held
        s2_miss["1.00"]:::held
        s2_drift["1.00"]:::held
        s2_leak["1.00"]:::held
        s2_lin["0.95"]:::weakened
        s2_sam["1.00"]:::held
        s2_dup["1.00"]:::held
        s2_con["1.00"]:::held
    end

    subgraph S3["active_events<br/>(filter)"]
        s3_schema["1.00"]:::held
        s3_miss["0.99"]:::weakened
        s3_drift["1.00"]:::held
        s3_leak["1.00"]:::held
        s3_lin["0.95"]:::held
        s3_sam["0.98"]:::weakened
        s3_dup["1.00"]:::held
        s3_con["1.00"]:::held
    end

    subgraph S4["grouped<br/>(group_by)"]
        s4_schema["1.00"]:::held
        s4_miss["0.99"]:::held
        s4_drift["1.00"]:::held
        s4_leak["1.00"]:::held
        s4_lin["0.95"]:::held
        s4_sam["0.78"]:::weakened
        s4_dup["1.00"]:::held
        s4_con["1.00"]:::held
    end

    subgraph S5["labeled<br/>(with_column)"]
        s5_schema["1.00"]:::held
        s5_miss["0.99"]:::held
        s5_drift["1.00"]:::held
        s5_leak["0.90"]:::weakened
        s5_lin["0.95"]:::held
        s5_sam["0.78"]:::held
        s5_dup["1.00"]:::held
        s5_con["1.00"]:::held
    end

    s1_schema --- s2_schema --- s3_schema --- s4_schema --- s5_schema
    s1_miss   --- s2_miss
    s2_miss   -.-> s3_miss
    s3_miss   --- s4_miss --- s5_miss
    s1_drift  --- s2_drift  --- s3_drift  --- s4_drift  --- s5_drift
    s1_leak   --- s2_leak   --- s3_leak   --- s4_leak
    s4_leak   -.-> s5_leak
    s1_lin    -.-> s2_lin
    s2_lin    --- s3_lin    --- s4_lin    --- s5_lin
    s1_sam    --- s2_sam
    s2_sam    -.-> s3_sam
    s3_sam    -.-> s4_sam
    s4_sam    --- s5_sam
    s1_dup    --- s2_dup    --- s3_dup    --- s4_dup    --- s5_dup
    s1_con    --- s2_con    --- s3_con    --- s4_con    --- s5_con

    classDef held fill:#e8f5e9,stroke:#2E7D32
    classDef weakened fill:#ffebee,stroke:#C62828

The chart makes one specific claim visually: the dashed edges are the only places confidence got weaker. Every other axis held flat. A reviewer can pin a downstream finding to the specific transformation that degraded the relevant axis, because the dashed edge is exactly where the audit trail records a transform_factor application.

This is the property that makes the audit trail useful, not just reproducible. Reproducibility says “two runs of the same pipeline match.” Localisability says “when something downstream is wrong, I can point to which step weakened the belief on the affected axis.” Locke gives both.

Where the formal model lives in code

If you want to read the actual implementation rather than the prose:

• crates/cjc-locke/src/belief.rs — BeliefScore, BeliefWeights, BeliefPenalty definitions and the meet operator.
• crates/cjc-locke/src/lineage.rs — LineageGraph, LockeImpression, LockeIdea, edge representations, content-addressed IDs.
• crates/cjc-locke/src/traced.rs — TracedDataFrame wrapper that applies transform_factor per operation.
• crates/cjc-locke/src/dispatch.rs — the 22 locke_* builtins that expose the model to .cjcl source.

Everything is pub and documented. The composition rules are unit-tested for monotonicity, idempotence, and associativity. The deterministic-update rules are property-tested by proptest. The “shipped vs planned” gaps above each have an issue or an // TODO(v0.2): marker in the source.

What determinism does not buy you

I want to be careful here, because there’s an easy and wrong way to tell this story: “determinism makes computation greener.” It doesn’t — not directly, and per-run it’s often the opposite. Kahan and binned summation do more arithmetic than a naive sum. A seeded RNG, ordered collections, audit trails, and bit-exact replay all cost extra cycles and memory. Measured one run at a time, “trustworthy” is more expensive than “fast and loose,” not less.

So the green argument is not “determinism reduces FLOPs.” It’s narrower and, I think, more defensible: determinism, numerical stability, and bounded execution reduce the number of runs you waste on the way to a result you can actually trust. The savings live in the denominator — fewer reruns — not the numerator. Everything below should be read in that light.

A deterministic energy estimate

Most energy tools (CodeCarbon, Scaphandre, RAPL counters) measure power from hardware — which is non-deterministic and machine-specific. I wanted energy to be a number you can assert in a test. So energy_estimate is a pure function of the workload, never of wall-clock time:

joules ≈ flops × 1e-10  +  bytes × 1e-10
         (~100 pJ/flop)     (~100 pJ/byte, DRAM-ish)

The two terms are computed in separate bindings so the compiler can’t fuse them into an FMA — the same no-FMA discipline as the SIMD kernels. Same program, same seed → same FLOP count → same joules, bit-for-bit. On the branch, energy_estimate(1000000, 2000) returns 0.00010020000000000001 — identical under every profile, thread count, and both executors. The absolute number is an order-of-magnitude estimate; the ratio between two runs is the meaningful, reproducible figure.

What the fixed cap costs

Measured on an 8-core machine, release build, cjcl bench (10 runs) on a 1024×1024 matmul:

So the cap genuinely throttles (max-perf is ~3× faster than single-threaded) — and the laptop-safe default costs ~27% versus all-out. That cost is real, and Phase 2 is about clawing most of it back.

Did it work? Two regimes, measured

Short workload (one 1024² matmul, finishes inside the burst window), minimum of 9 --time runs:

Adaptive-cool recovered ~1.8× (0.353 → 0.196 s) — landing within fixed startup overhead of max-perf. The short, interactive case now runs at full speed and never trips the cap.

Sustained workload (25× 1024² matmuls, ~runs well past the 2 s window):

This is the part I care about most: adaptive-cool (10.8 s) sits between fixed-cool and max-perf. It is not near max-perf (4.2 s) — meaning it genuinely throttled after the burst. The thermal bound held; the burst just bought back the first couple of seconds. Exactly the goal: fast when it’s safe, bounded when it matters.

Why this is safe

The burst-vs-throttle decision reads the wall clock, so the schedule is non-deterministic run-to-run. But the output isn’t, because the choice only changes how many bands run concurrently — never the per-element math. To prove it rather than assert it: a 256×256 matrix product (a real parallel reduction over varied values) hashed byte-identical across --threads 1, cool, balanced, max-perf, adaptive on/off, and both the AST and MIR executors:

threads=1  (eval)    : 79aaf9931d2b6c2004a20cdf5d5bd434
threads=8  (eval)    : 79aaf9931d2b6c2004a20cdf5d5bd434
cool (adaptive burst): 79aaf9931d2b6c2004a20cdf5d5bd434
cool --no-adaptive   : 79aaf9931d2b6c2004a20cdf5d5bd434
max-perf             : 79aaf9931d2b6c2004a20cdf5d5bd434
MIR cool (adaptive)  : 79aaf9931d2b6c2004a20cdf5d5bd434

For reproducible benchmarking — where you want a fixed schedule — --no-adaptive applies the cap uniformly.