Green by Construction: Thermal Bounds Without Losing a Bit
How I made CJC-Lang thermally bounded and more memory-efficient — while keeping output bit-identical.
The problem: a laptop that cooks itself
CJC-Lang’s core promise is same seed, same output, always. To get there it already does the unglamorous work: ordered collections, Kahan/binned summation, a seeded RNG, and — critically for this story — no FMA in the SIMD kernels, so a multiply-then-add rounds the same way every time.
But nothing in the runtime bounded how hard it pushed the machine. The parallel matmul and reduction kernels ran on Rayon’s default global pool — every core, all the time. On a laptop, a training loop would peg all 8 cores, spin the fans up, and eventually thermal-throttle: the CPU gets hot enough that the firmware lowers the clock to protect itself, and you end up slower than if you’d never saturated it.
I wanted an explicit, deterministic answer to “how much machine may this run use?” — without the OS power governor’s opacity, and without breaking the reproducibility contract.
The reframe that makes it tractable
Here’s the mental model I kept coming back to:
Energy = power × time. Capping cores cuts power but raises time. On a short burst that’s often a wash — you run cooler but longer. The levers that recover performance and save energy attack the other terms: total work (fewer cycles), memory traffic (the dominant energy term), and when you spend the thermal budget.
Crucially: bursts don’t overheat — only sustained load does. Thermal time constants are seconds to tens of seconds, so a sub-second burst at full tilt never heat-soaks the package. A fixed thread cap pays the throttle tax on every operation, including the short interactive runs that were never going to overheat.
And the linchpin that makes all of this safe: in CJC-Lang, thread count never changes the result. The parallel reductions sum in a fixed chunk order with Kahan/binned accumulators, so the numeric output is identical no matter how many workers are live. That single guarantee is what lets me treat parallelism as a pure heat/speed knob — and even make it adaptive — without ever risking the answer.
What determinism does not buy you
I want to be careful here, because there’s an easy and wrong way to tell this story: “determinism makes computation greener.” It doesn’t — not directly, and per-run it’s often the opposite. Kahan and binned summation do more arithmetic than a naive sum. A seeded RNG, ordered collections, audit trails, and bit-exact replay all cost extra cycles and memory. Measured one run at a time, “trustworthy” is more expensive than “fast and loose,” not less.
So the green argument is not “determinism reduces FLOPs.” It’s narrower and, I think, more defensible: determinism, numerical stability, and bounded execution reduce the number of runs you waste on the way to a result you can actually trust. The savings live in the denominator — fewer reruns — not the numerator. Everything below should be read in that light.
Phase 1: a Runtime Policy Layer
The first piece is a RuntimePolicy — the green-compute control surface. It lives in a thread-local in cjc-runtime, mirroring the existing profiler sink.
pub struct RuntimePolicy {
pub determinism: Determinism, // Strict
pub numeric_mode: NumericMode, // Kahan | Binned | FixedTree
pub thermal_mode: ThermalMode, // Cool | Balanced | MaxPerf
pub max_threads: usize, // 0 = auto (derive from thermal_mode)
pub batch_size: usize,
pub audit_mode: AuditMode, // Summary | Full | Forensic
pub adaptive: bool, // race-to-idle (Phase 2)
}Three thermal profiles, exposed as CLI flags applied once at startup so they cover run, bench, and every command:
| Profile | Threads (8-core host) | Batch | Use |
|---|---|---|---|
cool |
≈ ¼ cores → 2 | 32 | laptop, edge, gentle |
balanced (default) |
≈ ½ cores → 4 | 128 | normal — leaves headroom |
max-perf |
all → 8 | 512 | benchmarking |
The default is balanced, deliberately not “all cores forever.”
A deterministic energy estimate
Most energy tools (CodeCarbon, Scaphandre, RAPL counters) measure power from hardware — which is non-deterministic and machine-specific. I wanted energy to be a number you can assert in a test. So energy_estimate is a pure function of the workload, never of wall-clock time:
joules ≈ flops × 1e-10 + bytes × 1e-10
(~100 pJ/flop) (~100 pJ/byte, DRAM-ish)The two terms are computed in separate bindings so the compiler can’t fuse them into an FMA — the same no-FMA discipline as the SIMD kernels. Same program, same seed → same FLOP count → same joules, bit-for-bit. On the branch, energy_estimate(1000000, 2000) returns 0.00010020000000000001 — identical under every profile, thread count, and both executors. The absolute number is an order-of-magnitude estimate; the ratio between two runs is the meaningful, reproducible figure.
What the fixed cap costs
Measured on an 8-core machine, release build, cjcl bench (10 runs) on a 1024×1024 matmul:
| Profile | Threads | Mean |
|---|---|---|
--threads 1 |
1 | 628 ms |
cool |
2 | 468 ms |
balanced (default) |
4 | 268 ms |
max-perf |
8 | 210 ms |
So the cap genuinely throttles (max-perf is ~3× faster than single-threaded) — and the laptop-safe default costs ~27% versus all-out. That cost is real, and Phase 2 is about clawing most of it back.
Phase 2: race-to-idle adaptive scheduling
A fixed cap throttles short and long workloads alike. Race-to-idle fixes that: run at full width during a burst, and throttle to the cap only once load is sustained (a 2-second window; a ½-second idle gap resets it).
The mechanism is the interesting part. Rayon’s global pool can only be configured once, so I don’t shrink it — I leave it at full size so bursts can use every core. Throttling happens per-operation: when a workload has been busy past the window, the kernel’s parallel section is install-ed into a cached, cap-sized pool. Inside that pool rayon::current_num_threads() reports the cap, so the existing band/row/chunk splitters auto-scale — no per-kernel rewrites.
pub fn run_parallel<R: Send>(work: impl FnOnce() -> R + Send) -> R {
if rayon::current_thread_index().is_some() { return work(); } // nested → inline
let cap = effective_threads(&get(), detect_cores());
if cap >= detect_cores() { return work(); } // no cap
let throttle = if get().adaptive { is_sustained_now() } else { true };
if !throttle { return work(); } // burst → full pool
match capped_pool(cap) { Some(p) => p.install(work), None => work() }
}The four parallel kernels (large/medium/batch matmul, SIMD elementwise) wrap their bodies in run_parallel.
Did it work? Two regimes, measured
Short workload (one 1024² matmul, finishes inside the burst window), minimum of 9 --time runs:
| cool fixed | cool adaptive | max-perf | |
|---|---|---|---|
| time | 0.353 s | 0.196 s | 0.153 s |
Adaptive-cool recovered ~1.8× (0.353 → 0.196 s) — landing within fixed startup overhead of max-perf. The short, interactive case now runs at full speed and never trips the cap.
Sustained workload (25× 1024² matmuls, ~runs well past the 2 s window):
| cool fixed | cool adaptive | max-perf | |
|---|---|---|---|
| time | 16.7 s | 10.8 s | 4.2 s |
This is the part I care about most: adaptive-cool (10.8 s) sits between fixed-cool and max-perf. It is not near max-perf (4.2 s) — meaning it genuinely throttled after the burst. The thermal bound held; the burst just bought back the first couple of seconds. Exactly the goal: fast when it’s safe, bounded when it matters.
Why this is safe
The burst-vs-throttle decision reads the wall clock, so the schedule is non-deterministic run-to-run. But the output isn’t, because the choice only changes how many bands run concurrently — never the per-element math. To prove it rather than assert it: a 256×256 matrix product (a real parallel reduction over varied values) hashed byte-identical across --threads 1, cool, balanced, max-perf, adaptive on/off, and both the AST and MIR executors:
threads=1 (eval) : 79aaf9931d2b6c2004a20cdf5d5bd434
threads=8 (eval) : 79aaf9931d2b6c2004a20cdf5d5bd434
cool (adaptive burst): 79aaf9931d2b6c2004a20cdf5d5bd434
cool --no-adaptive : 79aaf9931d2b6c2004a20cdf5d5bd434
max-perf : 79aaf9931d2b6c2004a20cdf5d5bd434
MIR cool (adaptive) : 79aaf9931d2b6c2004a20cdf5d5bd434For reproducible benchmarking — where you want a fixed schedule — --no-adaptive applies the cap uniformly.
Phase 3a: fused kernels cut memory traffic
The third lever is memory. A plain a*b + c allocates an intermediate: mul → a fresh tensor, then add → another. That’s two allocations and ~5N words of traffic for an N-element result. A fused single-pass kernel does it in one allocation and one pass (~4N words).
I added three, joining the existing broadcast_fma:
| Builtin | Computes | Replaces |
|---|---|---|
fused_axpy(α, x, y) |
α·x + y |
scalar-mul + add (BLAS axpy) |
fused_mul_sub(a, b, c) |
a·b − c |
mul + sub |
fused_sub_sq(a, b) |
(a−b)² |
sub + mul (MSE/variance/distance) |
The subtle bit is why this is free: a*b + c in Rust compiles to a separate multiply then add — it does not contract into a hardware FMA unless you call .mul_add() explicitly. So the fused loop produces bit-identical results to the two-pass version while eliminating the intermediate. I pinned that with property tests asserting fused == unfused byte-for-byte over 256 random cases each.
And this compounds with Phase 2: lighter work finishes inside the burst window, so the cap engages even less. Memory efficiency and the adaptive scheduler reinforce each other instead of trading off.
Where the waste actually is
If determinism makes each run a little more expensive, why is any of this “green”? Because the dominant source of wasted compute in real numerical and ML work isn’t the cost of one run — it’s runs you have to throw away. Unstable, irreproducible systems waste compute, and they waste a lot of it:
- Nondeterministic drift. A number changes between runs and you can’t tell whether it’s a bug or noise — so you rerun, sometimes dozens of times, just to characterize the variance.
- NaN / instability cascades. An unstable reduction overflows or loses precision, a training run diverges, and everything downstream is garbage. You debug, patch, and rerun the whole pipeline.
- Failed reproductions. A reviewer, a teammate, or future-you can’t reproduce a result, so the experiment is repeated from scratch.
- Preprocessing redone because the previous run wasn’t pinned.
Every one of those is compute already spent that produced nothing trustworthy. That’s the waste CJC-Lang is actually attacking — with stable reductions, a seeded RNG, bit-exact replay, and now bounded, memory-aware execution. None of it makes a single run cheaper. It makes reruns unnecessary.
An honest metric: trustworthy results per joule
Throughput (FLOPs/second) is the wrong yardstick for this. A better one is closer to:
trustworthy_results / joule — useful, reproducible outputs per unit of energy, counting the energy burned on runs you had to repeat.
I ran a small experiment to make it concrete. A 512×512 matmul, with the deterministic energy estimate from Phase 1:
result summary : 2307326611210371000 p.sum()
energy estimate : 0.0274726912 J energy_estimate(2·512³ flops, 3·512²·8 bytes)Then I checked reproducibility — the same program across every profile, thread cap, adaptive on/off, and both executors:
e81183611e12dd95d56e79b9b36ff194 default
e81183611e12dd95d56e79b9b36ff194 --profile max-perf
e81183611e12dd95d56e79b9b36ff194 --profile cool
e81183611e12dd95d56e79b9b36ff194 --profile cool --no-adaptive
e81183611e12dd95d56e79b9b36ff194 --threads 1
e81183611e12dd95d56e79b9b36ff194 --mir-optIdentical, every time. So:
- One run yields a result you can trust → ~0.0275 J per trustworthy result (≈ 36 results/joule, at this analytical estimate).
- Certifying it — producing it and independently re-running to confirm bit-equality — costs two runs (~0.055 J) and reaches actual certainty.
Now the honest part, because this is exactly where the story could overreach. That 0.0275 J is the numerator, and determinism did not shrink it — the stable reductions arguably made it slightly bigger. The metric’s real lever is the rerun multiplier: if establishing a trustworthy result takes k runs, you spend k × 0.0275 J. In a nondeterministic pipeline chasing drift or instability, k can be many; here it is 1, and verification is exact replay rather than statistics-over-noise. I’m deliberately not fabricating a competitor’s k — I can’t measure someone else’s pipeline, and I’m not going to pretend to. The claim is narrow and checkable: CJC-Lang drives k toward 1, and that — not FLOPs-per-run — is where the energy is saved.
(Two caveats stack here: the joules are an analytical estimate, not a wattmeter reading, and the rerun multiplier is a model, not a benchmark of another system. Both are stated so you can discount them appropriately.)
Why bounded heat matters beyond one run
The thermal bound isn’t only about the run in front of you. Three stakes, in decreasing order of how confidently I can claim them:
- Short run — the throttle cliff (measured). Saturate every core long enough and the firmware lowers the clock to protect the chip, so a thermally-unbounded run can finish slower than a bounded one. Capping sustained load keeps you on the right side of that cliff.
- Long run — hardware stress (physics, not a benchmark). Sustained operation near thermal limits accelerates component aging — degradation roughly follows an Arrhenius curve (≈2× per 10 °C), and thermal cycling fatigues solder joints and packaging. Running cooler reduces that stress. I haven’t measured device lifetimes — that’s a multi-year claim — so I’ll state it as physics, not a result.
- At scale — heat is cooling cost. In a datacenter, a large slice of energy goes to removing the heat compute produces (it’s a big term in PUE). Bounding sustained load lowers the cooling burden — it complements cooling rather than replacing it (you still buy the heatsink). On a laptop the same lever buys quieter fans and longer battery.
And one correction worth making explicit, because it’s easy to get backwards: heat threatens performance, not the answer. Thermal throttling changes the clock, which changes timing — never the bits. CJC-Lang’s output is timing-independent by construction (fixed reduction order, seeded RNG, no result-affecting races), so a throttled run is byte-identical to a cool one, just slower. What heat actually breaks is performance reproducibility — a hot benchmark isn’t repeatable. So the two guarantees are separate and complementary: the thermal bound protects how fast you go; determinism protects what you get — and it survives the chip throttling. (In a non-deterministic, timing-dependent system, throttling-induced timing shifts genuinely can perturb results — which is exactly the failure mode CJC-Lang is built not to have.)
How unique is this?
Honest answer: the individual pieces all exist somewhere. Thread caps are everywhere (Rayon, OpenMP, GOMAXPROCS). OS governors and RAPL do power capping. Race-to-idle is textbook power management. CodeCarbon and friends estimate energy. Intel MKL has reproducible-numerics modes.
What I haven’t seen packaged together is the combination at the language-runtime level with a determinism guarantee: a thermal/energy policy where the knobs are provably result-invariant, paired with an adaptive scheduler and a reproducible energy estimate. In most stacks, changing thread count can change floating-point results; here it provably can’t, which is the whole reason an adaptive thermal scheduler is even safe to ship. It’s most at home in the niche CJC-Lang targets — reproducible scientific, regulated, and edge computing — not hyperscale throughput.
What I measured vs. what I estimated
In keeping with the project’s ethos, the honest accounting:
- Measured: all the wall-clock numbers (release
cjcl bench/--time), the determinism hashes, the test counts. These are real and on the branch. - A proxy, not a measurement: I measured wall-clock and active-core counts, not watts or junction temperature. Fewer active cores means less package power, but I don’t have hardware sensors in the loop — treat the thermal claims as a well-founded proxy, not a power reading.
- Analytical, not benchmarked: the fused-kernel traffic reduction (~5N → ~4N words, two allocations → one) is from counting, not a wall-clock microbenchmark. What I did verify is that the output is bit-identical to the unfused path.
- Deferred: the data-DSL group-by/join memory work (Phase 3b) is not in this post. The obvious approach turned out to change output order (the streaming path sorts keys; the materializing path preserves first-occurrence order), which would break determinism — so I stopped rather than ship a correctness regression. It needs an order-preserving design, which is its own phase.
Verification
Every claim above is backed by tests added for this work: 80 new tests — 20 unit and 43 integration for the policy/scheduling layer, plus 17 for the fused kernels — spanning unit, dual-executor parity (AST vs MIR), property (256 cases each), and fuzz checks. The existing suite stayed green and no determinism gate moved.
A note on tone, because it’s easy to get this wrong: none of this makes CJC-Lang “environmentally superior” to anything. It’s far too early for that claim, and against a hyperscale GPU cluster it isn’t even the right comparison. The honest framing is narrower and more interesting as a research question — CJC-Lang is exploring whether deterministic, memory-aware, thermally-bounded systems can reduce wasteful computation (the reruns, the drift-debugging, the throwaway experiments) in the niches where trust matters more than peak throughput. That’s the hypothesis this work is testing, not a verdict.
The throughline: you don’t have to choose between fast, reproducible, and cool. You can have a thermal bound that adapts to your workload, energy you can reason about, and lower memory traffic — and still get the same bits out every single time. In a deterministic language, that last clause isn’t a nice-to-have. It’s the whole point.