Mercury Counters: Less Memory Traffic, No Verified Speedup Yet

Mercury counters are the whole result of this campaign, and that is the point of writing it up. On FFAI master, a set of changes to the speech component deleted hundreds of megabytes of memset and copy traffic per test run. A same-day wall-clock pass could not resolve it and was recorded as inadmissible, so speed is unverified, not unchanged. The engineering lesson is the instrument: when the counter moves and the clock cannot resolve it, the honest output is a counter claim and no speed claim.

Release status first. Commits bae28c23 and ef6c355f are on master. The crate version is still 1.1.0, the changelog's Unreleased section is empty, and crates.io ffai-mercury 1.1.0 was built from an earlier commit on 31 October 2026. None of this is in a published crate yet.

Mercury is the ASR and TTS component of FFAI: Whisper-class recognition and VITS or piper-class synthesis on candle. The builder note is Mercury, and the comparison page is Pure Rust Alternative for whisper.cpp. The older figures on those pages were not part of this campaign. Every number below is repository-reported. Nothing was rerun for this article.

What the Mercury counters measured

For the Mercury counters, the baseline is a git worktree at 2635b6b9 with only the counters added. The candidate is bae28c23. The harness is the release example veins_gate, built with cargo build --release -p ffai-mercury --example veins_gate, on Windows with Rust 1.95, x86-64 with AVX2 and F16C. The CPU model is not named for the counter section.

Inputs: the first 12 LibriSpeech test-clean utterances plus longform-01, 13 clips, 176.3 s, 669 generated tokens, on whisper-tiny.en. TTS used the first 8 Harvard sentences, piper lessac-medium, seed 7. Beam identity used FFAI_BEAM_SIZE=3 on 4 clips.

• ASR memset: 573.3 MB to 3.3 MB, down 99.4 percent.
• ASR KV bytes copied: 510.0 MB to 14.4 MB, down 97.2 percent.
• ASR vocabulary heap: 142.5 MB to 0.
• ASR allocations: 914,722 to 463,103, down 49.4 percent.
• ASR bytes requested: 4,963,461,169 to 3,784,938,105, down 23.7 percent.
• ASR FFT calls: 31,000 to 18,972, down 38.8 percent.
• TTS memset: 502.6 MB to 34.4 MB. TTS tensor copies: 232.0 MB to 126.0 MB. TTS allocations: 86,944 to 74,075.
• Beam elements sorted: 17.9 M to 0.

These Mercury counters are totals over the run, not resident set sizes. The commit message cites 143 lib tests and the ledger lists a narrower 31-test module run plus 5 reachability and oracle-mel tests. Both statements are in the repository. Neither was rerun here.

Where the Mercury counters came from

The ledger is mercury-instruction-veins-ledger.md. It describes forty-five bricks. The big Mercury counters trace to a handful of them.

A flat KV cache read in place, the largest Mercury counter cut

The decoder's key and value cache became one flat in-place buffer, read directly by the decode attention kernel (flash_attn.rs). That is the 510.0 MB to 14.4 MB KV copy line. The encoder kernel also reads q and v unsplit.

Logits, bias, and buffers not zeroed twice

Logits are written into the caller's buffer, with one max pass reused. Bias is fused into the f16 GEMV. Buffers use spare capacity instead of zero-then-overwrite. The ASR vocabulary heap counter went from 142.5 MB to zero. Environment reads and name lookups are hoisted to load time. An exact grouped int8 reduction replaces the previous one.

Skipped FFTs, borrowed TTS reads, selection instead of sort

The mel front end skips the FFT on all-zero frames, the change behind the 31,000 to 18,972 FFT call counter. TTS flow, layer norm and relative attention read borrowed tensors instead of owned copies. Beam top-k uses selection instead of a full sort, so 17.9 M sorted elements become zero. FFAI_VEINS=off keeps legacy arms for veins 1, 3, 8 and 14.

Why timing could not yet verify the Mercury counters

The negative result is the article. It is worth reading as carefully as the Mercury counters themselves.

The inadmissible Mercury timing pass

Same day, ABBA, six rounds per arm after one discarded warm-up, n = 12 per binary per arm, pinned to logical CPUs 0 to 15 of an i7-14650HX. ASR ran with --no-tts, TTS with --no-asr. ASR wall medians read 6.07 s before and 8.17 s after, with the after arm faster in 2 of 6 paired rounds. TTS wall read 1.46 s and 1.40 s, 3 of 6. CPU time barely moved either way. The ledger rules the pass inadmissible: runs of the same binary inside one round differed by more than the before and after gap, and ASR wall drifted from 20 s to 4 s across the session. Quoting 8.17 s as a regression would be as wrong as quoting 1.40 s as a win.

Bandwidth arithmetic

The ledger's own estimate explains why timing could not resolve the Mercury counters. The removed bytes amount to about 0.1 s at memory bandwidth, below the 1 to 2 s spread between identical runs. Counting tools exist for exactly this case; the Rust Performance Book chapter on benchmarking notes that wall time can suffer from high variance and that instruction counts have lower variance. A counter can certify that work disappeared. It cannot certify that a user waits less.

Identity, bimodality, and release status

Greedy and beam identity sections were byte-identical to the baseline, and the legacy arm matched the baseline tree. That is output identity on these runs, not a word-error-rate study against the Whisper reference. A separate finding: 150 TTS-only runs per binary produced exactly two waveforms, the alternate in 5.3 percent of pre-campaign runs and 2.7 percent after, Fisher p about 0.38 (ledger). The README still says same text and seed give a byte-identical WAV. The ledger's narrower reading, true of the code path and not of every process, is the one to repeat. Repo landing: Mercury. Suite: FFAI. Cluster: Mind.