A Browser-Based Double-Blind ABX Comparator with Exact Binomial Inference

1. Introduction

Claims about audible differences between cables, formats or processing chains are usually argued from sighted listening, which is dominated by expectation effects. The standard corrective is the blind ABX test: the listener may audition known references A and B at will, and must identify a randomised X. Running ABX properly has historically required either a second person to administer randomisation, or dedicated software. A browser implementation lowers the barrier to the point where anyone with a laptop can run a defensible test on their own system.

2. Methodology

2.1 Stimulus handling

Stimuli are decoded in the browser with the Web Audio API's decodeAudioData and played back through a single gain stage at unity. No processing is applied; A, B and X share the same playback path, so the path itself cannot be a confound. Users may supply their own files, which are decoded locally and never leave the device, or use hosted sample pairs with documented, subtle differences, for example a +0.3 dB high shelf above 5 kHz or a 0.05% resampling drift.

2.2 Trial design and bias controls

Each trial selects X as A or B with equal probability, independently of previous trials, so sequences carry no exploitable pattern. The identity of X is never shown during the trial, and because randomisation happens in code rather than by a human administrator, the test is double-blind in the practical sense: neither the listener nor any administrator can influence or reveal the assignment. A and B may be replayed freely, which matches established small-impairment practice and avoids penalising listeners for imperfect auditory memory. Feedback is given only as a running score; individual trial answers are recorded without revealing X, to limit learning effects within a session.

2.3 Statistical inference: why exact binomial

Under the null hypothesis that the listener cannot hear a difference, the number of correct identifications k in n trials follows a binomial distribution with p = 0.5. We compute the exact two-sided p-value directly from the binomial probability mass function: the probability of a result at least as far from chance as the one observed, in either direction, doubled and capped at one.

Normal approximations to the binomial are common in informal tooling, but they misbehave exactly where listening tests live: small n and extreme k. A listener who scores 9 out of 10 is well outside the region where the approximation is trustworthy, and continuity corrections only partly repair it. The exact test costs a few floating-point operations at any realistic trial count, so there is no justification for the approximation. Reported verdicts follow conventional bands: p below 0.01 is treated as strong evidence of an audible difference, p below 0.05 as significant evidence, and anything higher as no proven difference, which is not the same as proof of no difference.

2.4 Session reporting

A session closes with a plain-text result card: sample source, trials, correct count, exact p-value and verdict band. The card is copyable so results can be shared verbatim. With the listener's explicit action, an anonymised summary (counts, p-value and sample source only) is submitted to our first-party analytics; no audio, file names, or identifying information is transmitted.

3. Limitations

A self-administered test cannot enforce controlled conditions: playback level matching between A and B is the listener's responsibility when supplying their own files, and an unmatched level pair will produce a "significant" result that reflects loudness rather than quality. Listeners who run many short sessions and report only the flattering ones are performing multiple comparisons; the honest unit of evidence is a pre-committed session of, say, 16 or more trials. Finally, "no audible difference proven" at a modest trial count is weak evidence either way; absence of significance is not significance of absence.

4. Conclusion

A browser-based ABX comparator with exact binomial scoring puts a defensible listening-test protocol within reach of any customer. It will not settle arguments about whether a difference matters, but it can settle whether a difference is heard at all, which is the right first question.

References

1. ITU-R BS.1116-3, Methods for the subjective assessment of small impairments in audio systems, International Telecommunication Union, 2015.
2. ITU-R BS.1534-3, Method for the subjective assessment of intermediate quality level of audio systems (MUSHRA), International Telecommunication Union, 2015.
3. Audio Engineering Society convention papers on listening-test methodology and double-blind testing practice (various years).