abxtest man page
abxtest — double-blind ABX comparison testing script
Synopsis
Description
abxtest is a tool for conducting listening (or other subjective) tests to determine whether a listener can discern a difference between two subjects under test, denoted A and B.
The listener conducts several trials in which an unknown subject, X, is randomly chosen to be either A or B. The probability that X is A or B is the same. The listener must decide whether X sounds the same as A or the same as B. The listener may repeat A, B, and X as many times as necessary before reaching a verdict.
In order to prove the hypothesis that a difference can be discerned between A and B, the listener must disprove the null hypothesis that A sounds the same as B. This is accomplished by correctly identifying X often enough that randomly guessing correctly the same number of times would be statistically improbable.
The A and B subjects for abxtest are actually arbitrary shell commands. The normal mode of use would be to give commands which play two slightly different sounds. The standard output and standard error from each command are suppressed so as not to reveal the identity of X. The listener may interrupt an executing command (usually by pressing ^C) without terminating the test.
Options
- -n min
Set the minimum number of trials. Trials will continue after this number if the statistical probability goal has not yet been met. The default is 10.
- -m max
Set the maximum number of trials. Trials will end after this number even if the statistical probability goal has not been met. The default is 20.
- -g goal
Set the minimum statistical probability goal. This is the probability that the number of correct identifications is the same as random guesses. In order to disprove the null hypothesis, the result of the test must be a probability less than or equal to this goal. The default is 0.05.
Files
- /dev/random
By default this file is read to generate a random sequence of X.
Environment
- RANDOM_FILE
If set, the file named by this variable will be read instead of /dev/random to generate a sequence of X.
Notes
abxtest uses the binomial distribution to accurately calculate the probability that the result is the same as random guesses.
Author
Robert Leslie <rob@mars.org>