mysql-test-run.pl - Man Page

run MariaDB test suite

Synopsis

mysql-test-run.pl [options]

Description

The mysql-test-run.pl Perl script is the main application used to run the MariaDB test suite. It invokes mysqltest to run individual test cases.

Invoke mysql-test-run.pl in the mysql-test directory like this:

shell> mysql-test-run.pl [options] [test_name] ...

Each test_name argument names a test case. The test case file that corresponds to the test name is t/test_name.test.

For each test_name argument, mysql-test-run.pl runs the named test case. With no test_name arguments, mysql-test-run.pl runs all .test files in the t subdirectory.

If no suffix is given for the test name, a suffix of .test is assumed. Any leading path name is ignored. These commands are equivalent:

shell> mysql-test-run.pl mytest
shell> mysql-test-run.pl mytest.test
shell> mysql-test-run.pl t/mytest.test

A suite name can be given as part of the test name. That is, the syntax for naming a test is:

[suite_name.]test_name[.suffix]

If a suite name is given, mysql-test-run.pl looks in that suite for the test. The test file corresponding to a test named suite_name.test_name is found in suite/suite_name/t/test_name.test. There is also an implicit suite name main for the tests in the top t directory. With no suite name, mysql-test-run.pl looks in the default list of suites for a match and runs the test in any suites where it finds the test. Suppose that the default suite list is main, binlog, rpl, and that a test mytest.test exists in the main and rpl suites. With an argument of mytest or mytest.test, mysql-test-run.pl will run mytest.test from the main and rpl suites.

To run a family of test cases for which the names share a common prefix, use the --do-test=prefix option. For example, --do-test=rpl runs the replication tests (test cases that have names beginning with rpl). --skip-test has the opposite effect of skipping test cases for which the names share a common prefix.

The argument for the --do-test and --skip-test options also allows more flexible specification of which tests to perform or skip. If the argument contains a pattern metacharacter other than a lone period, it is interpreted as a Perl regular expression and applies to test names that match the pattern. If the argument contains a lone period or does not contain any pattern metacharacters, it is interpreted the same way as previously and matches test names that begin with the argument value. For example, --do-test=testa matches tests that begin with testa, --do-test=main.testa matches tests in the main test suite that begin with testa, and --do-test=main.*testa matches test names that contain main followed by testa with anything in between. In the latter case, the pattern match is not anchored to the beginning of the test name, so it also matches names such as xmainytesta.

To perform setup prior to running tests, mysql-test-run.pl needs to invoke mysqld with the --bootstrap and --skip-grant-tables options. If MySQL was configured with the --disable-grant-options option, --bootstrap, --skip-grant-tables, and --init-file will be disabled. To handle this, set the MYSQLD_BOOTSTRAP environment variable to the full path name of a server that has all options enabled. mysql-test-run.pl will use that server to perform setup; it is not used to run the tests.

The init_file test will fail if --init-file is disabled. This is an expected failure that can be handled as follows:

shell> export MYSQLD_BOOTSTRAP
shell> MYSQLD_BOOTSTRAP=/full/path/to/mysqld
shell> make test force="--skip-test=init_file"

To run mysql-test-run.pl on Windows, you'll need either Cygwin or ActiveState Perl to run it. You may also need to install the modules required by the script. To run the test script, change location into the mysql-test directory, set the MTR_VS_CONFIG environment variable to the configuration you selected earlier (or use the --vs-config option), and invoke mysql-test-run.pl. For example (using Cygwin and the bash shell):

shell> cd mysql-test
shell> export MTR_VS_CONFIG=debug
shell> ./mysqltest-run.pl --force --timer
shell> ./mysqltest-run.pl --force --timer --ps-protocol

mysql-test-run.pl uses several environment variables. Some of them are listed in the following table. Some of these are set from the outside and used by mysql-test-run.pl, others are set by mysql-test-run.pl instead, and may be referred to in tests.

VariableMeaning
MTR_VERSIONIf set to 1, will run the older version 1 of mysql-test-run.pl. This will affect what functionailty is available and what command line options are supported.
MTR_MEMIf set to anything, will run tests with files in "memory" using tmpfs or
               ramdisk. Not available on Windows. Same as
               --mem option
MTR_PARALLELIf set, defines number of parallel threads executing tests. Same as
               --parallel option
MTR_BUILD_THREADIf set, defines which port number range is used for the server
MTR_PORT_BASEIf set, defines which port number range is used for the server
MTR_NAME_TIMEOUTSetting of a timeout in minutes or seconds, corresponding to command line option --name-timeout. Available timeout names are TESTCASE, SUITE (both in minutes) and START, SHUTDOWN (both in seconds). These variables are supported from MySQL 5.1.44.
MYSQL_TESTPath name to mysqltest binary
MYSQLD_BOOTSTRAPFull path name to mysqld that has all options enabled
MYSQLTEST_VARDIRPath name to the var directory that is used for
               logs, temporary files, and so forth
MYSQL_TEST_DIRFull path to the mysql-test directory where tests
               are being run from
MYSQL_TMP_DIRPath to temp directory used for temporary files during tests

The variable MTR_PORT_BASE was added in MySQL 5.1.45 as a more logical replacement for MTR_BUILD_THREAD. It gives the actual port number directly (will be rounded down to a multiple of 10). If you use MTR_BUILD_THREAD, the port number is found by multiplying this by 10 and adding 10000.

Tests sometimes rely on certain environment variables being defined. For example, certain tests assume that MYSQL_TEST is defined so that mysqltest can invoke itself with exec $MYSQL_TEST.

Other tests may refer to the last three variables listed in the preceding table, to locate files to read or write. For example, tests that need to create files will typically put them in $MYSQL_TMP_DIR/file_name.

If you are running mysql-test-run.pl version 1 by setting MTR_VERSION, note that this only affects the test driver, not the test client (and its language) or the tests themselves.

A few tests might not run with version 1 because they depend on some feature of version 2. You may have those tests skipped by adding the test name to the file lib/v1/incompatible.tests. This feature is available from MySQL 5.1.40.

mysql-test-run.pl supports the options in the following list. An argument of -- tells mysql-test-run.pl not to process any following arguments as options.

See Also

For more information, please refer to the MariaDB Knowledge Base, available online at https://mariadb.com/kb/

Author

MariaDB Foundation (http://www.mariadb.org/).

Info

15 May 2020 MariaDB 10.11 MariaDB Database System