nihtest - Man Page
testing tool
Synopsis
nihtest [-hqVv] [--all] [-C config] [--debug] [--keep-broken] [--no-cleanup] [--setup-only] [testcase ...]
Description
nihtest is a testing tool for command line utilities. It uses .test files as input. The format of these files is described in nihtest-case(5).
nihtest runs the test cases specified on the command line or, if the --all option is given, in the suite section of the config file. Shell globs can be used on the command line, test cases are expected in the test case directory.
nihtest searches the current directory and the source-directory specified in nihtest.conf(5) for test cases, input and output files.
nihtest supports the following options
- --all
Run all tests specified in the suite section of the config file.
- -C config, --config-file config
Use config as configuration file instead of ./nihtest.conf.
- --debug
Set up the sandbox, but run the program inside a debugger. Results are not checked.
- -h, --help
Display a short help message and exit.
- --keep-broken
Do not delete the sandbox if the test fails.
- --no-cleanup
Do not delete the sandbox after the test finishes (successfully or not).
- -q, --quiet
Do not print test results.
- --setup-only
Only populate the sandbox, but do not run the actual test.
- -v, --verbose
Print detailed test results.
- -V, --version
Print nihtest version number and exit.
A test run consists of the following steps:
- check if test should be skipped
- create sandbox directory
- populate sandbox directory with input files
- run program
- compare program output, exit code, and output files
- remove sandbox
Files
- nihtest.conf
nihtest configuration file, see nihtest.conf(5) for details
Exit Status
nihtest uses the following exit codes:
- 0
Test passed
- 1
Test failed
- 2
Other error
- 77
Test was skipped.
Diagnostics
In general, differences to the test case expectation are marked up with “-” if something expected did not happen and “+” if something unexpected happened.
In case the return value does not match, nihtest will report “Exit code not as expected”. If the standard output or standard error output are not correct, nihtest will report “Output not as expected” or “Error output not as expected” respectively. And finally, if the file contents of the sandbox are not correct, nihtest will report “Files in sandbox not as expected”.