beakerlib-testing - Man Page
testing — asserting functions, watchdog and report
Description
This file contains functions related directly to testing. These functions are various asserts affecting final result of the phase. Watchdog and the report result function is included as well.
Functions
Manual Asserts
rlPass
Manual assertion, asserts and logs PASS.
rlPass comment
- comment
Short test summary.
Returns 0 and asserts PASS.
rlFail
Manual assertion, asserts and logs FAIL.
rlFail comment
- comment
Short test summary.
Returns 1 and asserts FAIL.
Arithmetic Asserts
rlAssert0
Assertion checking for the equality of parameter to zero.
rlAssert0 comment value
- comment
Short test summary, e.g. “Test if compilation ended successfully”.
- value
Integer value (usually return code of a command).
Returns 0 and asserts PASS when value == 0
.
rlAssertEquals
Assertion checking for the equality of two parameters.
rlAssertEquals comment value1 value2
- comment
Short test summary, e.g. “Test if all 3 packages have been downloaded”.
- value1
First parameter to compare, can be a number or a string
- value2
Second parameter to compare, can be a number or a string
Returns 0 and asserts PASS when value1 == value2
.
rlAssertNotEquals
Assertion checking for the non-equality of two parameters.
rlAssertNotEquals comment value1 value2
- comment
Short test summary, e.g. “Test if return code is not 139”.
- value1
First parameter to compare, can be a number or a string
- value2
Second parameter to compare, can be a number or a string
Returns 0 and asserts PASS when value1 != value2
.
rlAssertGreater
Assertion checking whether first parameter is greater than the second one.
rlAssertGreater comment value1 value2
- comment
Short test summary, e.g. “Test whether there are running more instances of program.”
- value1
Integer value.
- value2
Integer value.
Returns 0 and asserts PASS when value1 > value2
.
rlAssertGreaterOrEqual
Assertion checking whether first parameter is greater or equal to the second one.
rlAssertGreaterOrEqual comment value1 value2
- comment
Short test summary (e.g. “There should present at least one...”)
- value1
Integer value.
- value2
Integer value.
Returns 0 and asserts PASS when value1 X= value2
.
rlAssertLesser
Assertion checking whether first parameter is lesser than the second one.
rlAssertLesser comment value1 value2
- comment
Short test summary, e.g. “Test whether there are running more instances of program.”
- value1
Integer value.
- value2
Integer value.
Returns 0 and asserts PASS when value1 < value2
.
rlAssertLesserOrEqual
Assertion checking whether first parameter is lesser or equal to the second one.
rlAssertLesserOrEqual comment value1 value2
- comment
Short test summary (e.g. “There should present at least one...”)
- value1
Integer value.
- value2
Integer value.
Returns 0 and asserts PASS when value1 X= value2
.
File Asserts
rlAssertExists
Assertion checking for the existence of a file or a directory.
rlAssertExists file|directory
- file|directory
Path to the file or directory.
Returns 0 and asserts PASS when file
exists.
rlAssertNotExists
Assertion checking for the non-existence of a file or a directory.
rlAssertNotExists file|directory
- file|directory
Path to the file or directory.
Returns 0 and asserts PASS when file
does not exist.
rlAssertGrep
Assertion checking if the file contains a pattern.
rlAssertGrep pattern file [options]
- pattern
Regular expression to be searched for.
- file
Path to the file.
- options
Optional parameters to be passed to grep, default is
-q
. Can be used to perform case insensitive matches (-i), or using extended (-E) or perl (-P) regular expressions.
Returns 0 and asserts PASS when file
exists and contains given pattern
.
rlAssertNotGrep
Assertion checking that the file does not contain a pattern.
rlAssertNotGrep pattern file [options]
- pattern
Regular expression to be searched for.
- file
Path to the file.
- options
Optional parameters to be passed to grep, default is
-q
. Can be used to perform case insensitive matches (-i), or using extended (-E) or perl (-P) regular expressions.
Returns 0 and asserts PASS when file
exists and does not contain given pattern
.
rlAssertDiffer
Assertion checking that two files differ (are not identical).
rlAssertDiffer file1 file2
- file1
Path to first file1
- file2
Path to second file
Returns 0 and asserts PASS when file1
and file2
differs.
rlAssertNotDiffer
Assertion checking that two files do not differ (are identical).
rlAssertNotDiffer file1 file2
- file1
Path to first file1
- file2
Path to second file
Returns 0 and asserts PASS when file1
and file2
do not differ.
Run, Watch, Report
rlRun
Run command with optional comment and make sure its exit code matches expectations.
rlRun [-t] [-l] [-c] [-s] command [status[,status...] [comment]]
- -t
If specified, stdout and stderr of the command output will be tagged with strings 'STDOUT: ' and 'STDERR: '.
- -l
If specified, output of the command (tagged, if -t was specified) is logged using rlLog function. This is intended for short outputs, and therefore only last 50 lines are logged this way. Longer outputs should be analysed separately, or uploaded via rlFileSubmit or rlBundleLogs.
- -c
Same as
-l
, but only log the command output if it failed.- -s
Store stdout and stderr to a file (mixed together, as the user would see it on a terminal) and set
$rlRun_LOG
variable to name of the file. Caller is responsible for removing the file. When -t option is used, the content of the file becomes tagged too.If the -s option is not used,
$rlRun_LOG
is not modified and keeps its content from the last “rlRun -s”.- command
Command to run.
- status
Expected exit code(s). Optional, default 0. If you expect more exit codes, separate them with comma (e.g. “0,1” when both 0 and 1 are OK and expected), or use from-to notation (i.e. “2-5” for “2,3,4,5”), or combine them (e.g. “2-4,26” for “2,3,4,26”).
- comment
Short summary describing the action (optional, but recommended - explain what you are doing here).
Returns the exit code of the command run. Asserts PASS when command\'s exit status is in the list of expected exit codes.
Note:
- The output of rlRun is buffered when using
-t
,-l
or-s
option (they use unix pipes, which are buffered by nature). If you need an unbuffered output just make sure thatexpect
package is installed on your system (its “unbuffer” tool will automatically be used to produce unbuffered output). - Be aware that there are some variables which can collide with your code executed within rlRun. You should avoid using __INTERNAL_rlRun_* variables.
When any of
-t -l
,-c
, or-s
option is used, special file descriptors 111 and 112 are used to avoid the issue with incomplete log file, bz1361246. As there might be an indefinite loop, there's a timeout of two minutes implemented as a fix for bz1416796. Also an error message is issued to signal the possibility of running subprocess which keeps the file descriptors open.Do not use these options if you expect process forking and continuouse run. Try your own apropriate solution instead.
Warning: using unbuffer
tool is now disabled because of bug 547686.
rlWatchdog
Run command
. If it does not finish in specified time, then kill it using signal
. Note that the command is executed in a sub-shell so modified environment (e.g. set variables) is not relfected in the test environment.
rlWatchdog command timeout [signal] [callback]
- command
Command to run.
- timeout
Timeout to wait, in seconds.
- signal
Signal to use (optional, default KILL).
- callback
Callback function to be called before the signal is send (optional, none by default). The callback function will have one argument available — PGID of the process group.
Returns 0 if the command ends normally, without need to be killed.
rlReport
Report test result to the test harness. The command to be used for reporting is set by the respective plugin. You can also use the $BEAKERLIB_COMMAND_REPORT_RESULT
variable to use your custom command.
rlReport name result [score] [log]
- name
Name of the test result.
- result
Result (one of PASS, WARN, FAIL). If called with something else, will use WARN.
- score
Test score (optional).
- log
Optional log file to be submitted instead of default
OUTPUTFILE
.
rlCmpVersion
Compare two given versions composed by numbers and letters divided by dot (.), underscore (_), or dash (-).
rlCmpVersion ver1 ver2
If ver1 = ver2, sign '=' is printed to stdout and 0 is returned. If ver1 > ver2, sign '>' is printed to stdout and 1 is returned. If ver1 < ver2, sign '<' is printed to stdout and 2 is returned.
rlTestVersion
Test releation between two given versions based on given operator.
rlTestVersion ver1 op ver2
- op
Operator defining the logical expression. It can be '=', '==', '!=', '<', '<=', '=<', '>', '>=', or '=>'.
Returns 0 if the expresison ver1 op ver2 is true; 1 if the expression is false and 2 if something went wrong.
Release Info
rlIsRHEL
rlIsRHEL [Num|opNum]
- Num
When used function returns 0 if the particular RHEL version is running. Multiple arguments can be passed separated with space as well as any particular release (5.1 5.2 5.3).
- opNum
Argument consisting of operator and number written together as one string. Operator can be '<', '<=', '=<', '=', '>', '>=' matching whether the currently installed version is lesser, lesser or equal, equal, equal or greater, greater than the version number supplied as second half of the argument. Note that ie. '=5' (unlike just '5') matches exactly 5 (5.0), not 5.N, where N > 0. Also note when that using >/< operators you have to either put the argument in quotes or escape the operators to avoid them being interpreted as bash redirection operator.
Returns 0 when we're running on RHEL.
Note that
rlIsRHEL '<6.9' || rlIsRHEL '<7.5'
would also cover 6.10 as it is less than 7.5, which is not what you want. So if you want to construct a condition for rhel<6.9 for rhel6 or rhel<7.5 for rhel7 you actually need to use following construct:
rlIsRHEL 6 && rlIsRHEL '<6.9' || rlIsRHEL 7 && rlIsRHEL '<7.5'
Prototype:
rlIsRHEL
Returns 0 if we are running on RHEL.
rlIsRHEL 4.8 5
Returns 0 if we are running RHEL 4.8 or any RHEL 5.
rlIsRHEL ">=6" or rlIsRHEL \>=6
Returns 0 if we are running RHEL 6 or higher.
rlIsFedora
rlIsFedora [Num|opNum]
Returns 0 when we're running on Fedora. With given number of version as parameter returns 0 if the particular Fedora version is running. Range matching can be used in the form used by rlIsRHEL.
rlIsFedora
Returns 0 if we are running on Fedora.
rlIsFedora 9 10
Returns 0 if we are running Fedora 9 or 10.
Release Info
rlIsCentOS
rlIsCentOS [Num|opNum]
Returns 0 when we're running on CentOS. With given number of version as parameter returns 0 if the particular CentOS version is running. Range matching can be used in the form used by rlIsRHEL.
rlIsCentOS
Returns 0 if we are running on CentOS.
rlIsCentOS 7.1 6
Returns 0 if we are running CentOS 7.1 or any CentOS 6.
Authors
- Ondrej Hudlicky <ohudlick@redhat.com>
- Petr Muller <pmuller@redhat.com>
- Jan Hutar <jhutar@redhat.com>
- Petr Splichal <psplicha@redhat.com>
- Ales Zelinka <azelinka@redhat.com>