nfstest.test_util - Man Page

Test utilities module

Description

Provides a set of tools for testing either the NFS client or the NFS server, most of the functionality is focused mainly on testing the client. These tools include the following:

   - Process command line arguments
   - Provide functionality for PASS/FAIL
   - Provide test grouping functionality
   - Provide multiple client support
   - Logging mechanism
   - Debug info control
   - Mount/Unmount control
   - Create files/directories
   - Provide mechanism to start a packet trace
   - Provide mechanism to simulate a network partition
   - Support for pNFS testing

In order to use some of the functionality available, the user id in all the client hosts must have access to run commands as root using the 'sudo' command without the need for a password, this includes the host where the test is being executed. This is used to run commands like 'mount' and 'umount'. Furthermore, the user id must be able to ssh to remote hosts without the need for a password if test requires the use of multiple clients.

Network partition is simulated by the use of 'iptables', please be advised that after every test run the iptables is flushed and reset so any rules previously setup will be lost. Currently, there is no mechanism to restore the iptables rules to their original state.

Classes

class TestUtil(nfstest.nfs_util.NFSUtil)

TestUtil object

TestUtil() -> New server object

Usage:
    x = TestUtil()

    # Process command line options
    x.scan_options()

    # Start packet trace using tcpdump
    x.trace_start()

    # Mount volume
    x.mount()

    # Create file
    x.create_file()

    # Unmount volume
    x.umount()

    # Stop packet trace
    x.trace_stop()

    # Exit script
    x.exit()


Methods defined here:
---------------------

__del__(self)
Destructor

Gracefully stop the packet trace, cleanup files, unmount volume,
and reset network.

__init__(self, **kwargs)
Constructor

Initialize object's private data.

        sid:
    Test script ID [default: '']
    This is used to have options targeted for a given ID without
    including these options in any other test script.
        usage:
    Usage string [default: '']
        testnames:
    List of test names [default: []]
    When this list is not empty, the --runtest option is enabled and
    test scripts should use the run_tests() method to run all the
    tests. Test script should have methods named as <testname>_test.
        testgroups:
    Dictionary of test groups where the key is the name of the test
    group and its value is a dictionary having the following keys:
    tests:
    A list of tests belonging to this test group
    desc:
    Description of the test group, this is displayed
    in the help if the name of the test group is also
    included in testnames
    tincl:
    Include a comma separated list of tests belonging to
    this test group to the description [default: False]
    wrap:
    Reformat the description so it fits in lines no
    more than the width given. The description is not
    formatted for a value of zero [default: 72]

Example:
    x = TestUtil(testnames=['basic', 'lock'])

    # The following methods should exist:
    x.basic_test()
    x.lock_test()

cleanup(self) Clean up test environment.

Remove any files created: test files, trace files.

close_files(self, *fdlist) Close all files opened by open_files() and all file descriptors given as arguments.

compare_data(self, data, offset=0, pattern=None, nlen=32, fd=None, msg='') Compare data to the given pattern and return a three item tuple: absolute offset where data differs from pattern, sample data at diff offset, and the expected data at diff offset according to pattern. If data matches exactly it returns (None, None, None).

data:

Data to compare against the pattern

offset:

Absolute offset to get the expected data from pattern [default: 0]

pattern:

Data pattern function or string. If this is a function, it must take offset and size as positional arguments. If given as a string, the pattern repeats over and over starting at offset = 0 [default: self.data_pattern]

nlen:

Size of sample data to return if a difference is found [default: 32]

fd:

Opened file descriptor for the data, this is used where the data comes from a file and a difference is found right at the end of the given data. In this case, the data is read from the file to return the sample diff of size given by nlen [default: None]

msg:

Message to append to debug message if a difference is found. If set to None, debug messages are not displayed [default: '']

compare_mount_args(self, mtopts1, mtopts2) Compare mount arguments

config(self, msg) Display config message and terminate test with an exit value of 2.

create_dir(self, dir=None, mode=493) Create a directory under the given directory with the given mode.

create_file(self, offset=0, size=None, dir=None, mode=None, **kwds) Create a file starting to write at given offset with total size of written data given by the size option.

offset:

File offset where data will be written to [default: 0]

size:

Total number of bytes to write [default: --filesize option]

dir:

Create file under this directory

mode:

File permissions [default: use default OS permissions]

pattern:

Data pattern to write to the file [default: data_pattern default]

ftype:

File type to create [default: FTYPE_FILE]

hole_list:

List of offsets where each hole is located [default: None]

hole_size:

Size of each hole [default: --wsize option]

verbose:

Verbosity level [default: 0]

dlevels:

Debug level list to use [default: ["DBG2", "DBG3", "DBG4"]]

Returns the file name created, the file name is also stored in the object attribute filename -- attribute absfile is also available as the absolute path of the file just created.

File created is removed at cleanup.

create_rexec(self, servername=None, **kwds) Create remote server object.

data_pattern(self, offset, size, pattern=None) Return data pattern.

offset:

Starting offset of pattern

size:

Size of data to return

pattern:

Data pattern to return, default is of the form: hex_offset(0x%08X) abcdefghijklmnopqrstn

delay_io(self, delay=None) Delay I/O by value given or the value given in --iodelay option.

exit(self) Terminate script with an exit value of 0 when all tests passed and a value of 1 when there is at least one test failure.

get_dirname(self, dir=None) Return a unique directory name under the given directory.

get_filename(self, dir=None) Return a unique file name under the given directory.

get_logname(self, remote=False) Get next log file name.

get_marker_index(self, marker_id=None) Find packet index of the trace marker given by the marker id

marker_id:

ID of trace marker to find in the packet trace, if this is not given the current marker id is used [default: None]

get_name(self) Get unique name for this instance.

insert_trace_marker(self, name=None) Send a LOOKUP for an unknown file to have a marker in the packet trace and return the trace marker id

name:

Use this name as the trace marker but the caller must make sure this is a unique name in order to find the correct index for this marker. This could also be used to add any arbitrary information to the packet trace [default: None]

lock_files(self, lock_type=None, offset=0, length=0) Lock all files opened by open_files().

need_run_test(self, testname) Return True only if user explicitly requested to run this test

open_files(self, mode, create=True) Open files according to given mode, the file descriptors are saved internally to be used with write_files(), read_files() and close_files(). The number of files to open is controlled by the command line option '--nfiles'.

The mode could be either 'r' or 'w' for opening files for reading or writing respectively. The open flags for mode 'r' is O_RDONLY while for mode 'w' is O_WRONLY|O_CREAT|O_SYNC. The O_SYNC is used to avoid the client buffering the written data.

process_client_option(self, option='client', remote=True, count=1) Process the client option

Clients are separated by a "," and each client definition can have the following options separated by ":":
   client:server:export:nfsversion:port:proto:sec:mtpoint

option:

Option name [default: "client"]

remote:

Expect a client hostname or IP address in the definition. If this is set to None do not verify client name or IP. [default: True]

count:

Number of client definitions to expect. If remote is True, return the number of definitions listed in the given option up to this number. If remote is False, return exactly this number of definitions [default: 1]

Examples:
   # Using positional arguments with nfsversion=4.1 for client1
   client=client1:::4.1,client2

   # Using named arguments instead
   client=client1:nfsversion=4.1,client2

process_option(self, value, arglist=[], typemap={}) Process option with a list of items separated by "," and each item in the list could have different arguments separated by ":".

value:

String of comma separated elements

arglist:

Positional order of arguments, if this list is empty, then use named arguments only [default: []]

typemap:

Dictionary to convert arguments to their given types, where the key is the argument name and its value is the type function to use to convert the argument [default: {}]

read_files(self) Read a block of data (size given by --rsize) from all files opened by open_files() for reading.

remove_test(self, testname) Remove all instances of test from the list of tests to run

run_func(self, func, *args, **kwargs) Run function with the given arguments and return the results. All positional arguments are passed to the function while the named arguments change the behavior of the method. Object attribute "oserror" is set to the OSError object if the function fails.

msg:

Test assertion message [default: None]

err:

Expected error number [default: 0]

run_tests(self, **kwargs) Run all test specified by the --runtest option.

testnames:

List of testnames to run [default: all tests given by --testnames]

All other arguments given are passed to the test methods.

scan_options(self) Process command line options.

Process all the options in the file given by '--file', then the ones in the command line. This allows for command line options to over write options given in the file.

Format of options file:
   # For options expecting a value
   <option_name> = <value>

   # For boolean (flag) options
   <option_name>

Process options files and make sure not to process the same file twice, this is used for the case where HOMECFG and CWDCFG are the same, more specifically when environment variable HOME is not defined. Also, the precedence order is defined as follows:
 1. options given in command line
 2. options given in file specified by the -f|--file option
 3. options given in file specified by ./.nfstest
 4. options given in file specified by $HOME/.nfstest
 5. options given in file specified by /etc/nfstest

NOTE:
 Must use the long name of the option (--<option_name>) in the file.

set_nfserr_list(self, nfs3list=[], nfs4list=[], nlm4list=[], mnt3list=[]) Temporaly set the NFS list of expected NFS errors in the next call to trace_open

setup(self, nfiles=None) Set up test environment.

Create nfiles number of files [default: --nfiles option]

str_args(self, args) Return the formal string representation of the given list where string objects are truncated.

test(self, expr, msg, subtest=None, failmsg=None, terminate=False) Test expr and display message as PASS/FAIL, terminate execution if terminate option is True.

expr:

If expr is true, display as a PASS message, otherwise as a FAIL message

msg:

Message to display

subtest:

If given, append this string to the displayed message and mark this test as a member of the sub-group given by msg

failmsg:

If given, append this string to the displayed message when expr is false [default: None]

terminate:

Terminate execution if true and expr is false [default: False]

If tverbose=normal or level 1:
   Sub-group message is displayed as a PASS/FAIL message including
   the number of tests that passed and failed within the sub-group If tverbose=verbose or level 2:
   All tests messages are displayed

test_description(self, tname=None) Return the test description for the current test

test_group(self, msg) Display heading message and start a test group.

If tverbose=group or level 0:
   Group message is displayed as a PASS/FAIL message including the
   number of tests that passed and failed within this test group. If tverbose=normal|verbose or level 1|2:
   Group message is displayed as a heading messages for the tests
   belonging to this test group.

test_info(self, msg) Display info message.

test_options(self, name=None) Get options for the given test name. If the test name is not given it is determined by inspecting the stack to find which method is calling this method.

testid_count(self, tid) Return the number of instances the testid has occurred.

trace_open(self, *kwts, **kwds) This is a wrapper to the original trace_open method where the packet trace is scanned for NFS errors and a failure is logged for each error found not given on the list of expected errors set with method set_nfserr_list. Scanning for NFS error is done only if --nfserrors option has been specified.

trace_start(self, *kwts, **kwds) This is a wrapper to the original trace_start method to reset the trace marker state

verify_client_option(self, tclient_dict, option='client') Verify the client option is required from the list of tests to run. Also, check if enough clients were specified to run the tests.

tclient_dict:

Dictionary having the number of clients required by each test

option:

Option name [default: "client"]

verify_file_data(self, msg=None, pattern=None, path=None, filesize=None, nlen=None, cmsg='') Verify file by comparing the data to the given pattern. It returns the results from the compare_data method.

msg:

Test assertion message. If set to None, no assertion is done it just returns the results [default: None]

pattern:

Data pattern function or string. If this is a function, it must take offset and size as positional arguments. If given as a string, the pattern repeats over and over starting at offset = 0 [default: self.data_pattern]

path:

Absolute path of file to verify [default: self.absfile]

filesize:

Expected size of file to be verified [default: self.filesize]

nlen:

Size of sample data to return if a difference is found [default: compare_data default]

cmsg:

Message to append to debug message if a difference is found. If set to None, debug messages are not displayed [default: '']

warning(self, msg) Display warning message.

write_data(self, fd, offset=0, size=None, pattern=None, verbose=0, dlevel='DBG5') Write data to the file given by the file descriptor

fd:

File descriptor

offset:

File offset where data will be written to [default: 0]

size:

Total number of bytes to write [default: --filesize option]

pattern:

Data pattern to write to the file [default: data_pattern default]

verbose:

Verbosity level [default: 0]

write_files(self) Write a block of data (size given by --wsize) to all files opened by open_files() for writing.

Static methods defined here: ----------------------------

get_list(value, nmap, sep=',') Given the value as a string of 'comma' separated elements, return a list where each element is mapped using the dictionary 'nmap'.
   nmap = {"one":1, "two":2}
   out = x.get_list("one", nmap)        # out = [1]
   out = x.get_list("one,two", nmap)    # out = [1,2]
   out = x.get_list("two,one", nmap)    # out = [2,1]
   out = x.get_list("one,three", nmap)  # out = None

str_list(value, vtype=<class 'str'>, sep=',') Return a list of <vtype> elements from the comma separated string.

See Also

baseobj(3), formatstr(3), nfstest.host(3), nfstest.nfs_util(3), nfstest.rexec(3), nfstest.utils(3), packet.nfs.nfs3_const(3), packet.nfs.nfs4_const(3)

Bugs

No known bugs.

Author

Jorge Mora (mora@netapp.com)

Referenced By

nfstest_alloc(1), nfstest_cache(1), nfstest_delegation(1), nfstest_dio(1), nfstest_fcmp(1), nfstest_interop(1), nfstest_lock(1), nfstest_pnfs(1), nfstest_posix(1), nfstest_rdma(1), nfstest_sparse(1), nfstest_ssc(1), nfstest_xattr(1).

21 March 2023 NFStest 3.2 test_util 1.8