multipath.conf man page

multipath.conf — multipath daemon configuration file.

Description

/etc/multipath.conf is the configuration file for the multipath daemon. It is used to overwrite the built-in configuration table of multipathd. Any line whose first non-white-space character is a '#' is considered a comment line. Empty lines are ignored.

Syntax

The configuration file contains entries of the form:

<section> {

<attribute> <value> ...<subsection> {

<attribute> <value> ...

}

}

Each section contains one or more attributes or subsections. The recognized keywords for attributes or subsections depend on the section in which they occur.

The following section keywords are recognized:

defaults

This section defines default values for attributes which are used whenever no values are given in the appropriate device or multipath sections.

blacklist

This section defines which devices should be excluded from the multipath topology discovery.

blacklist_exceptions

This section defines which devices should be included in the multipath topology discovery, despite being listed in the blacklist section.

multipaths

This section defines the multipath topologies. They are indexed by a World Wide Identifier(WWID). For details on the WWID generation see section WWID generation below.

devices

This section defines the device-specific settings.

overrides

This section defines values for attributes that should override the device-specific settings for all devices.

defaults section

The defaults section recognizes the following keywords:

verbosity

Default verbosity. Higher values increase the verbosity level. Valid levels are between 0 and 6.

The default is: 2

polling_interval

Interval between two path checks in seconds. For properly functioning paths, the interval between checks will gradually increase to max_polling_interval. This value will be overridden by the WatchdogSec setting in the multipathd.service definition if systemd is used.

The default is: 5

max_polling_interval

Maximal interval between two path checks in seconds.

The default is: 4 * polling_interval

reassign_maps

Enable reassigning of device-mapper maps. With this option multipathd will remap existing device-mapper maps to always point to multipath device, not the underlying block devices. Possible values are yes and no.

The default is: no

multipath_dir

Directory where the dynamic shared objects are stored. Defined at compile time, commonly /lib64/multipath/ or /lib/multipath/.

The default is: <system dependent>

path_selector

The default path selector algorithm to use; they are offered by the kernel multipath target. There are three selector algorithms:

round-robin 0

Loop through every path in the path group, sending the same amount of I/O to each. Some aspects of behavior can be controlled with the attributes: rr_min_io, rr_min_io_rq and rr_weight.

queue-length 0

(Since 2.6.31 kernel) Choose the path for the next bunch of I/O based on the amount of outstanding I/O to the path.

service-time 0

(Since 2.6.31 kernel) Choose the path for the next bunch of I/O based on the amount of outstanding I/O to the path and its relative throughput.

The default is: service-time 0

path_grouping_policy

The default path grouping policy to apply to unspecified multipaths. Possible values are:

failover

One path per priority group.

multibus

All paths in one priority group.

group_by_serial

One priority group per serial number.

group_by_prio

One priority group per priority value. Priorities are determined by callout programs specified as a global, per-controller or per-multipath option in the configuration file.

group_by_node_name

One priority group per target node name. Target node names are fetched in /sys/class/fc_transport/target*/node_name.

The default is: failover

uid_attrs

The udev attribute providing a unique path identifier for corresponding type of path devices. If this field is configured and matched with type of device, it would override any other methods providing for device unique identifier in config file, and it would activate merging uevents according to the identifier to promote effiecncy in processing uevents. It has no default value, if you want to identify path by udev attribute and to activate merging uevents for SCSI and DASD devices, you can set its value as:

uid_attrs "sd:ID_SERIAL dasd:ID_UID"

The default is: <unset>

uid_attribute

The udev attribute providing a unique path identifier.

The default is: for SCSI devices ID_SERIAL

The default is: for DASD devices ID_UID

The default is: for NVME devices ID_WWN

getuid_callout

(Superseded by uid_attribute) The default program and args to callout to obtain a unique path identifier. Should be specified with an absolute path.

prio

The name of the path priority routine. The specified routine should return a numeric value specifying the relative priority of this path. Higher number have a higher priority. "none" is a valid value. Currently the following path priority routines are implemented:

const

Return a constant priority of 1.

sysfs

Use the sysfs attributes access_state and preferred_path to generate the path priority. This prioritizer accepts the optional prio_arg exclusive_pref_bit

emc

(Hardware-dependent) Generate the path priority for DGC class arrays as CLARiiON CX/AX and EMC VNX and Unity families.

alua

(Hardware-dependent) Generate the path priority based on the SCSI-3 ALUA settings. This prioritizer accepts the optional prio_arg exclusive_pref_bit.

ontap

(Hardware-dependent) Generate the path priority for NetApp ONTAP class and OEM arrays as IBM NSeries.

rdac

(Hardware-dependent) Generate the path priority for LSI/Engenio/NetApp RDAC class as NetApp SANtricity E/EF Series, and OEM arrays from IBM DELL SGI STK and SUN.

hp_sw

(Hardware-dependent) Generate the path priority for HP/COMPAQ/DEC HSG80 and MSA/HSV arrays with Active/Standby mode exclusively.

hds

(Hardware-dependent) Generate the path priority for Hitachi AMS 2000 and HUS 100 families of arrays.

random

Generate a random priority between 1 and 10.

weightedpath

Generate the path priority based on the regular expression and the priority provided as argument. Requires prio_args keyword.

path_latency

Generate the path priority based on a latency algorithm. Requires prio_args keyword.

datacore

???. Requires prio_args keyword.

iet

???. Requires prio_args keyword.

The default is: const

prio_args

Arguments to pass to to the prio function. This only applies to certain prioritizers:

weighted

Needs a value of the form "<hbtl|devname|serial|wwn> <regex1> <prio1> <regex2> <prio2> ..."

hbtl

regex can be of SCSI H:B:T:L format. For example: 1:0:.:. , *:0:0:.

devname

regex can be of device name format. For example: sda , sd.e

serial

regex can be of serial number format. For example: .*J1FR.*324 . The serial can be looked up through sysfs or by running multipathd show paths format "%z". For example: 0395J1FR904324

wwn

regex can be of the form "host_wwnn:host_wwpn:target_wwnn:target_wwpn" these values can be looked up through sysfs or by running multipathd show paths format "%N:%R:%n:%r". For example: 0x200100e08ba0aea0:0x210100e08ba0aea0:.*:.* , .*:.*:iqn.2009-10.com.redhat.msp.lab.ask-06:.*

path_latency

Needs a value of the form "<io_num>|<base_num>"

io_num

The number of read IOs sent to the current path continuously, used to calculate the average path latency. Valid Values: Integer, [2, 200].

base_num

The base number value of logarithmic scale, used to partition different priority ranks. Valid Values: Integer, [2, 10]. And Max average latency value is 100s, min average latency value is 1us. For example: If base_num=10, the paths will be grouped in priority groups with path latency <=1us, (1us, 10us], (10us, 100us], (100us, 1ms], (1ms, 10ms], (10ms, 100ms], (100ms, 1s], (1s, 10s], (10s, 100s], >100s.

alua

If exclusive_pref_bit is set, paths with the preferred path bit set will always be in their own path group.

sysfs

If exclusive_pref_bit is set, paths with the preferred path bit set will always be in their own path group.

datacore

preferredsds ???.

iet

preferredip ???.

The default is: <unset>

features

Specify any device-mapper features to be used. Syntax is num list where num is the number, between 0 and 8, of features in list. Possible values for the feature list are:

queue_if_no_path

(Superseded by no_path_retry) (Since ??? kernel) Queue I/O if no path is active. Identical to the no_path_retry with queue value. See Known Issues.

no_partitions

Disable automatic partitions generation via kpartx.

pg_init_retries <times>

(Since ??? kernel) Number of times to retry pg_init, it must be between 1 and 50.

pg_init_delay_msecs <msecs>

(Since ??? kernel) Number of msecs before pg_init retry, it must be between 0 and 60000.

queue_mode <mode>

(Since ??? kernel) Select the the queue_mode per multipath device. Where <mode> can be bio, rq or mq. Which corresponds to bio-based, request_fn rq-based, and blk-mq rq-based respectively.

The default is: 0

path_checker

The default method used to determine the paths state. Possible values are:

readsector0

(Deprecated) Read the first sector of the device. This checker is being deprecated, please use tur instead.

tur

Issue a TEST UNIT READY command to the device.

emc_clariion

(Hardware-dependent) Query the DGC/EMC specific EVPD page 0xC0 to determine the path state for CLARiiON CX/AX and EMC VNX and Unity arrays families.

hp_sw

(Hardware-dependent) Check the path state for HP/COMPAQ/DEC HSG80 and MSA/HSV arrays with Active/Standby mode exclusively.

rdac

(Hardware-dependent) Check the path state for LSI/Engenio/NetApp RDAC class as NetApp SANtricity E/EF Series, and OEM arrays from IBM DELL SGI STK and SUN.

directio

(Deprecated) Read the first sector with direct I/O. This checker is being deprecated, it could cause spurious path failures under high load. Please use tur instead.

cciss_tur

(Hardware-dependent) Check the path state for HP/COMPAQ Smart Array(CCISS) controllers.

rbd

Check if the path is in the Ceph blacklist and remap the path if it is.

The default is: tur

alias_prefix

The user_friendly_names prefix.

The default is: mpath

failback

Tell multipathd how to manage path group failback. To select immediate or a value, it's mandatory that the device has support for a working prioritizer.

immediate

Immediately failback to the highest priority pathgroup that contains active paths.

manual

Do not perform automatic failback.

followover

Used to deal with multiple computers accessing the same Active/Passive storage devices. Only perform automatic failback when the first path of a pathgroup becomes active. This keeps a cluster node from automatically failing back when another node requested the failover.

values > 0

Deferred failback (time to defer in seconds).

The default is: manual

rr_min_io

Number of I/O requests to route to a path before switching to the next in the same path group. This is only for Block I/O(BIO) based multipath and only apply to round-robin path_selector.

The default is: 1000

rr_min_io_rq

Number of I/O requests to route to a path before switching to the next in the same path group. This is only for Request based multipath and only apply to round-robin path_selector.

The default is: 1

max_fds

Specify the maximum number of file descriptors that can be opened by multipath and multipathd. This is equivalent to ulimit -n. A value of max will set this to the system limit from /proc/sys/fs/nr_open. If this is not set, the maximum number of open fds is taken from the calling process. It is usually 1024. To be safe, this should be set to the maximum number of paths plus 32, if that number is greated than 1024.

The default is: max

rr_weight

If set to priorities the multipath configurator will assign path weights as "path prio * rr_min_io". Possible values are priorities or uniform . Only apply to round-robin path_selector.

The default is: uniform

no_path_retry

Specify what to do when all paths are down. Possible values are:

value > 0

number of retries until disable I/O queueing.

fail

for immediate failure (no I/O queueing).

queue

for never stop I/O queueing. Similar to queue_if_no_path.

See Known Issues.

The default is: fail

queue_without_daemon

If set to no , when multipathd stops, queueing will be turned off for all devices. This is useful for devices that set no_path_retry. If a machine is shut down while all paths to a device are down, it is possible to hang waiting for I/O to return from the device after multipathd has been stopped. Without multipathd running, access to the paths cannot be restored, and the kernel cannot be told to stop queueing I/O. Setting queue_without_daemon to no , avoids this problem.

The default is: no

checker_timeout

Specify the timeout to use for path checkers and prioritizers that issue SCSI commands with an explicit timeout, in seconds.

The default is: in /sys/block/sd<x>/device/timeout

flush_on_last_del

If set to yes , multipathd will disable queueing when the last path to a device has been deleted.

The default is: no

user_friendly_names

If set to yes , using the bindings file /etc/multipath/bindings to assign a persistent and unique alias to the multipath, in the form of mpath<n>. If set to no use the WWID as the alias. In either case this be will be overridden by any specific aliases in the multipaths section.

The default is: no

fast_io_fail_tmo

Specify the number of seconds the SCSI layer will wait after a problem has been detected on a FC remote port before failing I/O to devices on that remote port. This should be smaller than dev_loss_tmo. Setting this to off will disable the timeout.

The default is: in 5

dev_loss_tmo

Specify the number of seconds the SCSI layer will wait after a problem has been detected on a FC remote port before removing it from the system. This can be set to "infinity" which sets it to the max value of 2147483647 seconds, or 68 years. It will be automatically adjusted to the overall retry interval no_path_retry * polling_interval if a number of retries is given with no_path_retry and the overall retry interval is longer than the specified dev_loss_tmo value. The Linux kernel will cap this value to 300 if fast_io_fail_tmo is not set. See Known Issues.

The default is: 600

bindings_file

The full pathname of the binding file to be used when the user_friendly_names option is set.

The default is: /etc/multipath/bindings

wwids_file

The full pathname of the WWIDs file, which is used by multipath to keep track of the WWIDs for LUNs it has created multipath devices on in the past.

The default is: /etc/multipath/wwids

log_checker_err

If set to once , multipathd logs the first path checker error at logging level 2. Any later errors are logged at level 3 until the device is restored. If set to always , multipathd always logs the path checker error at logging level 2.

The default is: always

reservation_key

This is the service action reservation key used by mpathpersist. It must be set for all multipath devices using persistent reservations, and it must be the same as the RESERVATION KEY field of the PERSISTENT RESERVE OUT parameter list which contains an 8-byte value provided by the application client to the device server to identify the I_T nexus.

The default is: <unset>

retain_attached_hw_handler

If set to yes and the SCSI layer has already attached a hardware_handler to the device, multipath will not force the device to use the hardware_handler specified by mutipath.conf. If the SCSI layer has not attached a hardware handler, multipath will continue to use its configured hardware handler.

The default is: yes

detect_prio

If set to yes , multipath will try to detect if the device supports SCSI-3 ALUA. If so, the device will automatically use the sysfs prioritizer if the required sysf attributes access_state and preferred_path are supported, or the alua prioritizer if not. If set to no , the prioritizer will be selected as usual.

The default is: yes

detect_checker

if set to yes , multipath will try to detect if the device supports SCSI-3 ALUA. If so, the device will automatically use the tur checker. If set to no , the checker will be selected as usual.

The default is: yes

force_sync

If set to yes , multipathd will call the path checkers in sync mode only.  This means that only one checker will run at a time.  This is useful in the case where many multipathd checkers running in parallel causes significant CPU pressure.

The default is: no

strict_timing

If set to yes , multipathd will start a new path checker loop after exactly one second, so that each path check will occur at exactly polling_interval seconds. On busy systems path checks might take longer than one second; here the missing ticks will be accounted for on the next round. A warning will be printed if path checks take longer than polling_interval seconds.

The default is: no

deferred_remove

If set to yes , multipathd will do a deferred remove instead of a regular remove when the last path device has been deleted.  This means that if the multipath device is still in use, it will be freed when the last user closes it.  If path is added to the multipath device before the last user closes it, the deferred remove will be canceled.

The default is: no

partition_delimiter

If this value is not set, when multipath renames a device, it will act just like the kpartx default does, only adding a "p" to names ending in a number. If this parameter is set, multipath will act like kpartx does with the -p option is used, and always add delimiter.

The default is: <unset>

config_dir

If set to anything other than "", multipath will search this directory alphabetically for file ending in ".conf" and it will read configuration information from them, just as if it was in /etc/multipath.conf. config_dir must either be "" or a fully qualified directory name.

The default is: /etc/multipath/conf.d/

san_path_err_threshold

If set to a value greater than 0, multipathd will watch paths and check how many times a path has been failed due to errors.If the number of failures on a particular path is greater then the san_path_err_threshold then the path will not  reinstante till san_path_err_recovery_time.These path failures should occur within a san_path_err_forget_rate checks, if not we will consider the path is good enough to reinstantate.

The default is: no

san_path_err_forget_rate

If set to a value greater than 0, multipathd will check whether the path failures has exceeded  the san_path_err_threshold within this many checks i.e san_path_err_forget_rate . If so we will not reinstante the path till san_path_err_recovery_time.

The default is: no

san_path_err_recovery_time

If set to a value greater than 0, multipathd will make sure that when path failures has exceeded the san_path_err_threshold within san_path_err_forget_rate then the path will be placed in failed state for san_path_err_recovery_time duration.Once san_path_err_recovery_time has timeout  we will reinstante the failed path . san_path_err_recovery_time value should be in secs.

The default is: no

delay_watch_checks

If set to a value greater than 0, multipathd will watch paths that have recently become valid for this many checks. If they fail again while they are being watched, when they next become valid, they will not be used until they have stayed up for delay_wait_checks checks.

The default is: no

delay_wait_checks

If set to a value greater than 0, when a device that has recently come back online fails again within delay_watch_checks checks, the next time it comes back online, it will marked and delayed, and not used until it has passed delay_wait_checks checks.

The default is: no

find_multipaths

If set to yes , instead of trying to create a multipath device for every non-blacklisted path, multipath will only create a device if one of three condidions are met. 1 There are at least two non-blacklisted paths with the same WWID, 2 the user manually forces the creation, by specifying a device with the multipath command, or 3 a path has the same WWID as a multipath device that was previously created while find_multipaths was set (even if that multipath device doesn't currently exist). Whenever a multipath device is created with find_multipaths set, multipath will remember the WWID of the device, so that it will automatically create the device again, as soon as it sees a path with that WWID. This should allow most users to have multipath automatically choose the correct paths to make into multipath devices, without having to edit the blacklist.

The default is: no

uxsock_timeout

CLI receive timeout in milliseconds. For larger systems CLI commands might timeout before the multipathd lock is released and the CLI command can be processed. This will result in errors like "timeout receiving packet" to be returned from CLI commands. In these cases it is recommended to increase the CLI timeout to avoid those issues.

The default is: 1000

retrigger_tries

Sets the number of times multipathd will try to retrigger a uevent to get the WWID.

The default is: 3

retrigger_delay

Sets the amount of time, in seconds, to wait between retriggers.

The default is: 10

missing_uev_wait_timeout

Controls how many seconds multipathd will wait, after a new multipath device is created, to receive a change event from udev for the device, before automatically enabling device reloads. Usually multipathd will delay reloads on a device until it receives a change uevent from the initial table load.

The default is: 30

skip_kpartx

If set to yes , kpartx will not automatically create partitions on the device.

The default is: no

disable_changed_wwids

If set to yes, multipathd will check the path wwid on change events, and if it has changed from the wwid of the multipath device, multipathd will disable access to the path until the wwid changes back.

The default is: no

remove_retries

This sets how may times multipath will retry removing a device that is in-use. Between each attempt, multipath will sleep 1 second.

The default is: 0

max_sectors_kb

Sets the max_sectors_kb device parameter on all path devices and the multipath device to the specified value.

The default is: <device dependent>

blacklist section

The blacklist section is used to exclude specific device from inclusion in the multipath topology. It is most commonly used to exclude local disks or LUNs for the array controller.

The following keywords are recognized:

devnode

Regular expression of the device nodes to be excluded.

The default is: ^(ram|raw|loop|fd|md|dm-|sr|scd|st|dcssblk)[0-9] and ^(td|hd|vd)[a-z]

wwid

The World Wide Identification of a device.

property

Regular expression of the udev property to be excluded.

device

Subsection for the device description. This subsection recognizes the vendor and product keywords. For a full description of these keywords please see the devices section description.

blacklist_exceptions section

The blacklist_exceptions section is used to revert the actions of the blacklist section. For example to include specific device in the multipath topology. This allows one to selectively include devices which would normally be excluded via the blacklist section.

The following keywords are recognized:

devnode

Regular expression of the device nodes to be whitelisted.

wwid

The World Wide Identification of a device.

property

Regular expression of the udev property to be whitelisted.

device

Subsection for the device description. This subsection recognizes the vendor and product keywords. For a full description of these keywords please see the devices section description.

The property blacklist and whitelist handling is different from the usual handling in the sense that the whitelist has to be set, otherwise the device will be blacklisted. In these cases the message blacklisted, udev property missing will be displayed.

multipaths section

The only recognized attribute for the multipaths section is the multipath subsection.

The multipath subsection recognizes the following attributes:

wwid

(Mandatory) Index of the container.

alias

Symbolic name for the multipath map.

The following attributes are optional; if not set the default values are taken from the defaults or devices section:

path_grouping_policy

path_selector

prio

prio_args

failback

rr_weight

no_path_retry

rr_min_io

rr_min_io_rq

flush_on_last_del

features

reservation_key

user_friendly_names

deferred_remove

san_path_err_threshold

san_path_err_forget_rate

san_path_err_recovery_time

delay_watch_checks

delay_wait_checks

skip_kpartx

max_sectors_kb

devices section

The only recognized attribute for the devices section is the device subsection.

The device subsection recognizes the following attributes:

vendor, product, revision and product_blacklist are POSIX Extended regex.
vendor

(Mandatory) Vendor identifier.

product

(Mandatory) Product identifier.

revision

Revision identfier.

product_blacklist

Product strings to blacklist for this vendor.

alias_prefix

The user_friendly_names prefix to use for this device type, instead of the default "mpath".

hardware_handler

The hardware handler to use for this device type. The following hardware handler are implemented:

1 emc

(Hardware-dependent) Hardware handler for DGC class arrays as CLARiiON CX/AX and EMC VNX and Unity families.

1 rdac

(Hardware-dependent) Hardware handler for LSI/Engenio/NetApp RDAC class as NetApp SANtricity E/EF Series, and OEM arrays from IBM DELL SGI STK and SUN.

1 hp_sw

(Hardware-dependent) Hardware handler for HP/COMPAQ/DEC HSG80 and MSA/HSV arrays with Active/Standby mode exclusively.

1 alua

(Hardware-dependent) Hardware handler for SCSI-3 ALUA compatible arrays.

The default is: <unset>

The following attributes are optional; if not set the default values are taken from the defaults section:

path_grouping_policy

uid_attribute

path_selector

path_checker

prio

prio_args

features

failback

rr_weight

no_path_retry

rr_min_io

rr_min_io_rq

fast_io_fail_tmo

dev_loss_tmo

flush_on_last_del

retain_attached_hw_handler

detect_prio

detect_checker

deferred_remove

san_path_err_threshold

san_path_err_forget_rate

san_path_err_recovery_time

delay_watch_checks

delay_wait_checks

skip_kpartx

max_sectors_kb

overrides section

The overrides section recognizes the following optional attributes; if not set the values are taken from the devices or defaults sections:

path_grouping_policy

uid_attribute

getuid_callout

path_selector

path_checker

alias_prefix

features

prio

prio_args

failback

rr_weight

no_path_retry

rr_min_io

rr_min_io_rq

flush_on_last_del

fast_io_fail_tmo

dev_loss_tmo

user_friendly_names

retain_attached_hw_handler

detect_prio

detect_checker

deferred_remove

san_path_err_threshold

san_path_err_forget_rate

san_path_err_recovery_time

delay_watch_checks

delay_wait_checks

skip_kpartx

WWID generation

Multipath uses a World Wide Identification (WWID) to determine which paths belong to the same device. Each path presenting the same WWID is assumed to point to the same device.

The WWID is generated by three methods (in the order of preference):

getuid_callout

Use the specified external program; cf getuid_callout above. Care should be taken when using this method; the external program needs to be loaded from disk for execution, which might lead to deadlock situations in an all-paths-down scenario.

uid_attribute

Use the value of the specified udev attribute; cf uid_attribute above. This method is preferred to getuid_callout as multipath does not need to call any external programs here. However, under certain circumstances udev might not be able to generate the requested variable.

vpd_pg83

If none of the getuid_callout or uid_attribute parameters are present multipath will try to use the sysfs attribute vpd_pg83 to generate the WWID.

Known Issues

The usage of queue_if_no_path option can lead to D state processes being hung and not killable in situations where all the paths to the LUN go offline. It is advisable to use the no_path_retry option instead.

The use of queue_if_no_path or no_path_retry might lead to a deadlock if the dev_loss_tmo setting results in a device being removed while I/O is still queued. The multipath daemon will update the dev_loss_tmo setting accordingly to avoid this deadlock. Hence if both values are specified the order of precedence is no_path_retry, queue_if_no_path, dev_loss_tmo.

See Also

udev(8), dmsetup(8), multipath(8), multipathd(8).

Authors

multipath-tools was developed by Christophe Varoqui, <christophe.varoqui@opensvc.com> and others.

Referenced By

mpathconf(8), multipath(8).

2017-05-11 Linux