The DM-Multipath Configuration File

By default, DM-Multipath provides configuration values for the most common uses of multipathing. In addition, DM-Multipath includes support for the most common storage arrays that support DM-Multipath. The default configuration values and the supported devices can be found in the multipath.conf.defaults file.

You can override the default configuration values for DM-Multipath by editing the /etc/multipath.conf configuration file. If necessary, you can also add a storage array that is not supported by default to the configuration file. This chapter provides information on parsing and modifying the multipath.conf file. It contains sections on the following topics:

In the multipath configuration file, you need to specify only the sections that you need for your configuration, or that you wish to change from the default values specified in the multipath.conf.defaults file. If there are sections of the file that are not relevant to your environment or for which you do not need to override the default values, you can leave them commented out, as they are in the initial file.

The configuration file allows regular expression description syntax.

An annotated version of the configuration file can be found in /usr/share/doc/multipath-tools/examples/multipath.conf.annotated.gz.

Configuration File Overview

The multipath configuration file is divided into the following sections:

blacklist

Listing of specific devices that will not be considered for multipath.

blacklist_exceptions

Listing of multipath candidates that would otherwise be blacklisted according to the parameters of the blacklist section.

defaults

General default settings for DM-Multipath.

multipath

Settings for the characteristics of individual multipath devices. These values overwrite what is specified in the defaults and devices sections of the configuration file.

devices

Settings for the individual storage controllers. These values overwrite what is specified in the defaults section of the configuration file. If you are using a storage array that is not supported by default, you may need to create a devices subsection for your array.

When the system determines the attributes of a multipath device, first it checks the multipath settings, then the per devices settings, then the multipath system defaults.

Configuration File Blacklist

The blacklist section of the multipath configuration file specifies the devices that will not be used when the system configures multipath devices. Devices that are blacklisted will not be grouped into a multipath device.

Blacklisting By WWID

You can specify individual devices to blacklist by their World-Wide IDentification with a wwid entry in the blacklist section of the configuration file.

The following example shows the lines in the configuration file that would blacklist a device with a WWID of 26353900f02796769.

blacklist {
       wwid 26353900f02796769
}

Blacklisting By Device Name

You can blacklist device types by device name so that they will not be grouped into a multipath device by specifying a devnode entry in the blacklist section of the configuration file.

The following example shows the lines in the configuration file that would blacklist all SCSI devices, since it blacklists all sd* devices.

blacklist {
       devnode "^sd[a-z]"
}

You can use a devnode entry in the blacklist section of the configuration file to specify individual devices to blacklist rather than all devices of a specific type. This is not recommended, however, since unless it is statically mapped by udev rules, there is no guarantee that a specific device will have the same name on reboot. For example, a device name could change from /dev/sda to /dev/sdb on reboot.

By default, the following devnode entries are compiled in the default blacklist; the devices that these entries blacklist do not generally support DM-Multipath. To enable multipathing on any of these devices, you would need to specify them in the blacklist_exceptions section of the configuration file, as described in Blacklist Exceptions

blacklist {
       devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
       devnode "^hd[a-z]"
}

Blacklisting By Device Type

You can specify specific device types in the blacklist section of the configuration file with a device section. The following example blacklists all IBM DS4200 and HP devices.

blacklist {
       device {
               vendor  "IBM"
               product "3S42"       #DS4200 Product 10
       }
       device {
               vendor  "HP"
               product "*"
       }
}

Blacklist Exceptions

You can use the blacklist_exceptions section of the configuration file to enable multipathing on devices that have been blacklisted by default.

For example, if you have a large number of devices and want to multipath only one of them (with the WWID of 3600d0230000000000e13955cc3757803), instead of individually blacklisting each of the devices except the one you want, you could instead blacklist all of them, and then allow only the one you want by adding the following lines to the /etc/multipath.conf file.

blacklist {
        wwid "*"
}

blacklist_exceptions {
        wwid "3600d0230000000000e13955cc3757803"
}

When specifying devices in the blacklist_exceptions section of the configuration file, you must specify the exceptions in the same way they were specified in the blacklist. For example, a WWID exception will not apply to devices specified by a devnode blacklist entry, even if the blacklisted device is associated with that WWID. Similarly, devnode exceptions apply only to devnode entries, and device exceptions apply only to device entries.

Configuration File Defaults

The /etc/multipath.conf configuration file includes a defaults section that sets the user_friendly_names parameter to yes, as follows.

defaults {
        user_friendly_names yes
}

This overwrites the default value of the user_friendly_names parameter.

The configuration file includes a template of configuration defaults. This section is commented out, as follows.

#defaults {
#       udev_dir                /dev
#       polling_interval        5
#       selector                "round-robin 0"
#       path_grouping_policy    failover
#       getuid_callout          "/lib/dev/scsi_id --whitelisted --device=/dev/%n"
#	prio			const
#	path_checker		directio
#	rr_min_io		1000
#	rr_weight		uniform
#	failback		manual
#	no_path_retry		fail
#	user_friendly_names	no
#}

To overwrite the default value for any of the configuration parameters, you can copy the relevant line from this template into the defaults section and uncomment it. For example, to overwrite the path_grouping_policy parameter so that it is multibus rather than the default value of failover, copy the appropriate line from the template to the initial defaults section of the configuration file, and uncomment it, as follows.

defaults {
        user_friendly_names     yes
        path_grouping_policy    multibus
}

Table Multipath Configuration Defaults describes the attributes that are set in the defaults section of the multipath.conf configuration file. These values are used by DM-Multipath unless they are overwritten by the attributes specified in the devices and multipaths sections of the multipath.conf file.

Multipath Configuration Defaults

Attribute Description
polling_interval Specifies the interval between two path checks in seconds. For properly functioning paths, the interval between checks will gradually increase to (4 * polling_interval). The default value is 5.
udev_dir The directory where udev device nodes are created. The default value is /dev.
multipath_dir The directory where the dynamic shared objects are stored. The default value is system dependent, commonly /lib/multipath.
verbosity The default verbosity. Higher values increase the verbosity level. Valid levels are between 0 and 6. The default value is 2.
path_selector

Specifies the default algorithm to use in determining what path to use for the next I/O operation. Possible values include:

  • round-robin 0: Loop through every path in the path group, sending the same amount of I/O to each.

  • queue-length 0: Send the next bunch of I/O down the path with the least number of outstanding I/O requests.

  • service-time 0: Send the next bunch of I/O down the path with the shortest estimated service time, which is determined by dividing the total size of the outstanding I/O to each path by its relative throughput.

The default value is round-robin 0.

path_grouping_policy

Specifies the default path grouping policy to apply to unspecified multipaths. Possible values include:

  • failover = 1 path per priority group

  • multibus = all valid paths in 1 priority group

  • group_by_serial = 1 priority group per detected serial number

  • group_by_prio = 1 priority group per path priority value

  • group_by_node_name = 1 priority group per target node name.

The default value is failover.

getuid_callout Specifies the default program and arguments to call out to obtain a unique path identifier. An absolute path is required.

The default value is /lib/udev/scsi_id --whitelisted --device=/dev/%n.

prio

Specifies the default function to call to obtain a path priority value. For example, the ALUA bits in SPC-3 provide an exploitable prio value. Possible values include:

  • const: Set a priority of 1 to all paths.

  • emc: Generate the path priority for EMC arrays.

  • alua: Generate the path priority based on the SCSI-3 ALUA settings.

  • netapp: Generate the path priority for NetApp arrays.

  • rdac: Generate the path priority for LSI/Engenio RDAC controller.

  • hp_sw: Generate the path priority for Compaq/HP controller in active/standby mode.

  • hds: Generate the path priority for Hitachi HDS Modular storage arrays.

The default value is const.

prio_args

The arguments string passed to the prio function Most prio functions do not need arguments. The datacore prioritizer need one. Example, "timeout=1000 preferredsds=foo". The default value is (null) "".

features The extra features of multipath devices. The only existing feature is queue_if_no_path, which is the same as setting no_path_retry to queue. For information on issues that may arise when using this feature, see Section, "Issues with queue_if_no_path feature".
path_checker

Specifies the default method used to determine the state of the paths. Possible values include:

  • readsector0: Read the first sector of the device.

  • tur: Issue a TEST UNIT READY to the device.

  • emc_clariion: Query the EMC Clariion specific EVPD page 0xC0 to determine the path.

  • hp_sw: Check the path state for HP storage arrays with Active/Standby firmware.

  • rdac: Check the path status for LSI/Engenio RDAC storage controller.

  • directio: Read the first sector with direct I/O.

The default value is directio.

failback

Manages path group failback.

  • A value of immediate specifies immediate failback to the highest priority path group that contains active paths.

  • A value of manual specifies that there should not be immediate failback but that failback can happen only with operator intervention.

  • A numeric value greater than zero specifies deferred failback, expressed in seconds.

The default value is manual.

rr_min_io Specifies the number of I/O requests to route to a path before switching to the next path in the current path group.

The default value is 1000.

rr_weight If set to priorities, then instead of sending rr_min_io requests to a path before calling path_selector to choose the next path, the number of requests to send is determined by rr_min_io times the path's priority, as determined by the prio function. If set to uniform, all path weights are equal.

The default value is uniform.

no_path_retry A numeric value for this attribute specifies the number of times the system should attempt to use a failed path before disabling queueing. A value of fail indicates immediate failure, without queueing. A value of queue indicates that queueing should not stop until the path is fixed.

The default value is 0.

user_friendly_names If set to yes, specifies that the system should use the /etc/multipath/bindings file to assign a persistent and unique alias to the multipath, in the form of mpathn. If set to no, specifies that the system should use the WWID as the alias for the multipath. In either case, what is specified here will be overridden by any device-specific aliases you specify in the multipaths section of the configuration file.

The default value is no.

queue_without_daemon If set to no, the multipathd daemon will disable queueing for all devices when it is shut down.

The default value is yes.

flush_on_last_del If set to yes, then multipath will disable queueing when the last path to a device has been deleted.

The default value is no.

max_fds Sets the maximum number of open file descriptors that can be opened by multipath and the multipathd daemon. This is equivalent to the ulimit -n command. A value of max will set this to the system limit from /proc/sys/fs/nr_open. If this is not set, the maximum number of open file descriptors is taken from the calling process; it is usually 1024. To be safe, this should be set to the maximum number of paths plus 32, if that number is greater than 1024.
checker_timer The timeout to use for path checkers that issue SCSI commands with an explicit timeout, in seconds.

The default value is taken from /sys/block/sdx/device/timeout, which is 30 seconds as of 12.04 LTS

fast_io_fail_tmo The number of seconds the SCSI layer will wait after a problem has been detected on an FC remote port before failing I/O to devices on that remote port. This value should be smaller than the value of dev_loss_tmo. Setting this to off will disable the timeout.

The default value is determined by the OS.

dev_loss_tmo The number of seconds the SCSI layer will wait after a problem has been detected on an FC remote port before removing it from the system. Setting this to infinity will set this to 2147483647 seconds, or 68 years. The default value is determined by the OS.

Configuration File Multipath Attributes

Table Multipath Attributes shows the attributes that you can set in the multipaths section of the multipath.conf configuration file for each specific multipath device. These attributes apply only to the one specified multipath. These defaults are used by DM-Multipath and override attributes set in the defaults and devices sections of the multipath.conf file.

Multipath Attributes

Attribute Description
wwid Specifies the WWID of the multipath device to which the multipath attributes apply. This parameter is mandatory for this section of the multipath.conf file.
alias Specifies the symbolic name for the multipath device to which the multipath attributes apply. If you are using user_friendly_names, do not set this value to mpathn; this may conflict with an automatically assigned user friendly name and give you incorrect device node names.

In addition, the following parameters may be overridden in this multipath section

The following example shows multipath attributes specified in the configuration file for two specific multipath devices. The first device has a WWID of 3600508b4000156d70001200000b0000 and a symbolic name of yellow.

The second multipath device in the example has a WWID of 1DEC_____321816758474 and a symbolic name of red. In this example, the rr_weight attributes are set to priorities.

multipaths {
       multipath {
              wwid                  3600508b4000156d70001200000b0000
              alias                 yellow
              path_grouping_policy  multibus
              path_selector         "round-robin 0"
              failback              manual
              rr_weight             priorities
              no_path_retry         5
       }
       multipath {
              wwid                  1DEC_____321816758474
              alias                 red
              rr_weight             priorities
        }
}

Configuration File Devices

Table Device Attributes shows the attributes that you can set for each individual storage device in the devices section of the multipath.conf configuration file. These attributes are used by DM-Multipath unless they are overwritten by the attributes specified in the multipaths section of the multipath.conf file for paths that contain the device. These attributes override the attributes set in the defaults section of the multipath.conf file.

Many devices that support multipathing are included by default in a multipath configuration. The values for the devices that are supported by default are listed in the multipath.conf.defaults file. You probably will not need to modify the values for these devices, but if you do you can overwrite the default values by including an entry in the configuration file for the device that overwrites those values. You can copy the device configuration defaults from the multipath.conf.annotated.gz or if you wish to have a brief config file, multipath.conf.synthetic file for the device and override the values that you want to change.

To add a device to this section of the configuration file that is not configured automatically by default, you must set the vendor and product parameters. You can find these values by looking at /sys/block/device_name/device/vendor and /sys/block/device_name/device/model where device_name is the device to be multipathed, as in the following example:

# cat /sys/block/sda/device/vendor
WINSYS  
# cat /sys/block/sda/device/model
SF2372

The additional parameters to specify depend on your specific device. If the device is active/active, you will usually not need to set additional parameters. You may want to set path_grouping_policy to multibus. Other parameters you may need to set are no_path_retry and rr_min_io, as described in Table Multipath Attributes.

If the device is active/passive, but it automatically switches paths with I/O to the passive path, you need to change the checker function to one that does not send I/O to the path to test if it is working (otherwise, your device will keep failing over). This almost always means that you set the path_checker to tur; this works for all SCSI devices that support the Test Unit Ready command, which most do.

If the device needs a special command to switch paths, then configuring this device for multipath requires a hardware handler kernel module. The current available hardware handler is emc. If this is not sufficient for your device, you may not be able to configure the device for multipath.

Device Attributes

Attribute Description
vendor Specifies the vendor name of the storage device to which the device attributes apply, for example COMPAQ.
product Specifies the product name of the storage device to which the device attributes apply, for example HSV110 (C)COMPAQ.
revision Specifies the product revision identifier of the storage device.
product_blacklist Specifies a regular expression used to blacklist devices by product.
hardware_handler

Specifies a module that will be used to perform hardware specific actions when switching path groups or handling I/O errors. Possible values include:

  • 1 emc: hardware handler for EMC storage arrays

  • 1 alua: hardware handler for SCSI-3 ALUA arrays.

  • 1 hp_sw: hardware handler for Compaq/HP controllers.

  • 1 rdac: hardware handler for the LSI/Engenio RDAC controllers.

In addition, the following parameters may be overridden in this device section

Whenever a hardware_handler is specified, it is your responsibility to ensure that the appropriate kernel module is loaded to support the specified interface. These modules can be found in /lib/modules/`uname -r`/kernel/drivers/scsi/device_handler/ . The requisite module should be integrated into the initrd to ensure the necessary discovery and failover-failback capacity is available during boot time. Example,

# echo scsi_dh_alua >> /etc/initramfs-tools/modules  ## append module to file
# update-initramfs -u -k all

The following example shows a device entry in the multipath configuration file.

#devices {
#	device {
#		vendor			"COMPAQ  "
#		product			"MSA1000         "
#		path_grouping_policy	multibus
#		path_checker		tur
#		rr_weight		priorities
#	}
#}

The spacing reserved in the vendor, product, and revision fields are significant as multipath is performing a direct match against these attributes, whose format is defined by the SCSI specification, specifically the Standard INQUIRY command. When quotes are used, the vendor, product, and revision fields will be interpreted strictly according to the spec. Regular expressions may be integrated into the quoted strings. Should a field be defined without the requisite spacing, multipath will copy the string into the properly sized buffer and pad with the appropriate number of spaces. The specification expects the entire field to be populated by printable characters or spaces, as seen in the example above

  • vendor: 8 characters

  • product: 16 characters

  • revision: 4 characters

To create a more robust configuration file, regular expressions can also be used. Operators include ^ $ [ ] . * ? +. Examples of functional regular expressions can be found by examining the live multipath database and multipath.conf example files found in /usr/share/doc/multipath-tools/examples:

# echo 'show config' | multipathd -k