Multipath-usage.txt File for Red Hat Enterprise Linux 4 Update 3

Red Hat Cluster Suite: Configuring and Managing a Cluster
Prev |  | Next

# Appendix C. `Multipath-usage.txt` File for Red Hat Enterprise Linux 4 Update 3

This appendix contains the `Multipath-usage.txt` file. The file is included with the `dm-multipath` RPM and provides guidelines for using `dm-multipath` with Red Hat Cluster Suite for Red Hat Enterprise Linux 4 Update 3:

        RHEL4 U3 Device Mapper Multipath Usage

    Overview
    ------------
    Device Mapper Multipath (DM-MP) allows nodes to route I/O over
    multiple paths to a storage controller. A path refers to the
    connection from an HBA port to a storage controller port. As paths
    fail and new paths come up, DM-MP reroutes the I/O over the
    available paths.

    When there are multiple paths to a storage controller, each path
    appears as a separate device.  DM-MP creates a new device on top of
    those devices. For example, a node with two HBAs attached to a storage
    controller with two ports via a single unzoned FC switch sees four
    devices: /dev/sda, /dev/sdb, /dev/sdc, and /dev/sdd. DM-MP creates a
    single device, /dev/mpath/mpath1 that reroutes I/O to those four
    underlying devices.

    DM-MP consists of the following components:

    o dm-multipath kernel module -- This module reroutes I/O and fails
      over paths and path groups.

    o multipath command -- This command configures, lists, and removes
      multipath  devices. The command is run in rc.sysinit during startup,
       and by udev, whenever a block device    is added.

    o multipathd daemon -- This daemon monitors paths, checking to see
      if faulty paths have been fixed. As paths come back up, multipathd
      may also initiate path group switches to ensure that the optima
      path group is being used. Also, it is possible to interactively
      modify a multipath device.

    o kpartx command -- This command creates Device Mapper devices for the
      partitions on a device. It is necessary to use this command for DOS-
      based partitions with DM-MP.

    DM-MP works with a variety of storage arrays. It
    auto-configures the following storage arrays:

    o 3PARdata VV
    o Compaq HSV110
    o Compaq MSA1000
    o DDN SAN DataDirector
    o DEC HSG80
    o EMC SYMMETRIX
    o EMC CLARiiON
    o FSC CentricStor
    o HITACHI DF400
    o HITACHI DF500
    o HITACHI DF600
    o HP HSV110
    o HP HSV210
    o HP A6189A
    o HP Open-
    o IBM 3542
    o IBM ProFibre 4000R
    o NETAPP
    o SGI TP9100
    o SGI TP9300
    o SGI TP9400
    o SGI TP9500
    o STK OPENstroage D280
    o SUN StorEdge 3510
    o SUN T4

    Storage arrays not included in the list may require entries in the
    /etc/multipath.conf file.

    NOTE: Some storage arrays require special handling of I/O errors and
          path-group switching. Those require separate hardware handler
          kernel modules.

    Terms and Concepts
    ---------------------

    Hardware Handler:
        A kernel module that performs hardware-specific actions when
        switching path groups and dealing with I/O errors.

    Path:
        The connection from an HBA port to a storage controller port for
        a LUN. Each path appears as a separate device. Paths can be in
        various states (refer to "Path States").

    Path States:
        ready  - Path is able to handle I/O requests.
        shaky  - Path is up, but temporarily not available for normal
                 operations.
        faulty - Path is unable to handle I/O requests.
        ghost  - Path is a passive path, on an active/passive
                 controller.

            NOTE: The shaky and ghost states only exist for certain
                  storage arrays.

    Path Group:
        A grouping of paths. With DM-MP, only one path group--the
        active path group--receives I/O at any time. Within a path
        group, DM-MP selects which ready path should receive I/O in
        a round robin fashion. Path groups can be in various states
        (refer to "Path Group States").

    Path Group States:
        active   - Path group currently receiving I/O requests.
        enabled  - Path groups to try if the active path group has no paths
                   in the ready state.
        disabled - Path groups to try if the active path group and all
               enabled path groups have no paths in the active state.

            NOTE: The disabled state only exists for certain storage arrays.

    Path Priority:
        Each path can have a priority assigned to it by a callout program.
        Path priorities can be used to group paths by priority and change
        their relative weights for the round robin path selector.

    Path Group Priority:
        Each path group has a priority that is equal to the sum of the
        priorities of all the non-faulty paths in the group. By default, the
        multipathd daemon tries to ensure that the path group with the
        highest priority is always in the active state.

    Failover:
        When I/O to a path fails, the dm-multipath module tries to switch to
        an enabled path group. If there are no enabled path groups with
        any paths in the ready state, dm-multipath tries to switch to a
        disabled path group. If necessary, dm-multipath runs the hardware
        handler for the multipath device.

    Failback:
        At regular intervals, multipathd checks the current priority of
        all path groups. If the current path group is not the highest
        priority path group, multipathd reacts according to the failback
        mode. By default, multipathd immediately switches to the highest
        priority path group. Other options for multipathd are to (a) wait
        for a user-defined length of time (for the path groups to stabilize)
        and then switch or (b) for multipathd to do nothing and wait for
        manual intervention.  Failback can be forced at any time by
        running the multipath command.

    Multipath device:
        The multipath device is the device mapper device created by
        dm-multipath. A multipath device can be identified by either
        its WWID or its alias. A multipath device has one or more path
        groups. It also has numerous attributes defined in the
        following file:

    /usr/share/doc/device-mapper-multipathd-0.4.5/multipath.conf.annotated

    alias:
        The alias is the name of a multipath device. By default, the
        alias is set to the WWID. However, by setting the
        "user_friendly_names" configuration option, the alias is set to a
        unique name of the form mpath<n>. The alias name can also be
        explicitly set for each multipath device in the configuration file.

        NOTE: While the alias in guaranteed to be unique on a node, it
              is not guaranteed to be the same on all nodes using the
              multipath device. Also, it may change.

    WWID:
        The WWID (World Wide Identifier) is an identifier for the
        multipath device that is guaranteed to be globally unique and
        unchanging. It is determined by the getuid callout program.

    Using DM-MP
    ------------------------------------

    Initial setup:

    1. If it is not already installed. Install the device-mapper-multipath
       package.

    2. Edit /etc/multipath.conf. For new installations, all devices are
       blacklisted. The default blacklist is listed in the commented out
       section of /etc/multipath.conf.  If you comment out or delete
       the following lines in /etc/multipath.conf, the default blacklist
       takes effect:

       devnode_blacklist {
               devnode "*"
       }

       For some conditions, that may not be sufficient. If DM-MP is
       multipathing devices that you do not want it to work on, you can
       blacklist the devices by either device name or WWID.

       NOTE: It is safest to blacklist individual devices by WWID, because
             their device names may change.

       Several other configuration options are detailed later in this
       document. To check the  effects of configuration changes, you can
       do a dry run with the following command:

       # multipath -v2 -d

    3. Set the multipathd init script to run at boot time. by issuing
       the commands

       # chkconfig --add multipathd
       # chkconfig multipathd on

    4. start dm-multipath (This is only necessary the first time.  On
       reboot, this should happen automatically).

       # multipath
       # /etc/init.d/multipathd start

    After initial setup, all access to the multipathed storage should go
    through the multipath device.

    Configuration File:

    Many features of DM-MP are configurable using the configuration file,
    /etc/multipath.conf.

    For a complete list of all options with descriptions, refer to
    /usr/share/doc/device-mapper-multipathd-0.4.5/multipath.conf.annotated

    The configuration file is divided into four sections: system defaults,
    blacklisted devices (devnode_blacklist), per storage array model settings
    (devices), and per multipath device settings (multipaths).  The per
    multipath device settings are used for the multipath device with a
    matching "wwid" value. The per storage array model settings are used
    for all multipath devices with matching "vendor" and "product" values.
    To determine the attributes of a multipath device, first the per
    multipath settings are checked, then the per controller settings, then
    the system defaults.  The blacklisted device section is described
    setup step 2.

    NOTE: There are compiled-in defaults for the "defaults",
          "devnode_blacklist", and "devices" sections of the
          configuration file. To see what these are, refer to the
          following file:

    /usr/share/doc/device-mapper-multipathd-0.4.5/multipath.conf.synthetic

    If you are using one of the storage arrays listed in the preceding
    text (in "Overview"), you probably do not need to modify the "devices"
    subsection. If you are using a simple disk enclosure, the defaults
    should work. If you are using a storage array that is not
    listed, you may need to create a "devices" subsection for your array.

    Explanation of output
    -----------------------
    When you create, modify, or list a multipath device, you get a
    printout of the current device setup. The format is as follows.

    For each multipath device:

    action_if_any: alias (wwid_if_different_from_alias)
    [size][features][hardware_handler]

    For each path group:

    \_ scheduling_policy [path_group_priority_if_known]
    [path_group_status_if_known]

    For each path:

     \_ host:channel:id:lun devnode major:minor [path_status]
     [dm_status_if_known]

    NOTE: The preceding lines for path group and path
          were broken because of print limitations.

    The dm status (dm_status_if_known) is like the path status
    (path_status), but from the kernel's point of view.  The dm status
    has two states: "failed", which is analogous to "faulty",
    and "active" which covers all other path states. Occasionally,
    the path state and the dm state of a device will temporarily
    not agree.

    NOTE: When a multipath device is being created or modified, the
    path group status and the dm status are not known.  Also, the
    features are not always correct. When a multipath device is being
    isted, the path group priority is not known.

    Restrictions
    ---------------
    DM-MP cannot be run on either the root or boot device.

    Other Sources of information
    ----------------------------
    Configuration file explanation:
    /usr/share/doc/device-mapper-multipathd-0.4.5/multipath.conf.annotated

    Upstream documentation:
    http://christophe.varoqui.free.fr/wiki/wakka.php?wiki=Home

    mailing list:
    dm-devel@redhat.com
    Subscribe to this from https://www.redhat.com/mailman/listinfo/dm-devel.
    The list archives are at https://www.redhat.com/archives/dm-devel/

    Man pages:
    multipath.8, multipathd.8, kpartx.8 mpath_ctl.8

Prev | Home | Next
Selectively Installing Red Hat Cluster Suite Packages | Up | Index