I was at one of my customers the other day helping them set up MPIO that is built into SLES 10. While I was there I took a few notes for what we did to get things working for their environment. These same instructions should work with other SAN’s that can handle multi path IO.
SLES 10 supports a lot of SAN’s right out of the box and automatically detects them so you don’t really need an /etc/multipath.conf. My customer likes to be able to change the black list for various types of hardware they use and wanted user-friendly names. To do this I created a multipath.conf for them that looked like the following…
## /etc/multipath.conf
file for SLES 10
## You may find a full copy of this file, with comments,
here..
## /usr/share/doc/packages/multipath-tools/multipath.conf
# Setup user friendly names
# name :
user_friendly_names
# scope : multipath
# desc : If set to “yes”, using the bindings file
# /var/lib/multipath/bindings to assign a persistent
and
# unique alias to the multipath, in the form of mpath<n>.
# If set to “no” use the WWID as the alias. In either
case
# this be will be overriden by any specific aliases
in this
# file.
# values : yes|no
# default : no
defaults
{
user_friendly_names yes
}
# Setup the blacklisted devices….
# name :
blacklist
# scope : multipath & multipathd
# desc : list of device names that are not multipath
candidates
# default : cciss, fd, hd, md, dm, sr, scd, st, ram,
raw, loop
#
blacklist
{
devnode “^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*”
devnode “^hd[a-z][[0-9]*]”
devnode “^cciss!c[0-9]d[0-9]*[p[0-9]*]”}
If your curious about what platforms that SLES 10 supports out of the box a list is in the SLES documentation.
Assuming you already have your LUNs assigned to you. And once you have the /etc/multipath.conf file there are a few services that need to be started to make all this work.
# service
boot.multipth start
# service multipathd start
That should start the demons and load kernel modules that you need. To check that do an lsmod to see if you see dm_multipath and multipath. Once that is done you can check your setup to see if it is correct…
# multipath –v2 -d
create: mpath10 (360080480000290100601544032363831) EMC,SYMMETRIX
[size=200G][features=0][hwhandler=0]
\_ round-robin 0 [prio=4][undef]
\_ 11:0:0:39 sdbr 68:80 [undef][ready]
\_ 11:0:1:39 sdcc 69:0 [undef][ready]
\_ 10:0:0:39 sdl 8:176 [undef][ready]
\_ 10:0:1:39 sdw 65:96 [undef][ready]
create: mpath11 (360080480000290100601544032363832) EMC,SYMMETRIX
[size=400G][features=0][hwhandler=0]
\_ round-robin 0 [prio=4][undef]
\_ 11:0:0:40 sdbs 68:96 [undef][ready]
\_ 11:0:1:40 sdcd 69:16 [undef][ready]
\_ 10:0:0:40 sdm 8:192 [undef][ready]
\_ 10:0:1:40 sdx 65:112 [undef][ready]
That is what it looks like for the EMC Symmetrix I was working with so your mileage may vary.
Once you have the devices showing up correctly you need to make sure the multi path modules load on reboot. To do that run the following commands…
# chkconfig
multipathd on
# chkconfig boot.multipath on
The next thing is to configure LVM to scan these devices so you can use them in your volume groups. To do this you will need to edit /etc/lvm/lvm.conf in the following places…
filter = [ “a|/dev/disk/by-id/.*|”, “r|.*|” ]types = [ “device-mapper”, 253 ]
Above limits the devices that LVM will scan to only devices that show up by-id. If your using LVM to manage other disks that are not in that directory, think local scsi drives, you will need to make sure those are still available by adjusting your filter more like this…
filter = [ “a|/dev/disk/by-id/.*|”, “a|/dev/sda1$/”, “r|.*|” ]
Once that is done do a lvmdiskscan to get LVM to see the new drives.
A few other things that customers often ask for is how to have SLES scan for new LUNs on the san without rebooting. With SLES10 it’s s simple as passing a few parameters to the sys file system.
# echo 1 > /sys/class/fc_host/host<number>/issue_lip
That will make the kernel aware of the new devices at a very low level, but the devices are not yet usable. To make them usable do the following…
# echo “- - -” > /sys/class/scsi_host/host<number>/scan
That will scan all devices and add the new ones for you. All of this information is in the SLES 10 Storage Administration Guide, including various ways to recover from issues.
Also since SP1 SLES10 has been able to boot a mpio device from the SAN. The doc for doing that in SP1 is located here.
Have fun and enjoy..
Last 3 posts by Daniel
- Fun with your SAN and Multi-path - May 6th, 2008
Popularity: 27% [?]
July 17th, 2008 at 11:52 am
I would just add, for IBM DS8000, DS6000, ESS, or SVC 4.2 disk systems, you need to use the multipath.conf file located in a table here: http://www-1.ibm.com/support/docview.wss?rs=540&context=ST52G7&dc=D430&uid=ssg1S4000107&loc=en_US&cs=utf-8&lang=en
July 24th, 2008 at 10:58 am
Daniel - Nice article, helped me with MPIO. I’m trying MPIO with EMC CX-300 / SLES 10 SP2 and Xen. Thanks -Bruce
Think you have one typo:
# service boot.multipth start
should be
# service boot.multipath start