How to change IO scheduler and LUN IO queue depth

To change scheduler to cfq for a LUN

# echo cfq > /sys/block/sdxx/queue/scheduler
# cat /sys/block/sdxx/queue/scheduler
  noop anticipatory deadline [cfq]


To change schedule to deadline on for all paths of all multiapth devices

# multipath -l|awk '$3~/^sd/ {print $3}'|while read dev; do echo deadline > /sys/block/$dev/queue/scheduler; done 
# multipath -l| awk '$3~/^sd/ {print $3}'|while read dev; do echo -n "$dev "; cat /sys/block/$dev/queue/scheduler; done 

Add elevator=deadline in grub.conf to retain it over reboot.

To change queue depth from 16 to 32 for Q-Logic HBA

# cat /sys/module/qla2xxx/parameters/ql2xmaxqdepth
16
# echo 32 > /sys/module/qla2xxx/parameters/ql2xmaxqdepth
# cat /sys/module/qla2xxx/parameters/ql2xmaxqdepth

Modify /etc/modprobe.conf to retain it over reboot
options qla2xxx ql2xmaxqdepth=32 qlport_down_retry=14 ql2xloginretrycount=30


To monitor LUN queue

# cat  $(echo /proc/scsi/sg/device{_hdr,s})|more
host   chan    id      lun     type    opens   qdepth  busy   online
3       0       0       12      0       1       32     5       1

If busy is reaching near to qdepth, we need to increase LUN qdepth. We should not increase qdepth too much !

The column headers in 'device_hdr' are given below. If the device is not present (and one is present after it) then a line of "-1" entries is output. Each entry is separated by a whitespace (currently a tab):

host            host number (indexes 'hosts' table, origin 0)
chan            channel number of device
id              SCSI id of device
lun             Logical Unit number of device
type            SCSI type (e.g. 0->disk, 5->cdrom, 6->scanner)
opens           number of opens (by sd, sr, sr and sg) at this time
depth           maximum queue depth supported by device
busy            number of commands being processed by host for this device
online          1 indicates device is in normal online state, 0->offline