We are buying a lot of Dell PE860's with the SAS Raid card for
dedicated hosting. Here is some info for working with that card.
SAS 5/iR adapter is using chip SAS1068 from LSI. The adapter is supported by mptsas driver in Linux 2.6.18 kernel.
Monitoring of RAID status is possible by mpt-status program. We have the RedHat rpm
here
Make sure mptctl is loaded and the node is created.
mknod /dev/mptctl c 10 220
modprobe mptctl
Here is an output:
[root@bolt40 root]# mpt-status
ioc0 vol_id 0 type IM, 2 phy, 148 GB, state OPTIMAL, flags ENABLED
ioc0 phy 1 scsi_id 32 ATA WDC WD1600JS-75N 2E04, 149 GB, state ONLINE, flags NONE
ioc0 phy 0 scsi_id 1 ATA WDC WD1600JS-75N 2E04, 149 GB, state ONLINE, flags NONE
By default SAS 5/iR RAID controller came with cache set to write through. That really horribly degrades performance
[root@bolt40 lsiutil]# time (dd if=/dev/zero of=bigfile count=10240 bs=1M; sync)
10240+0 records in
10240+0 records out
10737418240 bytes (11 GB) copied, 257.313 seconds, 41.7 MB/s
real 6m32.576s
user 0m0.012s
sys 0m25.194s
The transfer rate is 10737418240 / (6*60 + 32.576) / 1024 / 1024 = 26.08 MiB/sec. YUCK!
after googling I found some answers.
The tool you need is LSIUtil, it is inside Dell SAS 5/iR Adapter Driver. Cache tuning is not accessible via RAID controller BIOS. If you prefer you can take pre-compiled version of
lsiutil-1.56 from our server.
Usage of lsiutil is simple:
[root@bolt40 lsiutil]# ./lsiutil
LSI Logic MPT Configuration Utility, Version 1.56, March 19, 2008
1 MPT Port found
Port Name Chip Vendor/Type/Rev MPT Rev Firmware Rev IOC
1. /proc/mpt/ioc0 LSI Logic SAS1068 B0 105 000a3100 0
Select a device: [1-1 or 0 to quit] 1
1. Identify firmware, BIOS, and/or FCode
2. Download firmware (update the FLASH)
4. Download/erase BIOS and/or FCode (update the FLASH)
8. Scan for devices
10. Change IOC settings (interrupt coalescing)
13. Change SAS IO Unit settings
16. Display attached devices
20. Diagnostics
21. RAID actions
22. Reset bus
23. Reset target
30. Beacon on
31. Beacon off
97. Reset SAS phy
98. Reset SAS link
99. Reset port
Main menu, select an option: [1-99 or e for expert or 0 to quit] 21
1. Show volumes
2. Show physical disks
3. Get volume state
23. Replace physical disk
30. Create volume
31. Delete volume
32. Change volume settings
RAID actions menu, select an option: [1-99 or e for expert or 0 to quit] 32
Volume: [0-1 or RETURN to quit]
Volume 0 Settings: write caching disabled, auto configure, priority resync
Volume 0 draws from Hot Spare Pools:
Enable write caching: [Yes or No, default is No] yes
Offline on SMART data: [Yes or No, default is No]
Auto configuration: [Yes or No, default is Yes]
Priority resync: [Yes or No, default is Yes]
Hot Spare Pools (bitmask of pool numbers): [00 to FF, default is 01]
RAID actions menu, select an option: [1-99 or e for expert or 0 to quit]
Main menu, select an option: [1-99 or e for expert or 0 to quit]
Port Name Chip Vendor/Type/Rev MPT Rev Firmware Rev
1. /proc/mpt/ioc0 LSI Logic SAS1068 B0 105 000a310
Select a device: [1-1 or 0 to quit]
Setting seams to be persistent, it survives system reboot.
[root@bolt40 lsiutil]# time (dd if=/dev/zero of=bigfile count=10240 bs=1M; sync)
10240+0 records in
10240+0 records out
10737418240 bytes (11 GB) copied, 158.017 seconds, 68.0 MB/s
real 3m59.514s
user 0m0.017s
sys 0m27.018s
Write rate is now a lot bigger: 10737418240 / (3 * 60 + 59.514) / 1024 / 1024 = 42.75 MiB/sec.
Possible reason why LSI do not provide this feature inside adapter BIOS might be because this card doesn't have it's own battery backed up memory, adapter does rely on cache of HDDs. I'm afraid that in case of power failure during massive I/O, system could end with badly corrupted file system.
