mr_sas driver - poor performance when Invader adapter in JBOD mode.
Some of the LSI chips which are supported by the mr_sas driver can be configured in HBA mode.
When in this mode, most things appear superficially to function correctly but disk IO performance is abysmal. This is (mainly) due to two things - Physical disk fast-path is not being properly enabled for requests and the firmware is not receiving a valid device handle.
Typical iostat output on a Dell R730 with a Dell HPERC H730p (LSI 3108/Invader-based) adapter in HBA mode. Writes to a mirrored ZFS root pool on 15K SAS-2 drives, system otherwise idle.
extended device statistics ---- errors --- r/s w/s Mr/s Mw/s wait actv wsvc_t asvc_t %w %b s/w h/w trn tot device 0.0 142.0 0.0 11.6 0.0 10.0 0.0 70.4 0 100 0 0 0 0 c0t0d1s0 0.0 142.6 0.0 11.6 0.0 10.0 0.0 70.1 0 100 0 0 0 0 c0t1d1s0
Average service time is very variable and can often exceed 300ms.
The RAID/HBA adapter is also logging errors at a rate of 300 errors/s:
03/20/15 23:00:40: C0:iopiSCSIIOCompleteError: FPESTATUS_DEVHANDLE_OUT_OF_RANGE mid x02e6 PtrMsg xc00ccc00 03/20/15 23:00:40: C0:Out of range devHandle x0000 from SMID x0000022b
I have an initial patch for this which I'll be sending to the developers' list for review this week.
Updated by Electric Monk about 5 years ago
- Status changed from In Progress to Closed
- % Done changed from 80 to 100
commit e65bc174f67bfc560c23a317bb16268ec18103f6 Author: Andy Fiddaman <email@example.com> Date: 2015-03-28T20:09:43.000Z 5737 mr_sas driver does not pass SCSI mode sense to disks when adapter in JBOD mode. 5738 mr_sas driver - poor performance when Invader adapter in JBOD mode. Reviewed by: Hans Rosenfeld <firstname.lastname@example.org> Reviewed by: Garrett D'Amore <email@example.com> Approved by: Dan McDonald <firstname.lastname@example.org>