Bug #4546

mpt_sas needs enhancing to support LSI MPI2.5

Added by Andy Giles about 5 years ago. Updated about 3 years ago.

Status:ClosedStart date:2014-01-29
Priority:NormalDue date:
Assignee:Andy Giles% Done:

100%

Category:-
Target version:-
Difficulty:Medium Tags:needs-triage

Description

In order to work with newer LSI HBA cards (eg, SAS3008) and support SAS3 devices mpt_sas needs enhancing to work with the LSI Fusion MPT MPI2.5 specification.

mpt_sas - i86 64 bit kernel module with DEBUG enabled. (343 KB) Andy Giles, 2014-02-01 01:21 PM

mpt_sas - Updated version - use this rather than the previous one. (358 KB) Andy Giles, 2014-02-10 09:01 AM

Related issues

Related to illumos gate - Feature #4675: Support LSI 12Gb/s SAS HBAs Closed 2014-03-09

History

#1 Updated by Rich Ercolani about 5 years ago

Is there a reason this should be an extension of the same driver versus a new driver? As far as I know, all the other platforms implement this as a different driver.

#2 Updated by Andy Giles about 5 years ago

Rich Ercolani wrote:

Is there a reason this should be an extension of the same driver versus a new driver? As far as I know, all the other platforms implement this as a different driver.

If you look at the differences in the FreeBSD drivers available from LSI the one that supports MPI2.5 also works with MPI2. Not sure if the same is true for Linux, but you are right in the case of Solaris 11.1, it's a separate driver called lsc. In fact, although they are at quite a low level the changes are not too extensive. I would question why anyone would want to write a separate driver. However, I know there are a number of existing issues with mpt_sas and I can understand the feeling that it would benefit from a re-write.

#3 Updated by Andy Giles about 5 years ago

  • % Done changed from 0 to 20
  • Assignee set to Andy Giles
  • Status changed from New to In Progress
  • File mpt_sas added

The following commit references a clone of illumos-gate with a branch called MPTSAS3 that contains new code to support MPI2.5. This is at a very early stage and will require a great deal more testing. The work has been done by analysis of the differences between the FreeBSD source code provided by LSI for the SAS2008 and SAS3008. I don't have access to the actual LSI MPI2.5 Spec so it's difficult to check on details of the implementation.

https://github.com/andy2gi/illumos-gate/commit/2ae6c429f39263dcef86d132a7e56090f3eba1f7

However I've certainly done some basic testing for data integrity and zpool manipulation.
I'm using both an LSI3008 currently with the following firmware:

# lsiutil

LSI Logic MPT Configuration Utility, Version 1.63, June 4, 2009

1 MPT Port found

     Port Name         Chip Vendor/Type/Rev    MPT Rev  Firmware Rev  IOC
 1.  mpt_sas4          LSI Logic 0096 02         205      01000200     0

Select a device:  [1-1 or 0 to quit] 1
Main menu, select an option:  [1-99 or e/p/w or 0 to quit] 1

Current active firmware version is 01000200 (1.00.02)
Firmware image's version is MPTFW-01.00.02.00-IT
  LSI Logic
  Not Packaged Yet
x86 BIOS image's version is MPT3BIOS-8.03.00.00 (2013.03.28)
EFI BIOS image's version is 0.02.00.00

Some specific important areas not yet tested are:
Machine with >4G memory
Any raid card or IR firmware
Firmware flashing (backup works though)
smartd/smartctl
Any SPARC based machine.

The commit does not contain any additional device PCI mapping in the manifest yet, but it's simple to add those manually by:

# update_drv -a -i '"pciex1000,97"' mpt_sas

Numbers in the range 1000,90 -> 1000,97 have the potential to work, running 'prtconf -v | grep "1000,"' should give you a clue as to the exact numbers you need to use.

For those not wanting to go compile the driver I've attached a binary compiled with DEBUG enabled. Just copy it over the top of /kernel/drv/amd64/mpt_sas. You might want to make a backup of the original and do this in a new boot environment! This binary was built using the latest OmniOS build environment.

If you do any testing please provide feedback in this issue log.
Feedback on the code changes and pull requests to the repo are also welcome.

#4 Updated by Rich Ercolani about 5 years ago

I happen to have one of these cards; I'll see if I can't find a minute to toss it in a machine running Illumos with this tree and provide feedback/fixes. :)

#5 Updated by Rich Ercolani about 5 years ago

Oh, here's a question - did you test the changes against any existing MPT2-era cards to make sure this doesn't blow them up? ;-)

#6 Updated by Andy Giles about 5 years ago

Rich Ercolani wrote:

Oh, here's a question - did you test the changes against any existing MPT2-era cards to make sure this doesn't blow them up? ;-)

Yes, I have 2 systems, one with LSI9207-8i and another with LSI9300-8i. I'm doing parallel testing with them. However there are obviously a lot of other cards and I would not trust any valuable data to the driver yet.

#7 Updated by Andy Giles about 5 years ago

  • % Done changed from 20 to 40
  • File mpt_sas added

Found one major problem over the previous week with large transfers, eg dd with bs=1024k.
Otherwise additional testing includes:

- Running on machines >4G memory.
 - Smartctl 
 - Updating 9300-8i HBA firmware with lsiutil.
 - Updating 9207-8i HBA firmware with lsiutil.
 - Running variations of filebench on zfs mirror.
 - Verify whole disk runs under format.

Testing continues.

#8 Updated by Rich Ercolani about 5 years ago

What version of lsiutil supports the 9300-family of cards? I see the output says 1.63, I'm just surprised that nothing catches fire, since that seems to date from 2011...

#9 Updated by Andy Giles about 5 years ago

Rich Ercolani wrote:

What version of lsiutil supports the 9300-family of cards? I see the output says 1.63, I'm just surprised that nothing catches fire, since that seems to date from 2011...

Correct, but I can't find anything newer!
I did specifically check the lsiutil code for firmware update before I tried it.

#10 Updated by Electric Monk over 4 years ago

  • % Done changed from 40 to 100
  • Status changed from In Progress to Closed

git commit ed7418ae3db79aed594bc486a3b52fc971f4ea4f

commit  ed7418ae3db79aed594bc486a3b52fc971f4ea4f
Author: Andy Giles <illumos@ang.homedns.org>
Date:   2014-07-21T20:08:32.000Z

    4546 mpt_sas needs enhancing to support LSI MPI2.5
    Reviewed by: Hans Rosenfeld <hans.rosenfeld@nexenta.com>
    Reviewed by: Albert Lee <trisk@nexenta.com>
    Reviewed by: Josef 'Jeff' Sipek <jeffpc@josefsipek.net>
    Reviewed by: Dan McDonald <danmcd@omniti.com>
    Approved by: Gordon Ross <gwr@nexenta.com>

Also available in: Atom