Project

General

Profile

Actions

Bug #14693

open

JMicron JMB585 AHCI SATA controller: System panics when trying to write data on pool

Added by Olivia Mocellini Merbeller 6 days ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
driver - device drivers
Start date:
Due date:
% Done:

0%

Estimated time:
Difficulty:
Medium
Tags:
Gerrit CR:

Description

The system consists of:
- SmartOS with PI 20220505T001410Z
- AMD B550 based motherboard (Asus ROG STRIX B550-I GAMING) whose onboard SATA Controller works as expected.
- JMicron JMB585 AHCI SATA controller exhibiting the problem.
- 2 SSDs connected to AMD's SATA controller.
- 2 HDDs (Seagate Ironwolf) connected to the JMicron's SATA controller.

$ prtconf -dD |grep -A2 -i jmicron
            pci197b,0 (pciex197b,585) [JMicron Technology Corp. JMB58x AHCI SATA controller], instance #0 (driver name: ahci)
                disk, instance #2 (driver name: sd)
                disk, instance #3 (driver name: sd)

Initially after a cold boot the system sees the disks attached to the JMicron controller, but after a few minutes many utilities that access the the disk subsystem (like zpool, diskinfo, etc) start to freeze and don't complete their run/output. No logs are generated on /var/adm/messages.

2022-05-13T21:40:32.821891+00:00 srv-smartos-1 ahci: [ID 405770 kern.info] NOTICE: ahci0: hba AHCI version = 1.31
2022-05-13T21:40:32.821894+00:00 srv-smartos-1 ahci: [ID 405770 kern.info] NOTICE: ahci1: hba AHCI version = 1.31
...
2022-05-13T21:40:32.822111+00:00 srv-smartos-1 sata: [ID 663010 kern.info] /pci@0,0/pci1022,1634@2,1/pci1b21,1062@0,1 :
2022-05-13T21:40:32.822115+00:00 srv-smartos-1 sata: [ID 761595 kern.info] #011SATA disk device at port 0
2022-05-13T21:40:32.822117+00:00 srv-smartos-1 sata: [ID 846691 kern.info] #011model INTEL SSDSA2CW120G3
2022-05-13T21:40:32.822121+00:00 srv-smartos-1 sata: [ID 693010 kern.info] #011firmware 4PC10362
2022-05-13T21:40:32.822131+00:00 srv-smartos-1 sata: [ID 163988 kern.info] #011serial number <REMOVED>
2022-05-13T21:40:32.822135+00:00 srv-smartos-1 sata: [ID 594940 kern.info] #011supported features:
2022-05-13T21:40:32.822138+00:00 srv-smartos-1 sata: [ID 981177 kern.info] #011 48-bit LBA, DMA, Native Command Queueing, SMART, SMART self-test
2022-05-13T21:40:32.822140+00:00 srv-smartos-1 sata: [ID 643337 kern.info] #011SATA Gen2 signaling speed (3.0Gbps)
2022-05-13T21:40:32.822143+00:00 srv-smartos-1 sata: [ID 349649 kern.info] #011Supported queue depth 32
2022-05-13T21:40:32.822145+00:00 srv-smartos-1 sata: [ID 349649 kern.info] #011capacity = 234441648 sectors
2022-05-13T21:40:32.822148+00:00 srv-smartos-1 scsi: [ID 583861 kern.info] sd0 at ahci1: target 0 lun 0
2022-05-13T21:40:32.822150+00:00 srv-smartos-1 genunix: [ID 936769 kern.info] sd0 is /pci@0,0/pci1022,1634@2,1/pci1b21,1062@0,1/disk@0,0
2022-05-13T21:40:32.822153+00:00 srv-smartos-1 genunix: [ID 408114 kern.info] /pci@0,0/pci1022,1634@2,1/pci1b21,1062@0,1/disk@0,0 (sd0) online
2022-05-13T21:40:32.822155+00:00 srv-smartos-1 sata: [ID 663010 kern.info] /pci@0,0/pci1022,1634@2,1/pci1b21,1062@0,1 :
2022-05-13T21:40:32.822158+00:00 srv-smartos-1 sata: [ID 761595 kern.info] #011SATA disk device at port 1
2022-05-13T21:40:32.822160+00:00 srv-smartos-1 sata: [ID 846691 kern.info] #011model MZ7LM480HCHP-000V3   00LF163      LENOVO
2022-05-13T21:40:32.822163+00:00 srv-smartos-1 sata: [ID 693010 kern.info] #011firmware GXT3CB30
2022-05-13T21:40:32.822165+00:00 srv-smartos-1 sata: [ID 163988 kern.info] #011serial number             <REMOVED>
2022-05-13T21:40:32.822167+00:00 srv-smartos-1 sata: [ID 594940 kern.info] #011supported features:
2022-05-13T21:40:32.822170+00:00 srv-smartos-1 sata: [ID 981177 kern.info] #011 48-bit LBA, DMA, Native Command Queueing, SMART, SMART self-test
2022-05-13T21:40:32.822172+00:00 srv-smartos-1 sata: [ID 996592 kern.info] #011SATA Gen3 signaling speed (6.0Gbps)
2022-05-13T21:40:32.822174+00:00 srv-smartos-1 sata: [ID 349649 kern.info] #011Supported queue depth 32
2022-05-13T21:40:32.822177+00:00 srv-smartos-1 sata: [ID 349649 kern.info] #011capacity = 937703088 sectors
2022-05-13T21:40:32.822184+00:00 srv-smartos-1 scsi: [ID 583861 kern.info] sd1 at ahci1: target 1 lun 0
2022-05-13T21:40:32.822189+00:00 srv-smartos-1 genunix: [ID 936769 kern.info] sd1 is /pci@0,0/pci1022,1634@2,1/pci1b21,1062@0,1/disk@1,0
2022-05-13T21:40:32.822192+00:00 srv-smartos-1 genunix: [ID 408114 kern.info] /pci@0,0/pci1022,1634@2,1/pci1b21,1062@0,1/disk@1,0 (sd1) online
2022-05-13T21:40:32.822203+00:00 srv-smartos-1 sata: [ID 663010 kern.info] /pci@0,0/pci1022,1634@2,2/pci197b,0@0 :
2022-05-13T21:40:32.822206+00:00 srv-smartos-1 sata: [ID 761595 kern.info] #011SATA disk device at port 0
2022-05-13T21:40:32.822209+00:00 srv-smartos-1 sata: [ID 846691 kern.info] #011model ST8000VN0022-2EL112
2022-05-13T21:40:32.822211+00:00 srv-smartos-1 sata: [ID 693010 kern.info] #011firmware SC61
2022-05-13T21:40:32.822214+00:00 srv-smartos-1 sata: [ID 163988 kern.info] #011serial number             <REMOVED1>
2022-05-13T21:40:32.822217+00:00 srv-smartos-1 sata: [ID 594940 kern.info] #011supported features:
2022-05-13T21:40:32.822219+00:00 srv-smartos-1 sata: [ID 981177 kern.info] #011 48-bit LBA, DMA, Native Command Queueing, SMART, SMART self-test
2022-05-13T21:40:32.822221+00:00 srv-smartos-1 sata: [ID 996592 kern.info] #011SATA Gen3 signaling speed (6.0Gbps)
2022-05-13T21:40:32.822224+00:00 srv-smartos-1 sata: [ID 349649 kern.info] #011Supported queue depth 32
2022-05-13T21:40:32.822226+00:00 srv-smartos-1 sata: [ID 349649 kern.info] #011capacity = 15628053168 sectors
2022-05-13T21:40:32.822228+00:00 srv-smartos-1 scsi: [ID 583861 kern.info] sd2 at ahci0: target 0 lun 0
2022-05-13T21:40:32.822231+00:00 srv-smartos-1 genunix: [ID 936769 kern.info] sd2 is /pci@0,0/pci1022,1634@2,2/pci197b,0@0/disk@0,0
2022-05-13T21:40:32.822233+00:00 srv-smartos-1 genunix: [ID 408114 kern.info] /pci@0,0/pci1022,1634@2,2/pci197b,0@0/disk@0,0 (sd2) online
2022-05-13T21:40:32.822235+00:00 srv-smartos-1 sata: [ID 663010 kern.info] /pci@0,0/pci1022,1634@2,2/pci197b,0@0 :
2022-05-13T21:40:32.822238+00:00 srv-smartos-1 sata: [ID 761595 kern.info] #011SATA disk device at port 4
2022-05-13T21:40:32.822240+00:00 srv-smartos-1 sata: [ID 846691 kern.info] #011model ST8000VN0022-2EL112
2022-05-13T21:40:32.822242+00:00 srv-smartos-1 sata: [ID 693010 kern.info] #011firmware SC61
2022-05-13T21:40:32.822251+00:00 srv-smartos-1 sata: [ID 163988 kern.info] #011serial number             <REMOVED2>
2022-05-13T21:40:32.822254+00:00 srv-smartos-1 sata: [ID 594940 kern.info] #011supported features:
2022-05-13T21:40:32.822256+00:00 srv-smartos-1 sata: [ID 981177 kern.info] #011 48-bit LBA, DMA, Native Command Queueing, SMART, SMART self-test
2022-05-13T21:40:32.822259+00:00 srv-smartos-1 sata: [ID 996592 kern.info] #011SATA Gen3 signaling speed (6.0Gbps)
2022-05-13T21:40:32.822262+00:00 srv-smartos-1 sata: [ID 349649 kern.info] #011Supported queue depth 32
2022-05-13T21:40:32.822264+00:00 srv-smartos-1 sata: [ID 349649 kern.info] #011capacity = 15628053168 sectors
2022-05-13T21:40:32.822268+00:00 srv-smartos-1 scsi: [ID 583861 kern.info] sd3 at ahci0: target 4 lun 0
2022-05-13T21:40:32.822270+00:00 srv-smartos-1 genunix: [ID 936769 kern.info] sd3 is /pci@0,0/pci1022,1634@2,2/pci197b,0@0/disk@4,0
2022-05-13T21:40:32.822274+00:00 srv-smartos-1 genunix: [ID 408114 kern.info] /pci@0,0/pci1022,1634@2,2/pci197b,0@0/disk@4,0 (sd3) online
$ cfgadm -avl sata
Ap_Id                          Receptacle   Occupant     Condition  Information
When         Type         Busy     Phys_Id
sata1/0::dsk/c1t0d0            connected    configured   ok         Mod: INTEL SSDSA2CW120G3 FRev: 4PC10362 SN: <REMOVED>
unavailable  disk         n        /devices/pci@0,0/pci1022,1634@2,1/pci1b21,1062@0,1:0
sata1/1::dsk/c1t1d0            connected    configured   ok         Mod: MZ7LM480HCHP-000V3   00LF163      LENOVO FRev: GXT3CB30 SN: <REMOVED>
unavailable  disk         n        /devices/pci@0,0/pci1022,1634@2,1/pci1b21,1062@0,1:1
sata1/2                        empty        unconfigured ok
unavailable  sata-port    n        /devices/pci@0,0/pci1022,1634@2,1/pci1b21,1062@0,1:2
sata1/3                        empty        unconfigured ok
unavailable  sata-port    n        /devices/pci@0,0/pci1022,1634@2,1/pci1b21,1062@0,1:3
sata1/4                        empty        unconfigured ok
unavailable  sata-port    n        /devices/pci@0,0/pci1022,1634@2,1/pci1b21,1062@0,1:4
sata1/5                        empty        unconfigured ok
unavailable  sata-port    n        /devices/pci@0,0/pci1022,1634@2,1/pci1b21,1062@0,1:5
sata2/0::dsk/c2t0d0            connected    configured   ok         Mod: ST8000VN0022-2EL112 FRev: SC61 SN: <REMOVED1>
unavailable  disk         n        /devices/pci@0,0/pci1022,1634@2,2/pci197b,0@0:0
sata2/1                        empty        unconfigured ok
unavailable  sata-port    n        /devices/pci@0,0/pci1022,1634@2,2/pci197b,0@0:1
sata2/2                        empty        unconfigured ok
unavailable  sata-port    n        /devices/pci@0,0/pci1022,1634@2,2/pci197b,0@0:2
sata2/3                        empty        unconfigured ok
unavailable  sata-port    n        /devices/pci@0,0/pci1022,1634@2,2/pci197b,0@0:3
sata2/4::dsk/c2t4d0            connected    configured   ok         Mod: ST8000VN0022-2EL112 FRev: SC61 SN: <REMOVED2>
unavailable  disk         n        /devices/pci@0,0/pci1022,1634@2,2/pci197b,0@0:4
$ diskinfo
TYPE    DISK                    VID      PID              SIZE          RMV SSD
SATA    c1t0d0                  INTEL    SSDSA2CW120G3     111.79 GiB   no  yes
SATA    c1t1d0                  ATA      MZ7LM480HCHP-000V3   00LF163      LENOVO  447.13 GiB   no  yes
SATA    c2t0d0                  ATA      ST8000VN0022-2EL112 7452.04 GiB   no  no
SATA    c2t4d0                  ATA      ST8000VN0022-2EL112 7452.04 GiB   no  no
$ fmadm faulty
$ zpool import
$ zpool status -v
  pool: gold
 state: ONLINE
  scan: resilvered 3.90M in 0 days 00:00:01 with 0 errors on Fri May  6 14:09:02 2022
config:

        NAME        STATE     READ WRITE CKSUM
        gold        ONLINE       0     0     0
          mirror-0  ONLINE       0     0     0
            c2t0d0  ONLINE       0     0     0
            c2t4d0  ONLINE       0     0     0

errors: No known data errors
...

And after some time...

$ diskinfo
TYPE    DISK                    VID      PID              SIZE          RMV SSD

^C^C
^C
$ zpool status
  pool: gold
 state: ONLINE
  scan: resilvered 3.90M in 0 days 00:00:01 with 0 errors on Fri May  6 14:09:02 2022

^C^C

'fmdump -eV' output inside of the attached zip file.

Copying something to the pool that sits on the JMicron controller panics the system.

2022-05-14T11:21:12.936247+00:00 srv-smartos-1 rsyslogd3: No files configured to be monitored [try http://www.rsyslog.com/e/-3 ]
2022-05-14T11:21:12.937117+00:00 srv-smartos-1 genunix: [ID 843051 kern.info] NOTICE: SUNW-MSG-ID: SUNOS-8000-0G, TYPE: Error, VER: 1, SEVERITY: Major#012
2022-05-14T11:21:12.937121+00:00 srv-smartos-1 unix: [ID 836849 kern.notice] #012#015panic[cpu14]/thread=fffffe007a51bc20:
2022-05-14T11:21:12.937124+00:00 srv-smartos-1 genunix: [ID 918906 kern.notice] I/O to pool 'gold' appears to be hung.
2022-05-14T11:21:12.937126+00:00 srv-smartos-1 unix: [ID 100000 kern.notice] #012
2022-05-14T11:21:12.937128+00:00 srv-smartos-1 genunix: [ID 655072 kern.notice] fffffe007a51ba00 zfs:vdev_deadman+108 ()
2022-05-14T11:21:12.937131+00:00 srv-smartos-1 genunix: [ID 655072 kern.notice] fffffe007a51ba50 zfs:vdev_deadman+43 ()
2022-05-14T11:21:12.937133+00:00 srv-smartos-1 genunix: [ID 655072 kern.notice] fffffe007a51baa0 zfs:vdev_deadman+43 ()
2022-05-14T11:21:12.937135+00:00 srv-smartos-1 genunix: [ID 655072 kern.notice] fffffe007a51bad0 zfs:spa_deadman+ab ()
2022-05-14T11:21:12.937138+00:00 srv-smartos-1 genunix: [ID 655072 kern.notice] fffffe007a51bb60 genunix:cyclic_softint+e2 ()
2022-05-14T11:21:12.937140+00:00 srv-smartos-1 genunix: [ID 655072 kern.notice] fffffe007a51bb80 unix:cbe_low_level+16 ()
2022-05-14T11:21:12.937152+00:00 srv-smartos-1 genunix: [ID 655072 kern.notice] fffffe007a51bbd0 unix:av_dispatch_softvect+72 ()
2022-05-14T11:21:12.937155+00:00 srv-smartos-1 genunix: [ID 655072 kern.notice] fffffe007a51bc00 apix:apix_dispatch_softint+35 ()
2022-05-14T11:21:12.937158+00:00 srv-smartos-1 genunix: [ID 655072 kern.notice] fffffe007a4ca9e0 unix:switch_sp_and_call+15 ()
2022-05-14T11:21:12.937160+00:00 srv-smartos-1 genunix: [ID 655072 kern.notice] fffffe007a4caa30 apix:apix_do_softint+60 ()
2022-05-14T11:21:12.937162+00:00 srv-smartos-1 genunix: [ID 655072 kern.notice] fffffe007a4caaa0 apix:apix_do_interrupt+298 ()
2022-05-14T11:21:12.937165+00:00 srv-smartos-1 genunix: [ID 655072 kern.notice] fffffe007a4caab0 unix:cmnint+c3 ()
2022-05-14T11:21:12.937167+00:00 srv-smartos-1 genunix: [ID 655072 kern.notice] fffffe007a4caba0 unix:i86_mwait+12 ()
2022-05-14T11:21:12.937170+00:00 srv-smartos-1 genunix: [ID 655072 kern.notice] fffffe007a4cabe0 unix:cpu_idle_mwait+142 ()
2022-05-14T11:21:12.937172+00:00 srv-smartos-1 genunix: [ID 655072 kern.notice] fffffe007a4cac00 unix:idle+ae ()
2022-05-14T11:21:12.937175+00:00 srv-smartos-1 genunix: [ID 655072 kern.notice] fffffe007a4cac10 unix:thread_start+b ()
2022-05-14T11:21:12.937176+00:00 srv-smartos-1 unix: [ID 100000 kern.notice]
2022-05-14T11:21:12.937179+00:00 srv-smartos-1 genunix: [ID 111219 kern.notice] dumping to /dev/zvol/dsk/zones/dump, offset 65536, content: kernel
2022-05-14T11:21:12.937182+00:00 srv-smartos-1 ahci: [ID 405573 kern.info] NOTICE: ahci1: ahci_tran_reset_dport port 1 reset port
2022-05-14T11:21:12.937185+00:00 srv-smartos-1 genunix: [ID 100000 kern.notice]
2022-05-14T11:21:12.937187+00:00 srv-smartos-1 genunix: [ID 665016 kern.notice] #015100% done: 2245708 pages dumped,
2022-05-14T11:21:12.937190+00:00 srv-smartos-1 genunix: [ID 851671 kern.notice] dump succeeded
2022-05-14T11:21:12.937194+00:00 srv-smartos-1 genunix: [ID 107833 kern.notice] #015illumos Version joyent_20220505T001410Z 64-bit

Files

fmdump_eV.out.zip (900 KB) fmdump_eV.out.zip Olivia Mocellini Merbeller, 2022-05-14 12:44 PM

No data to display

Actions

Also available in: Atom PDF