Project

General

Profile

Actions

Bug #14916

open

ehci_qh_pool_size is probably too low

Added by Joshua M. Clulow about 1 month ago. Updated about 1 month ago.

Status:
New
Priority:
Normal
Category:
driver - device drivers
Start date:
Due date:
% Done:

0%

Estimated time:
Difficulty:
Medium
Tags:
Gerrit CR:
External Bug:

Description

On a system with a moderate amount of USB devices, it's possible to exhaust the supply of Endpoint Descriptors (QH) that ehci allocates. The value ehci_qh_pool_size is, by default, set to EHCI_QH_POOL_SIZE which is 100. In ehci_alloc_qh() we can see that the first EHCI_NUM_STATIC_NODES (63) entries are set aside for some ... very ehci... purpose, and that therefore there are only 37 left for activities.

On a system with lots of devices, it appears that we need (at least while those devices are in use) more of these. For example we have discovered now that we are routinely running out of these with a tree like:

jclulow@igor ~ $ prtconf
...
        pci1028,577, instance #0
            hub, instance #0
                hub, instance #5
                    miscellaneous, instance #6
                        interface (driver not attached)
                        storage, instance #7
                            disk, instance #9
                        interface-association, instance #7
                            modem (driver not attached)
                            data (driver not attached)
                    device, instance #8
                        serdev, instance #29
                    miscellaneous, instance #8
                        interface (driver not attached)
                        storage, instance #8
                            disk, instance #8
                        interface-association, instance #8
                            modem (driver not attached)
                            data (driver not attached)
                    miscellaneous, instance #18
                        interface (driver not attached)
                        interface-association, instance #13
                            modem (driver not attached)
                            data (driver not attached)
                hub, instance #4
                    miscellaneous, instance #3
                        interface (driver not attached)
                        storage, instance #4
                            disk, instance #6
                        interface-association, instance #4
                            modem (driver not attached)
                            data (driver not attached)
                    device, instance #6
                        serdev, instance #24
                        serdev, instance #25
                        serdev, instance #26
                        serdev, instance #27
                    device, instance #5
                        serdev, instance #20
                    miscellaneous, instance #13
                        interface (driver not attached)
                        storage, instance #12
                            disk, instance #14
                        interface-association, instance #12
                            modem (driver not attached)
                            data (driver not attached)
...

When this exhaustion occurs, we can see an odd return from a ugen(4D) open:

64486820 R usb_ugen_open(fffffe0bb1435698, e400001004) -> 12 (ENOMEM)

This somewhat upsets the apple cart, and appears to, at times, permanently cork things. There are obviously other bugs here (perhaps in ugen or in the hub driver) but at minimum this resource cap should probably be bumped up.

Actions #1

Updated by Joshua M. Clulow about 1 month ago

Working around this with /etc/system at least appears to improve the situation:

jclulow@igor ~ $ cat /etc/system.d/ehci
set ehci:ehci_qh_pool_size = 1000
Actions #2

Updated by Joshua M. Clulow about 1 month ago

After tuning up to 1000, I confirmed that even at rest we are perilously close to 37 used descriptors on at least one ehci controller in this system:

jclulow@igor ~ $ for n in 0 1; do
        echo instance $n
        pfexec mdb -ke "*ehci_statep::softstate $n | ::print ehci_state_t ehci_qh_pool_addr | ::eval '.::array ehci_qh_t 0t1000 | ::print ehci_qh_t qh_state' ! sort | uniq -c | sort -n" 
        echo
    done
instance 0
  34 qh_state = 0x3
  63 qh_state = 0x2
 903 qh_state = 0x1

instance 1
   2 qh_state = 0x3
  63 qh_state = 0x2
 935 qh_state = 0x1
Actions

Also available in: Atom PDF