Bug #8381
Convert ipsec_alg_lock from mutex to rwlock
100%
Description
This is an early step of rewhacking some bits of IPsec. Some of the early attempted code included this particular bugfix, which is not a large performance win, but does keep exclusion out of IPsec algorithm instances except in cases of ipsecalgs(1M) instantiations.
Original work done by Bayard Bell.
Files
Updated by Dan McDonald over 3 years ago
Initial testing.
1.) Find address of global netstack's ipsec_alg_lock. Using mdb -k...
a.) ::netstack
b.) <global-netstack>::print netstack_t
c.) <ru_ipsec>::print -at ipsec_stack_t ipsec_alg_lock
Pre-this-fix it's a mutex, post-this-fix it's an rwlock.
Then use lockstat -A...
MUTEX VERSION:
# lockstat -A -l 0xffffff025c1aa598,8 ipsecalgs -s Adaptive mutex hold: 3 events in 0.069 seconds (44 events/sec) Count indv cuml rcnt nsec Lock Caller ------------------------------------------------------------------------------- 1 33% 33% 0.00 127641 0xffffff025c1aa598 spdsock_merge_algs+0x2e3 1 33% 67% 0.00 4289 0xffffff025c1aa598 esp_register_out+0x2b4 1 33% 100% 0.00 3579 0xffffff025c1aa598 ah_register_out+0x1fe ------------------------------------------------------------------------------- # lockstat -A -l 0xffffff025c1aa598,8 ipsecalgs -l | tail -4 Count indv cuml rcnt nsec Lock Caller ------------------------------------------------------------------------------- 1 100% 100% 0.00 22319 0xffffff025c1aa598 spdsock_dumpalgs+0x34a ------------------------------------------------------------------------------- #
RWLOCK VERSION:
# lockstat -A -l 0xffffff025c01d598,8 ipsecalgs -s R/W writer hold: 1 events in 0.069 seconds (14 events/sec) Count indv cuml rcnt nsec Lock Caller ------------------------------------------------------------------------------- 1 100% 100% 0.00 125397 0xffffff025c01d598 spdsock_merge_algs+0x2e3 ------------------------------------------------------------------------------- R/W reader hold: 2 events in 0.069 seconds (29 events/sec) Count indv cuml rcnt nsec Lock Caller ------------------------------------------------------------------------------- 1 50% 50% 0.00 5116 0xffffff025c01d598 ah_register_out+0x1fe 1 50% 100% 0.00 4496 0xffffff025c01d598 esp_register_out+0x2bc ------------------------------------------------------------------------------- # Count indv cuml rcnt nsec Lock Caller ------------------------------------------------------------------------------- 1 100% 100% 0.00 22990 0xffffff025c01d598 spdsock_dumpalgs+0x352 -------------------------------------------------------------------------------
Updated by Dan McDonald over 3 years ago
- File acqspray.sh acqspray.sh added
Further testing provided by the acqspray.sh attached here. (A variant of which may get upstreamed to usr/src/test later.)
Sample output:
MUTEX VERSION:
# ./acqspray.sh WARNING -- this test flushes out IPsec policy... GLOBAL_NETSTACK = ffffff02545b2dc0 GLOBAL_IPSEC = 0xffffff025b08b000 IPSEC_ALG_LOCK = ffffff025b08b598 IPsec policy should be managed using smf(5). Modifying the IPsec policy from the command line while the 'policy' service is enabled could result in an inconsistent security policy. IPsec policy should be managed using smf(5). Modifying the IPsec policy from the command line while the 'policy' service is enabled could result in an inconsistent security policy. WARNING : New policy entries that are being added may affect the existing connections. Existing connections that are not subjected to policy constraints, may be subjected to policy constraints because of the new policy. This can disrupt the communication of the existing connections. #INDEX 13 { raddr 10.21.12.0/24 dir out } ipsec { encr_algs aes-cbc(128..256) encr_auth_algs hmac-sha512(512) sa shared } #INDEX 14 { raddr 10.21.12.0/24 dir in } ipsec { encr_algs aes-cbc(128..256) encr_auth_algs hmac-sha512(512) sa shared } Launching in.iked temporarily, using /dev/null as config file. Running: lockstat -A -l 0xffffff025b08b598,8 sleep 30 Adaptive mutex hold: 765 events in 30.077 seconds (25 events/sec) Count indv cuml rcnt nsec Lock Caller ------------------------------------------------------------------------------- 510 67% 67% 0.00 1672 0xffffff025b08b598 sadb_new_algdesc+0x134 255 33% 100% 0.00 26694 0xffffff025b08b598 esp_send_acquire+0xfb ------------------------------------------------------------------------------- #
RWLOCK VERSION:
# ./acqspray.sh WARNING -- this test flushes out IPsec policy... GLOBAL_NETSTACK = ffffff02545b2dc0 GLOBAL_IPSEC = 0xffffff025be4b000 IPSEC_ALG_LOCK = ffffff025be4b598 IPsec policy should be managed using smf(5). Modifying the IPsec policy from the command line while the 'policy' service is enabled could result in an inconsistent security policy. IPsec policy should be managed using smf(5). Modifying the IPsec policy from the command line while the 'policy' service is enabled could result in an inconsistent security policy. WARNING : New policy entries that are being added may affect the existing connections. Existing connections that are not subjected to policy constraints, may be subjected to policy constraints because of the new policy. This can disrupt the communication of the existing connections. #INDEX 7 { raddr 10.21.12.0/24 dir out } ipsec { encr_algs aes-cbc(128..256) encr_auth_algs hmac-sha512(512) sa shared } #INDEX 8 { raddr 10.21.12.0/24 dir in } ipsec { encr_algs aes-cbc(128..256) encr_auth_algs hmac-sha512(512) sa shared } Launching in.iked temporarily, using /dev/null as config file. Running: lockstat -A -l 0xffffff025be4b598,8 sleep 30 R/W reader hold: 765 events in 30.085 seconds (25 events/sec) Count indv cuml rcnt nsec Lock Caller ------------------------------------------------------------------------------- 510 67% 67% 0.00 1611 0xffffff025be4b598 sadb_new_algdesc+0x123 255 33% 100% 0.00 24576 0xffffff025be4b598 esp_send_acquire+0xfb ------------------------------------------------------------------------------- #
Updated by Electric Monk over 3 years ago
- Status changed from New to Closed
- % Done changed from 0 to 100
git commit 69e7133115efafd5527a1c8b7735b0e8937633f6
commit 69e7133115efafd5527a1c8b7735b0e8937633f6 Author: Bayard Bell <buffer.g.overflow@gmail.com> Date: 2017-06-17T02:42:11.000Z 8381 Convert ipsec_alg_lock from mutex to rwlock Reviewed by: Dan McDonald <danmcd@joyent.com> Reviewed by: Gordon Ross <gordon.w.ross@gmail.com> Approved by: Richard Lowe <richlowe@richlowe.net>