TSC sync detection should be NUMA friendly
On i86pc, certain precautions must be taken before using the TSC as a source for gethrtime. It's possible that the TSC will be skewed between different CPUs in a system. To compensate for that fact, logic attempts to measure any potential skew.
The algorithm used to make this measurement is sensitive to memory write latency between each pair of cores under consideration. The current implementation chooses to activate the delta-sensitive gethrtime routines if the largest TSC delta is greater than the shortest write latency. While this was adequate on older SMP systems with a shared memory bus, it poses a challenge in the face of NUMA. On such systems, large TSC skews detected are often accompanied by long memory write times (like when the compared core is off-socket).
In order to make an accurate assessment of TSC skew on NUMA systems, the global tsc_max_delta and write_time_min should be eschewed for thresholds which are set per-CPU. This will allow systems with synchronized TSCs to avoid the cost of the "delta variant" gethrtime routines.
Upstreaming: Joyent OS-5309
Updated by Electric Monk over 5 years ago
- Status changed from New to Closed
- % Done changed from 0 to 100
commit 86cb0be22449bbfe13a5d6679a60201c4b258fdf Author: Patrick Mooney <firstname.lastname@example.org> Date: 2017-04-06T18:10:34.000Z 8035 TSC sync detection should be NUMA friendly Reviewed by: Jerry Jelinek <email@example.com> Reviewed by: Robert Mustacchi <firstname.lastname@example.org> Approved by: Dan McDonald <email@example.com>