Project

General

Profile

Bug #8035

TSC sync detection should be NUMA friendly

Added by Patrick Mooney over 3 years ago. Updated over 3 years ago.

Status:
Closed
Priority:
Normal
Category:
kernel
Start date:
2017-04-04
Due date:
% Done:

100%

Estimated time:
Difficulty:
Medium
Tags:
needs-triage
Gerrit CR:

Description

On i86pc, certain precautions must be taken before using the TSC as a source for gethrtime. It's possible that the TSC will be skewed between different CPUs in a system. To compensate for that fact, logic attempts to measure any potential skew.

The algorithm used to make this measurement is sensitive to memory write latency between each pair of cores under consideration. The current implementation chooses to activate the delta-sensitive gethrtime routines if the largest TSC delta is greater than the shortest write latency. While this was adequate on older SMP systems with a shared memory bus, it poses a challenge in the face of NUMA. On such systems, large TSC skews detected are often accompanied by long memory write times (like when the compared core is off-socket).

In order to make an accurate assessment of TSC skew on NUMA systems, the global tsc_max_delta and write_time_min should be eschewed for thresholds which are set per-CPU. This will allow systems with synchronized TSCs to avoid the cost of the "delta variant" gethrtime routines.

Upstreaming: Joyent OS-5309

History

#1

Updated by Electric Monk over 3 years ago

  • Status changed from New to Closed
  • % Done changed from 0 to 100

git commit 86cb0be22449bbfe13a5d6679a60201c4b258fdf

commit  86cb0be22449bbfe13a5d6679a60201c4b258fdf
Author: Patrick Mooney <pmooney@pfmooney.com>
Date:   2017-04-06T18:10:34.000Z

    8035 TSC sync detection should be NUMA friendly
    Reviewed by: Jerry Jelinek <jerry.jelinek@joyent.com>
    Reviewed by: Robert Mustacchi <rm@joyent.com>
    Approved by: Dan McDonald <danmcd@omniti.com>

Also available in: Atom PDF