Project

General

Profile

Feature #9595

rtld should conditionally save AVX-512 state

Added by Robert Mustacchi over 2 years ago. Updated over 2 years ago.

Status:
Closed
Priority:
Normal
Category:
lib - userland libraries
Start date:
2018-06-13
Due date:
% Done:

100%

Estimated time:
Difficulty:
Medium
Tags:
Gerrit CR:

Description

One of the challenges with the AVX-512 state is that using AVX-512 registers can lead to a known decrease in performance of surrounding code by causing some of the logical cores to downclock while executing AVX-512 related instructions. We've also seen related cases where the processor can be tricked into thinking all FPU activity is AVX-512 related because the AVX-512 register state is considered valid.

One such case where this occurs is in rtld. Currently, rtld will attempt to save and restore the FPU registers when transitioning through the PLT. This is done such that constructors and other code that might be loaded as side effects don't clobber the state of the FPU registers in unexpected ways. This is expected effectively by the amd64 ABI where the first set of registers are callee saved.

What it appears that we need to do instead is to look at is not saving and restoring larger register sizes if they're not in use. For example, the xgetbv instruction when ecx is set to one allows for us to determine which register sets are in use by the process and therefore not save and restore all of them, but rather only save the relevant portion of the register space.

There is an alternative approach that Intel has taken in Linux in recent glibc. There, what they do is actually use the fnsave/xsave/xsavec, etc. to try and get at the information. We've opted to use this approach which will make maintenance here a lot simpler.

History

#1

Updated by Electric Monk over 2 years ago

  • Status changed from New to Closed

git commit d0158222a5936ac26b7a03241b7d2df18cc544c8

commit  d0158222a5936ac26b7a03241b7d2df18cc544c8
Author: Robert Mustacchi <rm@joyent.com>
Date:   2018-06-19T19:34:37.000Z

    9596 Initial xsave xstate_bv should not include all features
    9595 rtld should conditionally save AVX-512 state
    Reviewed by: John Levon <john.levon@joyent.com>
    Reviewed by: Patrick Mooney <patrick.mooney@joyent.com>
    Reviewed by: Jerry Jelinek <jerry.jelinek@joyent.com>
    Reviewed by: Bryan Cantrill <bryan@joyent.com>
    Reviewed by: Toomas Soome <tsoome@me.com>
    Approved by: Richard Lowe <richlowe@richlowe.net>

Also available in: Atom PDF