Project

General

Profile

Actions

Bug #13990

closed

nfssrv: get rid of one crget for nfsv4

Added by Vitaliy Gusev 10 months ago. Updated 4 months ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
nfs - NFS server and client
Start date:
Due date:
% Done:

100%

Estimated time:
Difficulty:
Medium
Tags:
Gerrit CR:

Description

Before fixed #13848 there were 3 crdup/crget calls per NFSv4 request (in general) and one for NFSv3.

After fix of #13848 there are still 2 crdup/crget calls per NFSv4 request.

vetal@oi151:/codepool$ sudo dtrace -n '::common_dispatch:entry,::crget:entry,::crdup:entry{@[probefunc]=count();}'
dtrace: description '::common_dispatch:entry,::crget:entry,::crdup:entry' matched 3 probes
^C
  common_dispatch                                               86942
  crdup                                                         86942
  crget                                                         86951

Each crget/crdup call is bottleneck as it calls zone_cred_hold() that takes single zone_lock. And heavy workload on many cores (more that 100 and 500K iops) shows CPU usage in mutex spinning.

Result of fixing that issue: no crget() calls per NFSv4 request.

Actions #1

Updated by Marcel Telka 10 months ago

  • Gerrit CR set to 1629
Actions #2

Updated by Adam Stylinski 5 months ago

Ping on this? We were seeing some NFS performance issues a while back while hitting a particularly heavy random read (namely a few million small files) a while back.

Actions #3

Updated by Vitaliy Gusev 5 months ago

Adam Stylinski wrote in #note-2:

Ping on this? We were seeing some NFS performance issues a while back while hitting a particularly heavy random read (namely a few million small files) a while back.

Adam Stylinski, Adam, could you check current PR? Does it solve your performance issue?

Link to PR:
https://code.illumos.org/c/illumos-gate/+/1629/3

Actions #4

Updated by Vitaliy Gusev 5 months ago

Result of testing.

For NFSv4
without fix from review for ozone workload (iozone -A -g 1g -r 64K -I) dtrace script shows:

  common_dispatch                                    131408
  crdup                                              131408
  crget                                              131410

with fix:

  crget                                                  14
  crdup                                              131410
  common_dispatch                                    131416

NFSv3 result is the same as expected:

  crget                                                    3
  common_dispatch                                     131305

So after fix, crget() call is eliminated, but crdup() is still present and should be resolved in another ticket.

Actions #5

Updated by Electric Monk 4 months ago

  • Status changed from New to Closed
  • % Done changed from 0 to 100

git commit 214d537cc1e7896ae3489ebec3ae766e01f32b2a

commit  214d537cc1e7896ae3489ebec3ae766e01f32b2a
Author: Vitaliy Gusev <gusev.vitaliy@gmail.com>
Date:   2022-01-04T23:28:15.000Z

    13990 nfssrv: get rid of one crget for nfsv4
    Reviewed by: Dan McDonald <danmcd@joyent.com>
    Reviewed by: Evan Layton <elayton@tintri.com>
    Reviewed by: Marcel Telka <marcel@telka.sk>
    Approved by: Robert Mustacchi <rm@fingolfin.org>

Actions

Also available in: Atom PDF