Project

General

Profile

Feature #11083

support NFS server in zone

Added by Jorge Schrauwen over 1 year ago. Updated 9 months ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
nfs - NFS server and client
Start date:
Due date:
% Done:

0%

Estimated time:
Difficulty:
Hard
Tags:
nfs-zone
Gerrit CR:

Description

This comes up from time to time, it would be nice to be able to run an nfs server inside a zone.
It looks like Nexenta added support for this a while ago, as NEX-15279.

https://github.com/Nexenta/illumos-nexenta/commit/ba0b8b57c486aa2559b813f219899a5077e01c01

It would be nice if we could pull this in, not sure if this is all of it or that we are missing other commits.
This also seems to fix sharenfs to work inside a zone, so that so maybe that can be used to get sharesmb to work too... but that is a whole other subject.


Files

runit.bash (964 Bytes) runit.bash Dan McDonald, 2019-11-27 07:08 PM
nfs-zone-in-zone-smartos.txt (175 KB) nfs-zone-in-zone-smartos.txt Dan McDonald, 2019-12-02 06:08 PM
nfs-zone-global-smartos.txt (177 KB) nfs-zone-global-smartos.txt Dan McDonald, 2019-12-02 06:08 PM
stock-global-smartos.txt (177 KB) stock-global-smartos.txt Dan McDonald, 2019-12-02 06:08 PM
nfs-zone-in-zone-omnios.txt (175 KB) nfs-zone-in-zone-omnios.txt Dan McDonald, 2019-12-03 05:05 AM
nfs-zone-global-omnios.txt (175 KB) nfs-zone-global-omnios.txt Dan McDonald, 2019-12-03 05:05 AM
stock-global-omnios.txt (175 KB) stock-global-omnios.txt Dan McDonald, 2019-12-03 05:05 AM

Related issues

Related to illumos gate - Bug #12294: rfs3_readdir()/rfs3_readdirplus(): Duplicate vattr_to_post_op_attr() callClosedMarcel Telka

Actions
Related to illumos gate - Bug #12278: nfs-zone needs man page changesClosedDan McDonald

Actions
Related to illumos gate - Bug #12300: Memory leak in rfs3_readdirplus()ClosedMarcel Telka

Actions
Related to illumos gate - Feature #12303: nfs/nfs4_drc.h: No such file or directoryClosedAndy Fiddaman

Actions
Related to illumos gate - Bug #12505: Answer KEBE question about cred in unexport()ClosedDan McDonald

Actions
#1

Updated by Dan McDonald about 1 year ago

  • Tags changed from needs-triage to needs-triage, nfs-zone

Tagging with nfs-zone

#2

Updated by Dan McDonald 12 months ago

  • Category set to nfs - NFS server and client
  • Assignee set to Dan McDonald
  • Priority changed from Low to Normal
#3

Updated by Dan McDonald 12 months ago

Many commits from illumos-nexenta, plus more from testing, comprise what will be this. Some of the commits have been spun off into their own bugs (e.g. #2988).

My webrevs directory has a bunch of versions and incremental webrevs: http://kebe.com/~danmcd/webrevs/nfs-zone/ (Note: in the future this will move into the "old/" subdirectory, i.e. http://kebe.com/~danmcd/webrevs/old/nfs-zone/ ).

Included here from illumos-nexenta are (at least initially):

NEX-15279 support NFS server in zone
NEX-15520 online NFS shares cause zoneadm halt to hang in nfs_export_zone_fini
NEX-16502 libshare needs to support SMB in a zone
NEX-16452 NFS server in a zone state database needs to be per zone
NEX-16712 NFS dtrace providers do not support per-share filtering
NEX-16812 Timing window where dtrace probe could try to access share info after unshared
NEX-17125 NFS: nbmand lock entered but not exited on error path
NEX-18312 Max number of nfsd threads is set too high, thus affecting NFS performance
(Reduces nfsd thread count from 1024 down to 256.)
NEX-18369 SMB & NFS dtrace providers do not support per-zone filtering
NEX-18716 The NFS grace period is too long causing long failover times.
(Changes NFS lock manager grace period from 90secs to 30secs)
NEX-15740 NFS deadlock in rfs4_compound with hundreds of threads waiting for lock owned by rfs4_op_rename
NEX-19178 Changing the NFS export path makes the SMB share offline
NEX-19996 exi_id_get_next() calls should be WRITER locked
NEX-20014 NFS v4 state lock mutex exited before entered (on error path)
NEX-20260 NFS hung in transitional state when RSF marks it maintenance
NEX-20423 NFSv4 state database entry locking is not always used around reference count.
NEX-18681 NFS does not quiesce client state to the pool
(Changes NFS lock manager grace period AGAIN, from 30secs to 60secs)

Additional fixes post-project-commencement include:

- Try to remove assumption that zone's root vnode is marked VROOT
- curzone reality check and teardown changes to use the RIGHT zone
- nfs_export_zone_init() can't assume called in zone-context.
- NFSv4 may zone-cleanup before exports do.
- Fix nlm_unexport
- Fix NFS design problems re. multiple zone keys
- Make NFS server zone-specific data all have the same lifetime
- Fix rfs4_clean_state_exi
- Fix exi_cache_reclaim
- Fix mistakes in zone keys work
- More fixes re. exi_zoneid and exi_tree
- Be far more judicious in the use of curzone-using macros.
- Untrip aggressive assert AND use EXI_TO_ZONEROOTVP
- Send zone's rootvp to untraverse()
- Plug memory leaks:
- Plug sharefs zone-shutdown leaks
- Go ahead and destroy the NFSv4 database tables
- Do rfs4_ss_fini() BEFORE rfs4_servinst_destroy_all().
- dss_paths[] entries need cleanup too
- NFS Auth per-zone needs better cleanup
- Two NLM fixes: use zone_kcred() and plug cl_auth leaks
- Zone ID only matches once, don't loop after a match
- Caution with use after exi_rele()

#4

Updated by Dan McDonald 11 months ago

Testing notes 1/N:

Since this project's inception: smoke & mild-stress testing including:

- Doing simple operations via MacOS (NFSv3) client using automount paths.
- Doing simple operations via illumos (NFSv4) client using automount paths.
- Building illumos-gate where illumos-gate is using the same aforementioed NFSv4 client.

#5

Updated by Dan McDonald 11 months ago

Testing notes 2/N:

On a Centos BHYVE VM, I installed the "nfstest" YUM package. Because the package supports NFSv4.1 as well, I needed to run a subset of the tests. The attached bash script will run all of the single-client tests from Centos to illumos. Non-global zone (NGZ) tests merely require using the IP address of the NGZ in question.

The client setup should have a sudo-with-no-password regular user, and a remote filesystem whose owner is said regular user (i.e. same uid).

For my tests, I will run it three times for a given deployment (e.g. SmartOS, OmniOS, OI).

1.) "stock" version (pre-this-change) served from global-zone.
2.) This version served from global-zone.
3.) This version served from NGZ.

#6

Updated by Dan McDonald 11 months ago

SmartOS showed no differences in global-zone response to the Linux NFS test suite, and its non-global-zone had more passes than the global zone (possibly because of the lack of user "danmcd" in the SmartOS global zone, or SmartOS's more stringent global zone policies).

#7

Updated by Dan McDonald 11 months ago

OmniOS, which in my test setup had a user "danmcd" in the global zone, passed equally as much through all three cases, which matched the most-passes SmartOS NGZ case.

#8

Updated by Dan McDonald 10 months ago

A few constant-changes arrived with this wad of fixes from Nexenta. The comment listing the Nexenta bug list should have indicators about them now.

#9

Updated by Dan McDonald 9 months ago

Note that this issue implements the majority of IPD 11 .

#10

Updated by Vitaliy Gusev 9 months ago

pynfs test-suite with

./testserver.py  $SERVER_IP:/data --maketree --rundep all

Shows the same results for pre-change-global-zone and nfs-in-zone cases:

Command line asked for 587 of 672 tests
Of those: 95 Skipped, 47 Failed, 4 Warned, 441 Passed
#11

Updated by Dan McDonald 9 months ago

  • Status changed from New to In Progress
  • % Done changed from 0 to 70
  • Tags changed from needs-triage, nfs-zone to nfs-zone
#12

Updated by Electric Monk 9 months ago

git commit 0dfe541e13279d277d838d6a27e55188b9486cb1

commit  0dfe541e13279d277d838d6a27e55188b9486cb1
Author: Evan Layton <evan.layton@nexenta.com>
Date:   2020-02-07T17:52:15.000Z

    11083 support NFS server in zone
    Portions contributed by: Dan Kruchinin <dan.kruchinin@nexenta.com>
    Portions contributed by: Stepan Zastupov <stepan.zastupov@gmail.com>
    Portions contributed by: Joyce McIntosh <joyce.mcintosh@nexenta.com>
    Portions contributed by: Mike Zeller <mike@mikezeller.net>
    Portions contributed by: Dan McDonald <danmcd@joyent.com>
    Portions contributed by: Gordon Ross <gordon.w.ross@gmail.com>
    Portions contributed by: Vitaliy Gusev <gusev.vitaliy@gmail.com>
    Reviewed by: Rick McNeal <rick.mcneal@nexenta.com>
    Reviewed by: Rob Gittins <rob.gittins@nexenta.com>
    Reviewed by: Sanjay Nadkarni <sanjay.nadkarni@nexenta.com>
    Reviewed by: Jason King <jbk@joyent.com>
    Reviewed by: C Fraire <cfraire@me.com>
    Approved by: Garrett D'Amore <garrett@damore.org>

#13

Updated by Dan McDonald 9 months ago

  • Status changed from In Progress to Feedback
#14

Updated by Dan McDonald 9 months ago

  • Status changed from Feedback to In Progress
#15

Updated by Dan McDonald 9 months ago

  • Status changed from In Progress to New
#16

Updated by Dan McDonald 9 months ago

  • Status changed from New to In Progress
#17

Updated by Marcel Telka 9 months ago

  • Related to Bug #12294: rfs3_readdir()/rfs3_readdirplus(): Duplicate vattr_to_post_op_attr() call added
#18

Updated by Marcel Telka 9 months ago

  • Related to Bug #12278: nfs-zone needs man page changes added
#19

Updated by Marcel Telka 9 months ago

  • Status changed from In Progress to Closed
#20

Updated by Marcel Telka 9 months ago

  • Related to Bug #12300: Memory leak in rfs3_readdirplus() added
#21

Updated by Andy Fiddaman 9 months ago

  • Related to Feature #12303: nfs/nfs4_drc.h: No such file or directory added
#22

Updated by Marcel Telka 7 months ago

  • Related to Bug #12505: Answer KEBE question about cred in unexport() added

Also available in: Atom PDF