Project

General

Profile

Bug #7658

nfs4: server returns ESTALE if unlink opened filehandle

Added by Vitaliy Gusev over 3 years ago. Updated over 3 years ago.

Status:
New
Priority:
Normal
Assignee:
Category:
nfs - NFS server and client
Start date:
2016-12-09
Due date:
% Done:

0%

Estimated time:
Difficulty:
Medium
Tags:
needs-triage
Gerrit CR:

Description

NFS4 server should support delete-on-last-close functionality as NFS version 4 protocol is stateful.

if clients does:

OPEN file.vmx
RENAME file.vmx~ file.vmx
CLOSE file.vmx

The call with CLOSE operation fails with NFS4ERR_STALE. More precisely it fails on PUTFH operation.

It could be that underlying filesystem should take care about this.


Files

nfs40-close-estale.pcap (122 KB) nfs40-close-estale.pcap traffic between sever and clients Vitaliy Gusev, 2016-12-09 02:04 PM

History

#1

Updated by Marcel Telka over 3 years ago

Please publish an excerpt from the communication showing the problem. Thanks.

#2

Updated by Vitaliy Gusev over 3 years ago

pcap nfs4.0 traffic. 2 clients and 1 server.

client1: 192.168.1.101
client2: 192.168.1.113
server: 192.168.1.108

Steps to reproduce:

1. client1: "less list_pkgs"
2. client2: "mv list_pkgs.mv list_pkgs"
3. client1: quit from less command.

#3

Updated by Marcel Telka over 3 years ago

For this case the 16.26.5. section of RFC 7530 applies. Namely this part of it:

   o  If the file was not opened with OPEN4_SHARE_DENY_WRITE or
      OPEN4_SHARE_DENY_BOTH, the server SHOULD delete the file's
      directory entry.  However, until the last CLOSE of the file, the
      server MAY continue to allow access to the file via its
      filehandle.

IOW, the NFSv4.0 server is free to either allow or do not allow the access to the removed file via its file handle. This also means that clients should not expect to be able to access the removed file via its file handle.

We might change the behavior of the NFSv4 server, but this won't make the NFSv4 server more or less conforming to the RFC. In any case, all clients should be prepared that any file opened without a share reservation could disappear any time and such a file could be no longer accessible via its file handle.

If a client needs to be sure that a file cannot be removed when it is still open by the client, the client should open the file with either OPEN4_SHARE_DENY_WRITE or OPEN4_SHARE_DENY_BOTH.

#4

Updated by Vitaliy Gusev over 3 years ago

Marcel Telka wrote:

For this case the 16.26.5. section of RFC 7530 applies. Namely this part of it:

...

We might change the behavior of the NFSv4 server, but this won't make the NFSv4 server more or less conforming to the RFC. In any case, all clients should be prepared that any file opened without a share reservation could disappear any time and such a file could be no longer accessible via its file handle.

I believe that RFC says in case of OPEN4_SHARE_DENY_WRITE, rename/remove operation MUST fail with NFS4ERR_FILE_OPEN.

And it is good to a server to support delete-on-last-close semantic. At least RFC do not deny that.
Moreover, a linux nfs client and ESX nfs client do open without any SHARE_DENY flags.

If a client needs to be sure that a file cannot be removed when it is still open by the client, the client should open the file with either OPEN4_SHARE_DENY_WRITE or OPEN4_SHARE_DENY_BOTH.

See note above. Also a Linux nfs server handles this fine, i.e. ESX works well with RHEL7 Linux nfs server.

#5

Updated by Marcel Telka over 3 years ago

Vitaliy Gusev wrote:

Marcel Telka wrote:

For this case the 16.26.5. section of RFC 7530 applies. Namely this part of it:

...

We might change the behavior of the NFSv4 server, but this won't make the NFSv4 server more or less conforming to the RFC. In any case, all clients should be prepared that any file opened without a share reservation could disappear any time and such a file could be no longer accessible via its file handle.

I believe that RFC says in case of OPEN4_SHARE_DENY_WRITE, rename/remove operation MUST fail with NFS4ERR_FILE_OPEN.

Yes. But you opened the file with OPEN4_SHARE_DENY_NONE. See packet 159 in the capture file. So you should not expect the file handle will be valid all the time while you have the file open.

And it is good to a server to support delete-on-last-close semantic. At least RFC do not deny that.

As I said, and as RFC says: the server MAY support it. So this feature is purely optional. If you want, you can implement it to help a non-conforming clients (or clients with strange expectations) to behave better. But the better way is to fix the client (provided there is really a bug in the NFS client), or change it's configuration somehow to avoid such strange expectations.

Moreover, a linux nfs client and ESX nfs client do open without any SHARE_DENY flags.

Then they must be prepared that the file (and its file handle) might disappear. They should not expect that the file handle will be valid after the file remove.

If a client needs to be sure that a file cannot be removed when it is still open by the client, the client should open the file with either OPEN4_SHARE_DENY_WRITE or OPEN4_SHARE_DENY_BOTH.

See note above. Also a Linux nfs server handles this fine, i.e. ESX works well with RHEL7 Linux nfs server.

That's okay. But you need to be prepared that the linux nfs server might change anytime to stop supporting it.

#7

Updated by Vitaliy Gusev over 3 years ago

Marcel Telka wrote:

See note above. Also a Linux nfs server handles this fine, i.e. ESX works well with RHEL7 Linux nfs server.

That's okay. But you need to be prepared that the linux nfs server might change anytime to stop supporting it.

Marcel, I opened this issue for server-side not for client-side. And main reason is that ESX-client wants to work with fully stateful delete-on-last-close NFS4 server.
We cannot change ESX-client because this client is closed and it works fine with other NFS4 servers. And we can choose only way to improve illumos NFS server to satisfy ESX-client, in respect that it does not contravene the RFC-s.

#8

Updated by Marcel Telka over 3 years ago

Vitaliy Gusev wrote:

Marcel Telka wrote:

See note above. Also a Linux nfs server handles this fine, i.e. ESX works well with RHEL7 Linux nfs server.

That's okay. But you need to be prepared that the linux nfs server might change anytime to stop supporting it.

Marcel, I opened this issue for server-side not for client-side. And main reason is that ESX-client wants to work with fully stateful delete-on-last-close NFS4 server.
We cannot change ESX-client because this client is closed and it works fine with other NFS4 servers. And we can choose only way to improve illumos NFS server to satisfy ESX-client, in respect that it does not contravene the RFC-s.

Yes, you are right. The only way how to make sure the unix semantics for NFSv4 clients is implemented is to do what you are suggesting at the NFSv4 server.

The RFC 7530 is too vague and insufficient re the topic:
https://mailarchive.ietf.org/arch/msg/nfsv4/RfU4UvKcxl8xHLFxfIamUJwh9J8

The main problem with our implementation seems to be that functions like zfs_vget() fails to translate file handle to vnode for deleted (but still open) files. So we should either change zfs_vget() and friends or implement some workaround in the NFSv4 server.

Also available in: Atom PDF