Project

General

Profile

Actions

Bug #6741

open

CIFS ignores TCP buffers settings after a few hours

Added by Dave FL over 5 years ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
networking
Start date:
2016-03-10
Due date:
% Done:

0%

Estimated time:
Difficulty:
Medium
Tags:
needs-triage
Gerrit CR:

Description

I am using adapters that require my TCP send and receive buffers to be set to a value different from the defaults. They are set as follows:

tcp max_buf rw 4194304 4194304 1048576 8192-1073741824
tcp recv_buf rw 1048576 1048576 128000 2048-4194304
tcp send_buf rw 1048576 1048576 49152 4096-4194304

What I am noticing is that my transfers after an immediate CIFS restart will allow my client downloads to hit max speeds but after a few hours these values will degrade significantly. It appears that somehow down the line CIFS does not enforce a new or existing connection to use the default buffer size.

Below are the results from dtrace. You can see how the window changes completely after a restart but the issue is that this value does not persist.

#!/usr/sbin/dtrace -s

#pragma D option quiet

tcp:::send
/ (args4->tcp_flags & (TH_SYN|TH_RST|TH_FIN)) == 0 / {
@unacked["unacked(bytes)", args2->ip_daddr, args4->tcp_sport] =
quantize(args3->tcps_snxt - args3->tcps_suna);
}

tcp:::receive
/ (args4->tcp_flags & (TH_SYN|TH_RST|TH_FIN)) == 0 / {
@swnd["SWND", args2->ip_saddr, args4->tcp_dport] =
quantize((args4->tcp_window)*(1 << args3->tcps_snd_ws));

}

oot@storage1:/root# dtrace -s tcp_tput.d
^C

unacked(bytes)                                      10.250.0.3                                          2049
value ------------- Distribution ------------- count
32 | 0
64 |@@@@@@@@@@@@@@@@@@@@ 11
128 | 0
unacked(bytes)                                      10.250.0.2                                          2049
value ------------- Distribution ------------- count
-1 | 0
0 |@@@@ 63
1 | 0
2 | 0
4 | 0
8 | 0
16 | 0
32 | 0
64 | 5
128 |
@@@@@@@@@@@ 195
256 |
@@@ 54
512 |@ 13
1024 |
5
2048 | 0
unacked(bytes)                                      10.255.0.55                                          445
value ------------- Distribution ------------- count
-1 | 0
0 | 7
1 | 0
2 | 0
4 | 0
8 | 0
16 | 0
32 | 10
64 | 48
128 | 9
256 | 0
512 | 0
1024 | 1
2048 | 0
4096 | 0
8192 | 0
16384 | 2
32768 |@@@@@@@@@@@@@@@@@@@@ 46626
65536 | 6
131072 | 0
SWND(bytes)                                         10.255.0.55                                           22
value ------------- Distribution ------------- count
16384 | 0
32768 |@@@@@@@@@@@@@@@@@@@@ 1
65536 | 0
SWND(bytes)                                         10.250.0.3                                          2049
value ------------- Distribution ------------- count
32768 | 0
65536 |@@@@@@@@@@@@@@@@@@@@ 17
131072 | 0
SWND(bytes)                                         10.250.0.2                                          2049
value ------------- Distribution ------------- count
131072 | 0
262144 |@@@@@@@@@@@@@@@@@@@@ 369
524288 | 0
SWND(bytes)                                         10.255.0.55                                          445
value ------------- Distribution ------------- count
16384 | 0
32768 | 367
65536 |@@@@@@@@@@@@@@@@@@@@ 49052
131072 | 0

Good

root@storage1:/root# svcadm restart smb/server
root@storage1:/root# dtrace -s tcp_tput.d
^C

unacked(bytes)                                      10.255.0.55                                           22
value ------------- Distribution ------------- count
32 | 0
64 |@@@@@@@@@@@@@@@@@@@@ 1
128 | 0
unacked(bytes)                                      10.250.0.3                                          2049
value ------------- Distribution ------------- count
32 | 0
64 |@@@@@@@@@@@@@@@@@@@@ 11
128 | 0
unacked(bytes)                                      10.250.0.2                                          2049
value ------------- Distribution ------------- count
-1 | 0
0 |@@@ 23
1 | 0
2 | 0
4 | 0
8 | 0
16 | 0
32 | 0
64 | 4
128 |
@@@@@@@@@@@@ 84
256 |@@ 12
512 | 4
1024 |
@ 6
2048 | 0
unacked(bytes)                                      10.255.0.55                                          445
value ------------- Distribution ------------- count
-1 | 0
0 | 9
1 | 0
2 | 0
4 | 0
8 | 0
16 | 0
32 | 10
64 | 50
128 | 9
256 | 0
512 | 0
1024 | 1
2048 | 0
4096 | 0
8192 | 0
16384 | 0
32768 | 1
65536 | 2
131072 | 260
262144 |@@@@@@@@@@@@@@@@@@@@ 27135
524288 | 0
SWND(bytes)                                         10.255.0.55                                           22
value ------------- Distribution ------------- count
16384 | 0
32768 |@@@@@@@@@@@@@@@@@@@@ 1
65536 | 0
SWND(bytes)                                         10.250.0.3                                          2049
value ------------- Distribution ------------- count
32768 | 0
65536 |@@@@@@@@@@@@@@@@@@@@ 17
131072 | 0
SWND(bytes)                                         10.250.0.2                                          2049
value ------------- Distribution ------------- count
131072 | 0
262144 |@@@@@@@@@@@@@@@@@@@@ 155
524288 | 0
SWND(bytes)                                         10.255.0.55                                          445
value ------------- Distribution ------------- count
16384 | 0
32768 | 54
65536 | 63
131072 | 306
262144 | 330
524288 |@@@@@@@@@@@@@@@@@ 210636
1048576 |@@@ 28037
2097152 | 0

No data to display

Actions

Also available in: Atom PDF