illumos gate

The SMB2 stack attempts to validate the Security Mode passed by the client by comparing it to its own Security Mode and if the two are not consistent it fails with STATUS_INVALID_PARAMETER and closes the socket:

/*
 * Negotiation itself.  First the Security Mode.
 */
secmode = SMB2_NEGOTIATE_SIGNING_ENABLED;
if (sr->sr_cfg->skc_signing_required) {
    secmode |= SMB2_NEGOTIATE_SIGNING_REQUIRED;
    /* Make sure client at least enables signing. */
    if ((s->cli_secmode & secmode) == 0) {
        sr->smb2_status = NT_STATUS_INVALID_PARAMETER;
    }
}

The Security Mode is not meant to be used this way. It is only meant as a way for the client to inform the server that it has signing enabled or requires signing. Or at least that is my interpretation of the SMB2 spec. This is also consistent with what Samba does.

Fix: https://www.illumos.org/rb/r/2287/

Also attached are the results from Protocol Test Manager version 3.19.9.0 before and after the change, and a playlist that can be used to reproduce the results.