illumos gate

Most of the PKCS#11 crypto functions have a form similar to:

C_xxxxInit() - initialize/start the operation (encrypt, decrypt, digest, etc). The mechanism parameters (mechanism to use, keys, etc.) are provided as a parameter to this call.
C_xxxUpdate() - Perform the operations (encrypt, decrypt, ...) as a multi-part operation. This can be called 0 or more times to add more data to process.
C_xxxFinal() - Complete the operation.
C_xxx() - Perform the operation as a single operation. This is equivalent to calling C_xxxUpdate() and then C_xxxFinal().

Once an operation has been initialized, it is an error to attempt to initialize the same operation again until the operation has been terminated. Operations are terminated in two ways:

1. C_xxxFinal() is called
2. C_xxx() (single part) is called.

Note that in both cases, there are two exceptions where the active operation is not terminated:

1. The output buffer passed is NULL and CKR_OK is returned. This is used to allow the caller to ask for the size required to hold the output of the operation.
2. The size of the output buffer passed to the function is not NULL, but is also too small to hold the output. In this instance, CKR_BUFFER_TOO_SMALL is returned, and size of the output buffer required is set in address given by the appropriate function parameter (instead of merely passing the size of the output buffer, these functions instead pass the address of the variable holding the output buffer size -- e.g. CK_ULONG *pulValueLen to allow the called function to pass back the size required to the caller).

In these two instances, the operation is not terminated, and the function call can be retried. In all other situations, once these functions are called, the operation is terminated.

Additionally, after the operation has been initialized, one cannot switch between single and multi-op in the same operation. E.g. calling C_EncryptInit(), C_EncryptUpdate(), C_Encrypt() is an invalid sequence.

Currently we either return CKR_FUNCTION_FAILED or allow certain invalid sequences of calls. In all of these instances, we should be returning CKR_OPERATION_ACTIVE.