illumos gate

Doing some experimenting with the kfpu interfaces (in the hopes they might be useful with the in-kernel crypto acceleration), I encountered this panic:

assertion failed: pf->fpu_flags & 0x02 != 0 (0x0 != 0x0), file: ../../intel/ia32/os/fpu.c, line: 1296

Corresponding to:

ASSERT3U(pf->fpu_flags & FPU_VALID, !=, 0);

When the kfpu_state_t instance is first allocated, kernel_fpu_alloc(), kernel_fpu_fpstate_init() is called which sets the FPU_VALID flag. Since we are using our own kfpu state, we call kernel_fpu_begin(ctx->ac_kfpu, 0); and later kernel_fpu_end(ctx->ac_kfpu, 0);.

The first time kernel_fpu_begin() is called for an initialized kfpu_state_t, FPU_VALID is set. The last thing that occurs in this instance is calling kernel_fpu_ctx_restore(). Since FPU_VALID is set, the ASSERT at line 1296 passes, and fp_restore() is called which clears FPU_VALID. This leaves FPU_VALID cleared after the first call to kernel_fpu_begin().

kernel_fpu_end() does not reset FPU_VALID, so the next time kernel_fpu_begin() is called with the same kfpu_state_t instance, FPU_VALID is cleared, and the ASSERT is triggered.