illumos gate

My first attempt to fix this caused a problem when a thread from a
dynamic taskq attempted a door_upcall (which smbsrv uses).

My change to taskq_bucket_extend() naively used the same
logic as taskq_thread_create(). Unfortunately the way that
taskq_bucket_extend creates the thread in TS_STOP state
and later starts it ends up leaving the TP_HOLDLWP flag set
in t_proc_flag when we create and start a new LWP.

Later, we end up in an endless loop with a stack like this:

  (typicall some interrupt)
  ffffff0006436140 0xf()
  ffffff00064362a0 stop+0x26()
  ffffff0006436350 issig_forreal+0x38c()
  ffffff0006436370 issig+0x3d(0)
  ffffff0006436420 door_upcall+0x4bf(ffffff0174dd0200, ffffff00064364b0, 0, ffffffffffffffff, 0)
  ffffff00064364a0 door_ki_upcall_limited+0x67(ffffff0160bc7be8, ffffff00064364b0, 0, ffffffffffffffff, 0)
  ffffff0006436510 smb_kdoor_upcall_private+0x65(ffffff0176ed7580, ffffff0006436530)
  ffffff00064365f0 smb_kdoor_send+0x73(ffffff0176ed7580, ffffff0006436610)
  ffffff0006436710 smb_kdoor_upcall+0x1c7(ffffff0176ed7580, 2, ffffff0006436780, fffffffff81c4060, ffffff0174228958, fffffffff81c43e0)
  ffffff0006436750 smb_get_token+0x56(ffffff016185f5a8, ffffff0006436780)
  ffffff0006436870 smb_authenticate_core+0x18e(ffffff016460ecc8, ffffff016a54df28)
  ffffff00064368c0 smb_authenticate+0x44(ffffff016460ecc8, ffffff016a54df28)
  ffffff0006436940 smb_com_session_setup_andx+0x2e(ffffff016460ecc8)
  ffffff0006436a30 smb_dispatch_request+0x64a(ffffff016460ecc8)
  ffffff0006436a70 smb_session_worker+0xd8(ffffff016460ecc8)
  ffffff0006436b20 taskq_d_thread+0x123(ffffff016a2f5f58)
  ffffff0006436b30 thread_start+8()

Rich Lowe helped me figure out that having the
TP_HOLDLWP flag set when in door_upcall
makes us loop in issig_forreal() calling stop()
in an endless loop.

New, version of this out for review. See: https://www.illumos.org/rb/r/24

It turns out that there's no need to create an LWP in taskq_bucket_extend() which is how most threads are created for dynamic taskqs. In fact, the only place we really need an LWP (as opposed to a simple kthread) is when we're creating a taskq with the TASKQ_DUTY_CYCLE flag. That's not supported in combination with DYNAMIC so we don't need to make taskq_bucket_extend deal with LWPs at all. All I really need to do here was make sure the proc passed to taskq_create_common (and stored in tq_proc) is used when we're creating threads.

This version of the fix works with the "smb server in a zone" code (coming to illumos soon).

git commit c883468ae561a0b211f3633eb2a24aa4306661f2

commit  c883468ae561a0b211f3633eb2a24aa4306661f2
Author: Gordon Ross <gwr@nexenta.com>
Date:   2015-04-21T18:59:04.000Z

    backout: 1501 taskq_create_proc ... TQ_DYNAMIC puts tasks in p0 (needs work)