Problem handling groups with Winbind
I have run into a problem handling large numbers of groups when they're being provided by Winbind (part of Samba) from an Active Directory environment. This issue shows up with the opencsw.org package for Samba 3.6.11. I've also compiled (using gcc, not Sun cc) 3.6.12 and seen the same behavior.
The operating system is OpenIndiana 151a7 (latest updates) based on the Illumos
kernel. I've got "set ngroups_max = 128" in /etc/system and verified that this
is working. I'm able to manipulate users with up to 128 groups. At least, I
can do so as long as they're defined locally in /etc/passwd and /etc/group. If taken from Winbind, they appear to be active but fail (for example) with "cd" or chdir().
My problem is that AD users obtained via Winbind are running into a groups
limitation which appears to be at 16. This is confusing because it works most
places. If I "su GGC\\\\abelletti" and then run "id -a", I'm able to see every
assigned group. I've added a large number (but less than 128) for testing.
However, if I then attempt to "cd" into a directory which I should be able to
access due to group permissions, I fail. This same operation works when those
groups are assigned locally. If running Samba, I get basically the same
[2013/02/15 18:13:29.427975, 4] smbd/vfs.c:780(vfs_ChDir)
vfs_ChDir to /shares/oet-networking
[2013/02/15 18:13:29.428073, 3] smbd/service.c:190(set_current_service)
chdir (/shares/oet-networking) failed, reason: Permission denied
I've run "pcred" on the smbd that belongs to GGC\\abelletti and verified that all group ids are present. That would have me assume that this is an Illumos/OpenIndiana problem except that it works fine with locally assigned users/groups as above.
I've tried modifying NGROUPS_MAX in limits.h and NGROUPS_MAX_DEFAULT in sys/param.h and recompiling, also without success.
I'm happy to provide any logs and debugging information which might help to debug this. Illumos/OpenIndiana is still the best ZFS platform, and I'd like to be able to run Samba on the server itself. Last I checked, kernel CIFS did not provide everything that I need and does not seem to be an active development target.
Updated by Björn Jacke over 7 years ago
I got some information on this topic from a oracle developer: The change that was done to the opensolaris kernel to add support for 1024 supplelentary groups for was incomplete. To make sure testing for groups is efficient they implemented a binary search when the user is in more that 16 groups. However not all places where groups are being set where changed accordingly and they missed to sort the list of groups, like n setgroups() for example. As a result of that the binary search fails in many cases. Build 167 and Solaris 10u11 (7044891 groups aren't always sorted in the credential) and Solaris 11 have fixed that I've been told.
Hope that information helps you to fix this issue in illumos. Good luck.