Bug #11873

SMB file access audit logging (remainder)

Added by Gordon Ross 8 months ago. Updated 7 months ago.

In Progress
cifs - CIFS server and client
Start date:
Due date:
% Done:


Estimated time:
Gerrit CR:


In order to avoid conflicts over ID assignment in the auditing code, We had to split 11037 into two parts:
#11037 SMB File access audit logging (reserve IDs)
11873 SMB File access audit logging (remainder)
See #11037 for additional information.



Updated by Rich Lowe 8 months ago

  • Project changed from site to illumos gate

Updated by Gordon Ross 8 months ago

  • Description updated (diff)
  • Category set to cifs - CIFS server and client
  • Status changed from New to In Progress

Updated by Gordon Ross 7 months ago

Matt's design write-up:

Auditing in Illumos: How it Works

(fun fact: FreeBSD and Mac OS X have an implementation of this called "OpenBSM"!)
At startup, main calls audit_init(). If c2audit hasn't been specifically excluded in /etc/system, audit_init() initializes the auditing subsystem. This includes adding a per-thread auditing structure to all existing threads, and initializing "audit_active", so that all future threads include this auditing structure. At this point, no auditing configuration is in use.
When auditd() starts up, it uses the auditon() SETCOND ioctl to turn on auditing in the kernel. This tells the kernel to begin auditing based on its (not-yet-set) configuration. The kernel then generates a 'SYSTEMBOOT' audit record. (This appears to be a relic of when auditing couldn't be reconfigured without a reboot.) auditd then reads in the audit configuration from SMF, and the event-to-class mappings from /etc/security/audit_event.txt, and passes the relevant configuration to the kernel using more auditon() ioctls.
auditd() then initializes and configures the auditd_plugins (loaded from SMF). From then on, auditd()'s job is to take audit records passed to it by the kernel (via doorcall) and enqueue it onto each active plugin's queue. Each plugin has its own thread that dequeues these records for processing by the plugin.
audit records can originate either from user space via the audit() syscall, or from the kernel, either due to syscall auditing or a 'non-attributable' event (e.g. SYSTEMBOOT, PROMENTER/PROMEXIT). Audit events are generated either directly due to the actions of a user, known as the 'process model', or on behalf of a user, known as the 'session model'. All auditing begins as a session model: a program requests (using libbsm) that it be granted an auditing session handle. It then initializes that handle with the necessary information to audit that user's actions, including the user-specific audit mask and its auditing IDs. The program can then generate audit events on that handle. As an example, SMBD uses this to generate login/logout audit events for SMB users (see smbd_user_auth_logon()). A session can then be converted to the 'process model' via libbsm. libbsm uses another syscall to copy the auditing information from the session handle into the user process's credential. When that process performs an auditable syscall, the kernel uses the information stored in the process credential to generate an audit record. Auditable syscalls first get redirected to c2audit for initialization, including initializing the per-thread audit structure. Syscalls then store auditable information in that structure, and once the syscall is complete, c2audit determines whether or not to commit the record. PAM_unix uses this model to propogate auditing configuration through all of a user's processes forked after login. The syscall hook then pushes the written record onto a queue, and the writer thread eventually dequeues it and passes it down via doorcall to auditd for processing.
How do we determine what audit records to generate?
Part of the information stored in the session handle and the user's credential is an 'audit mask'. This mask is two
bitfields of audit classes - one for success and one for failure - that are configured to be audited. This is a combination of the 'default' audit mask, configured through auditconfig, and a per-user mask stored as the 'audit_flags' 'user_attr' (i.e. usermod -K audit_flags:<flags>). auditd passes the 'default' audit mask to the kernel, and libbsm() combines the two in the user's audit mask. When an auditable event occurs, it is first mapped to its corresponding class mask (based on /etc/security/audit_events.txt), and then if that class mask intersects with the appropriate audit mask, a record is generated from that event.
How do SACLs fit into this?
SACL events are, in essence, a process model event originating from the kernel, bypassing all of the syscall machinery. All of the underlying mechanisms for generating audit records are sufficiently generic that the syscall hooks are just a wrapper for it all. We can make sufficiently generic alternatives to some of the more syscall-specific functions (like asking the question 'should we generate SACL audit records for this user?') with minor modifications to existing code.
The current design for SACL-based auditing looks like this:
1. SMBD creates an auditing session handle. This retrieves the audit mask, uid, and session ID. It then passes this information up to the kernel, along with the rest of the authentication information.
2. SMBSRV writes this information into the auditing structure of the user's credential.
3. If auditing is configured, When a user tries to perform certain SMB2 operations (namely, open and rename), SMBSRV initializes the per-thread auditing structure with a value indicating to the underlying filesystem that it should create an auditing mask.
4. When ZFS performs an access check, it first determines whether SACL auditing is configured. If it is, it calls a version of zfs_zaccess_aces_check that also processes 'audit' and 'alarm' type acceses into success and failure masks, then writes those masks into the per-thread audit structure. It then disables further auditing, in case multiple access checks are performed, in order to prevent overwriting the audit mask.
5. When the filesystem call returns, SMBSRV compares the request's DesiredAccess to the appropriate audit mask, depending on whether or not the access was succesful. If the masks intersect, SMBSRV provides sufficient information to c2audit (via 'audit_sacl'), including by providing the credential used, so that an record of the access may be generated.
You may wonder why SMB is in charge of determining whether or not to generate on audit record. Access on Windows is, unfortunately, not determined solely by the security descriptor of an object. Accesses can additionally be granted by the security descriptor on the parent directory (namely, Delete granted by DeleteChild on the parent, or ReadAttributes granted by ListDirectory on the parent), or by specific grants of privilege, or by a combination of privilege and specific options in the Open request. The end result is that a single protocol-level access check can comprise of several filesystem-level access checks, and those access checks can be for a subset of the access actually granted. As such, while there may be ways to propogate sufficent knowledge to the underlying filesystem to allow it to generate records, SMBSRV is ultimately in the best position to know which accesses ought to be audited and for which requests and filesystem calls.


Updated by Gordon Ross 7 months ago

Audit Configuration Notes (from Matt)

audit config notes:
auditconfig -setflags sa #set user default preselection flags to audit SACLs
auditconfig -setflags -sa #set user default preselection flags to audit only failure SACLs
auditconfig -setflags +sa #set user default preselection flags to audit only success SACLs
auditconfig -setkmask sa #set non-attributes selection flags of machine to audit SACLs. This is necessary to audit non-IDMU domain users (i.e. those with ephemeral IDs.)
auditconfig -setnaflags sa #set non-attributable audit flags non-persistently.
use "audit" to control, and "auditconfig" to configure
audit -n #tell auditd to start a new file
audit -s #tell auditd to reload and validate audit configuration. Also starts auditd.
audit -t #tell auditd to shutdown.
audit -v #validate audit configuration.
auditstat #see audit statistics
I wrote up these instructions while (trying) to prepare a demo for auditing; they're probably a good place to start for investigating how to expose these commands.
auditconfig -setflags sa #set user default preselection flags to audit SACLs
auditconfig -setnaflags sa #domain users that don't get mapped to non-ephemeral UIDs are audited based on naflags.
  1. Note: One can set the active non-attributable audit flags temporarily via 'auditconfig -setkmask sa'. This will not record the update in smf.
    audit -v #should be "configuration ok" 
    audit -s #load configuration and enable auditing.

    Then, audit specific files by setting 'audit type ACLS:
    chmod A+everyone@:rwxpdDaARWcCos:fdSF:audit /export/junk/audit #audit all successful (S) and failed (F) accesses from everyone, and inherit to files and directories.
    chmod A+<type>:<id>:rwpdD:S:audit <file> #audit successful Read, Write, Append, and Delete accesses from <id>
    audit -n #close the current audit file, and open a new one
    auditreduce <options> # select some subset of audit records. see manpage for record selection options.
    auditreduce | praudit <opts> # print audit records selected by auditreduce. use '-r' for raw, '-s' for 'short form', and -l for 'one line per record'. -x can be combined with any of these to print in XML.
    audit -n; auditreduce -D <suffix-or-path> # Delete completed audit records, storing the selected records in a new file. Note that 'notterminated' files are ignored by this, unless specified explicilty on the command line.
    auditconfig -setkaudit <ipv4 or ipv6> <ip address>
    auditconfig -setplugin <name> <(in)active> <attributes>
    auditconfig -setpolicy <policy flag> #auditconfig -lspolicy to see descriptions of each.
    some useful commands. see auditconfig(1M) for more info.
    auditconfig -getplugin #see the installed plugins and their configurations
    auditconfig -getpolicy #kernel audit policy. auditconfig -lspolicy to see descriptions of each.
    - notable: 'group' includes group info in audit records, 'perzone' allows configuring auditd per-zone, 'seq' adds a sequence number to records, 'zonename' includes zone name in records, 'trail' adds a trailer token to records (like header)
    auditconfig -getqctrl #get write buffer size, hiwater mark, lowater mark, and delay interval.
    auditconfig -getstat #get audit stats (see auditstat(1M))

    NOTE: SMBD also offers the ability to audit SMB logons; simply select the 'lo' (login/logoff) audit class, as well as 'sa' (auditconfig -setflags sa,lo). One can then corrolate who logged in with who accessed what files. Note that, unless the default event mappings are changed, several other logon events may be generated as well. Local accounts can be given user-specific audit preselection masks by setting the 'audit_flags' keyword in the user_attr database (usermod -K audit_flags=<always-audit-flags>:<never-audit-flags> ...)

Also available in: Atom PDF