Simplify SMAP relocations with krtld
Today, SMAP is enabled and disabled by making a function call to smap_enable and smap_disable at the appropriate times. It would be cleaner, and faster, to simply arrange for the appropriate instructions to appear in place of those calls. We can achieve this in krtld by adjusting the program text in a similar fashion to that used for DTrace SDT probe sites.
Summary of Changes
- The old definitions of "smap_enable" and "smap_disable" are removed. These empty definitions are replaced by stubs created in /usr/src/uts/i86pc/Makefile.rules.
- krtld now ignores the symbols, "smap_enable" and "smap_disable" for AMD64 machines. Currently SMAP is not supported for i386 machines running illumos.
- A new functionality is introduced: "hotinlines". This is a list of symbols we would like to have replaced with inlined assembly instructions. Note that these instructions can be no larger than the original call. In the case of "smap_enable" and "smap_disable" this is 5 bytes. This list is now part of the module struct in /usr/src/uts/common/sys/kobj.h.
- In the relocation phase of module loading, we now construct a list of these "hotinlines", by noting the addresses of calls to "smap_enable" or "smap_disable". Upon creation of a new hotinline, the location of the original call is replaced with NOPs.
- These "hotinlines" are then performed by do_hotinlines() just before the module is started in /usr/src/uts/common/os/modctl.c.
- The "unix" module is started in a different code path than other modules such as "dtrace". Thus "hotinlines" are performed just before SMAP itself is enabled in /usr/src/uts/i86pc/os/startup.c.
Because this feature is only supported on AMD64 machines with SMAP it is
important to test several types of machines. Most importantly:
- AMD64 machines w/SMAP
- AMD64 machines w/o SMAP
Because this code affects machine independent code, we should also test other architectures and platforms, in particular Xen and i386 machines.
Boot to kmdb Run: startup_smap::dis Success for machines that support SMAP: The "smap_enable" at the end of the function should be replaced with: "clac, nop, nop" Success for machines that do not support SMAP, the "smap_enable" at the end of the function should be replaced with: "nop, nop, nop, nop, nop" Failure: Crash Boot to login Run: mdb -k; dtrace_copy::dis Success for machines that support SMAP: The "smap_disable" at the beginning of the function should be replaced with: "stac, nop, nop" The "smap_enable" at the end of the function should be replaced with: "clac, nop, nop" Success for machines that do not support SMAP: The "smap_disable" at the beginning of the function should be replaced with: "nop, nop, nop, nop, nop" the "smap_enable" at the end of the function should be replaced with: "nop, nop, nop, nop, nop" Failure: Crash
- Boot VMWare Fusion (AMD64 SMAP supported): PASS
- Boot AMD64 machine w/o SMAP supported: PASS
- Purposefully dereference a userspace pointer on a SMAP-enabled machine. PASS
- Purposefully derefernce a userspace pointer on a machine without SMAP. PASS
- Boot with disable_smap =1 (on SMAP enabled machine) PASS
Robert Mustacchi shipped me a driver to test out SMAP. Here is the output from a driver that attempts to dereference a userspace address. This proves SMAP is working correctly on machines that support it.
[root@00-0c-29-58-5d-8d /zones/home/vmloop-smap-new]# ./a.out panic[cpu0]/thread=ffffff01ab3770c0: BAD TRAP: type=e (#pf Page fault) rp=ffffff000468bb70 addr=8050ff0 occurred in module "vmloop" due to an illegal access to a user address a.out: #pf Page fault Bad kernel fault at addr=0x8050ff0 pid=108003, pc=0xfffffffff7f40e71, sp=0xffffff000468bc60, eflags=0x10282 cr0: 8005003b<pg,wp,ne,et,ts,mp,pe> cr4: 3406b8<smap,smep,osxsav,xmme,fxsr,pge,pae,pse,de> cr2: 8050ff0cr3: 1f120000cr8: 0 rdi: c500000000 rsi: 42 rdx: 8050ff0 rcx: 100003 r8: ffffff02100d37e0 r9: ffffff000468be48 rax: ffffffffc0122f40 rbx: ffffff021c650100 rbp: ffffff000468bc60 r10: c5 r11: 0 r12: 3 r13: 42 r14: 8050ff0 r15: 3 fsb: 0 gsb: fffffffffbc478e0 ds: 4b es: 4b fs: 0 gs: 1c3 trp: e err: 1 rip: fffffffff7f40e71 cs: 30 rfl: 10282 rsp: ffffff000468bc60 ss: 38 CPU ADDRESS TIMESTAMP TYPE VC HANDLER PC 0 fffffffffbc28e80 c74f6cc3a1 trap e #pf vmloop_ioctl+4 0 fffffffffbc28d08 c74f6caad9 syse 36 ioctl feee6c35 0 fffffffffbc28b90 c74f6ae351 syse 5 open32 feee9133 0 fffffffffbc28a18 c74f6a58ae trap e #pf feeb8f31 0 fffffffffbc288a0 c74f69b77d trap e #pf feed17ae 0 fffffffffbc28728 c74f69842f syse 32 sysi86 feee9185 0 fffffffffbc285b0 c74f69373f sysc ac lwp_cond_broadca fefe4527 0 fffffffffbc28438 c74f6916e1 sysc ac lwp_cond_broadca fefe4527 0 fffffffffbc282c0 c74f68886d trap e #pf fee5a750 0 fffffffffbc28148 c74f6858d7 trap e #pf fee5a750 ffffff000468ba50 unix:real_mode_stop_cpu_stage2_end+bae4 () ffffff000468bb60 unix:trap+17db () ffffff000468bb70 unix:_cmntrap+1ca () ffffff000468bc60 vmloop:vmloop_ioctl+4 () ffffff000468bca0 genunix:cdev_ioctl+39 () ffffff000468bcf0 specfs:spec_ioctl+60 () ffffff000468bd80 genunix:fop_ioctl+55 () ffffff000468bea0 genunix:ioctl+9b () ffffff000468bf00 unix:brand_sys_sysenter+2be ()
A machine without SMAP should run the same code without a panic. The following output shows this is the case:
[root@92-49-1a-ba-df-af /var/tmp/sam/vmloop-smap-panic]# sh install.sh [root@92-49-1a-ba-df-af /var/tmp/sam/vmloop-smap-panic]# ./a.out ioctl returned 0