Project

General

Profile

Bug #4282

Race between ipmi_close() and kcs_loop() could lead to panic

Added by Marcel Telka about 7 years ago. Updated about 7 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
driver - device drivers
Start date:
2013-10-31
Due date:
% Done:

100%

Estimated time:
Difficulty:
Medium
Tags:
Gerrit CR:

Description

ipmi_complete_request() could access freed memory after the race:

> ::status
debugging crash dump vmcore.0 (64-bit) from raven
operating system: 5.11 illumos-ad17c01 (i86pc)
image uuid: b5da6336-99b4-ca7a-abbd-d52f33818f95
panic message: BAD TRAP: type=e (#pf Page fault) rp=ffffff001f675a10 addr=ffffff001f983000
dump content: kernel pages only
> ::stack
ipmi_complete_request+0x4e(ffffffffc0394ec0, ffffff04edfa7b80)
kcs_loop+0x86(ffffffffc0394ec0)
taskq_thread+0x2d0(ffffff04e0e577f0)
thread_start+8()
> ipmi_complete_request+0x4e::dis
ipmi_complete_request+0x21:     movl   $0x3,0x54(%rsi)
ipmi_complete_request+0x28:     call   +0x2c70133       <cv_signal>
ipmi_complete_request+0x2d:     movq   0x10(%rbx),%rax
ipmi_complete_request+0x31:     testq  %rax,%rax
ipmi_complete_request+0x34:     je     +0x28    <ipmi_complete_request+0x5e>
ipmi_complete_request+0x36:     movq   $0x0,(%rbx)
ipmi_complete_request+0x3d:     movq   0x8(%rax),%rdx
ipmi_complete_request+0x41:     movl   $0x41,%esi
ipmi_complete_request+0x46:     movq   %rdx,0x8(%rbx)
ipmi_complete_request+0x4a:     movq   0x8(%rax),%rdx
ipmi_complete_request+0x4e:     movq   %rbx,(%rdx)
ipmi_complete_request+0x51:     movq   0x10(%rax),%rdi
ipmi_complete_request+0x55:     movq   %rbx,0x8(%rax)
ipmi_complete_request+0x59:     call   +0x2d1b772       <pollwakeup>
ipmi_complete_request+0x5e:     movq   -0x18(%rbp),%rbx
ipmi_complete_request+0x62:     leave  
ipmi_complete_request+0x63:     ret    
ipmi_complete_request+0x64:     nopl   0x0(%rax)
ipmi_complete_request+0x68:     movq   %rsi,%rdi
ipmi_complete_request+0x6b:     call   -0xb0    <ipmi_free_request>
ipmi_complete_request+0x70:     movq   -0x18(%rbp),%rbx
> <rdx::whatis
ffffff001f983000 is freed from segkp_4096
>

If the ipmi_close() is called when the IPMI_LOCK is not held (for example when kcs_polled_request() is running), the ipmi_close() will not notice the kcs_loop() is just processing a request and will free everything. The ipmi_complete_request() will panic during the attempt to insert the req into the ipmi_completed_requests list.

The problem could be reproduced using this test (the loop count (1000) might need to be adjusted to match the machine and BMC speed):

#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <stdio.h>
#include <sys/ipmi.h>
#include <unistd.h>
#include <stropts.h>
#include <libipmi.h>

int
main(void)
{
    struct ipmi_req r;
    struct ipmi_system_interface_addr a;
    uint8_t d;
    int i;

    int f = open("/dev/ipmi0", O_RDWR);
    if (f == -1) {
        perror("ipmi open failed");
        return 1;
    }

    a.addr_type = IPMI_SYSTEM_INTERFACE_ADDR_TYPE;
    a.channel = IPMI_BMC_CHANNEL;
    a.lun = 0;

    r.addr = (unsigned char*)&a;
    r.addr_len = sizeof a;
    r.msgid = 0;
    r.msg.netfn = IPMI_NETFN_STORAGE;
    r.msg.cmd = IPMI_CMD_GET_FRU_INV_AREA;
    r.msg.data = &d;
    r.msg.data_len = sizeof d;

    for (i = 0; i < 1000; i++)
        if (ioctl(f, IPMICTL_SEND_COMMAND, &r) == -1) {
            perror("ipmi ioctl failed");
            return 1;
        }

    close(f);

    return 0;
}

Related issues

Related to illumos gate - Bug #3902: Race between ipmi_submit_driver_request() and kcs_loop()ResolvedMarcel Telka2013-07-21

Actions
#1

Updated by Robert Mustacchi about 7 years ago

  • Status changed from In Progress to Resolved
  • % Done changed from 0 to 100
  • Tags deleted (needs-triage)

Resolved in cc944374608a39fb5622959cc9a210fb116539cb.

Also available in: Atom PDF