Project

General

Profile

Actions

Bug #6062

closed

Workaround broken KVM handling of directed EOIs

Added by Dan McDonald almost 7 years ago. Updated almost 7 years ago.

Status:
Closed
Priority:
Normal
Category:
-
Start date:
2015-07-16
Due date:
% Done:

100%

Estimated time:
Difficulty:
Medium
Tags:
needs-triage
Gerrit CR:

Description

A fallout of #3644 is that we need to work around broken Linux or OpenStack KVM implementations of directed End-of-interrupts (EOIs).

This is a fix that must accompany the vioif driver, for its users have discovered directed EOIs in Linux or OpenStack KVMs do not work.

A fix was already proposed, based on an earlier one by Delphix, but improved to not require /etc/system tuning:


--- illumos-gate/usr/src/uts/i86pc/io/pcplusmp/apic_regops.c    Mon Jul  6 10:52:54 2015
+++ 3644/usr/src/uts/i86pc/io/pcplusmp/apic_regops.c    Wed Jul 15 17:52:23 2015
@@ -24,6 +24,7 @@
  */
 /*
  * Copyright 2014 Josef 'Jeff' Sipek <jeffpc@josefsipek.net>
+ * Copyright (c) 2014 by Delphix. All rights reserved.
  */

 #include <sys/cpuvar.h>
@@ -66,6 +67,9 @@
 int    x2apic_enable = 1;
 apic_mode_t apic_mode = LOCAL_APIC;    /* Default mode is Local APIC */

+/* Enable directed EOIs */
+boolean_t apic_allow_kvm_directed_eoi = B_FALSE;
+
 /* Uses MMIO (Memory Mapped IO) */
 static apic_reg_ops_t local_apic_regs_ops = {
     local_apic_read,
@@ -294,6 +298,21 @@
 {
     uint32_t ver;

+    /*
+     * There are some known issues with some versions of Linux KVM and QEMU
+     * where by directed EOIs do not properly function and instead get
+     * coalesced at the hypervisor, causing the host not to see interrupts.
+     * Thus, when the platform is KVM, we disable it by default. If
+     * apic_allow_kvm_directed_eoi is set to B_TRUE, then that will escape
+     * this situation and allow them to be used.
+     *
+     * Note, at this time illumos KVM does not identify as KVM. If it does,
+     * we'll need to do some work to determine if it should be caught by
+     * this or if it should show up as its own value of platform_type.
+     */
+    if (get_hwenv() == HW_KVM && apic_allow_kvm_directed_eoi == B_FALSE)
+        return (0);
+
     ver = apic_reg_ops->apic_read(APIC_VERS_REG);
     if (ver & APIC_DIRECTED_EOI_BIT)
         return (1);

Actions #1

Updated by Electric Monk almost 7 years ago

  • Status changed from New to Closed
  • % Done changed from 0 to 100

git commit 3f745f41d6d087602fbb2f748e1baabc3768f5fb

commit  3f745f41d6d087602fbb2f748e1baabc3768f5fb
Author: Dan McDonald <danmcd@omniti.com>
Date:   2015-07-20T16:50:43.000Z

    6062 Workaround broken KVM handling of directed EOIs
    Reviewed by: Robert Mustacchi <rm@joyent.com>
    Reviewed by: Josef 'Jeff' Sipek <josef.sipek@nexenta.com>
    Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
    Approved by: Hans Rosenfeld <rosenfeld@grumpf.hope-2000.org>

Actions

Also available in: Atom PDF