Project

General

Profile

Bug #4601

memory leak in ILB daemon on startup

Added by Serghei Samsi over 6 years ago. Updated over 6 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
networking
Start date:
2014-02-15
Due date:
% Done:

100%

Estimated time:
Difficulty:
Bite-size
Tags:
Gerrit CR:

Description

Starting ILBD (L3/L4 Integrated Load Balancer daemon) with health-check rules leads to memory leak:

umem_alloc_40 leak: 2 buffers, 40 bytes each, 80 bytes total
ADDR BUFADDR TIMESTAMP THREAD
CACHE LASTLOG CONTENTS
8682a58 8680f58 21bc434a6bc569 1
862d010 0 0
libumem.so.1`umem_cache_alloc_debug+0x157
libumem.so.1`umem_cache_alloc+0x157
libumem.so.1`umem_alloc+0xd0
libumem.so.1`malloc+0x2d
libumem.so.1`calloc+0x63
ilbd_hc_associate_rule+0x6b
ilbd_create_rule+0x11c
consume_common_struct+0x154
handle_req+0x6c
main_loop+0xdf
main+0x197
_start+0x7d

How to reproduce:
1. Run /usr/lib/inet/ilbd with LD_PRELOAD=/usr/lib/libumem.so

2. Run ilbadm import-config <path_to_ilb_config_file>

3. Run ilbadm import-config <path_to_ilb_config_file>

4. Run gcore on ilbd PID, MDB findleaks output:
CACHE LEAKED BUFCTL CALLER
0862d010 1 08682a58 ilbd_hc_associate_rule+0x6b
0862d010 3 08682ad0 ilbd_hc_associate_rule+0x6b
0862d010 2 08682968 ilbd_hc_associate_rule+0x6b

Main condition: presence of health-check rule

History

#1

Updated by Serghei Samsi over 6 years ago

Possible fix for ilbd_hc_associate_rule leak
(file /illumos-gate/usr/src/cmd/cmd-inet/usr.lib/ilbd/ilbd_hc.c):

/* Associate a rule with its hc object. */
ilb_status_t
ilbd_hc_associate_rule(const ilbd_rule_t *rule, int ev_port)
{
    ilbd_hc_t    *hc;
    ilbd_hc_rule_t    *hc_rule;
+    ilbd_hc_rule_t    *hc_rule_next;
    ilb_status_t    ret;
    ilbd_sg_t    *sg;
    ilbd_srv_t    *ilbd_srv;

    /* The rule is assumed to be initialized appropriately. */
    if ((hc = ilbd_get_hc(rule->irl_hcname)) == NULL) {
        logdebug("ilbd_hc_associate_rule: healthcheck %s does not " 
            "exist", rule->irl_hcname);
        return (ILB_STATUS_ENOHCINFO);
    }
    if ((hc->ihc_test_type == ILBD_HC_TCP &&
        rule->irl_proto != IPPROTO_TCP) ||
        (hc->ihc_test_type == ILBD_HC_UDP &&
        rule->irl_proto != IPPROTO_UDP)) {
        return (ILB_STATUS_RULE_HC_MISMATCH);
    }
+      hc_rule_next = NULL;
+      for (hc_rule = list_head(&hc->ihc_rules); hc_rule != NULL; hc_rule = hc_rule_next) {
+           hc_rule_next = list_next(&hc->ihc_rules, hc_rule);
+           if (hc_rule->hcr_rule == rule) {
+                ilbd_hc_srv_rem_all(hc_rule);
+                list_remove(&hc->ihc_rules, hc_rule);
+                hc->ihc_rule_cnt--;
+                list_destroy(&hc_rule->hcr_servers);
+                free(hc_rule);
+           }
+      }
    if ((hc_rule = calloc(1, sizeof (ilbd_hc_rule_t))) == NULL) {
        logdebug("ilbd_hc_associate_rule: out of memory");
        return (ILB_STATUS_ENOMEM);
    }

    hc_rule->hcr_rule = rule;
    list_create(&hc_rule->hcr_servers, sizeof (ilbd_hc_srv_t),
        offsetof(ilbd_hc_srv_t, shc_srv_link));

    /* Add all the servers. */
    sg = rule->irl_sg;
    for (ilbd_srv = list_head(&sg->isg_srvlist); ilbd_srv != NULL;
        ilbd_srv = list_next(&sg->isg_srvlist, ilbd_srv)) {
        if ((ret = ilbd_hc_srv_add(hc, hc_rule, &ilbd_srv->isv_srv,
            ev_port)) != ILB_STATUS_OK) {
            /* Remove all previously added servers */
            ilbd_hc_srv_rem_all(hc_rule);
+                      list_destroy(&hc_rule->hcr_servers);
            free(hc_rule);
            return (ret);
        }
    }
    list_insert_tail(&hc->ihc_rules, hc_rule);
    hc->ihc_rule_cnt++;

    return (ILB_STATUS_OK);
}
#2

Updated by David Höppner over 6 years ago

Please post a webrev to the developer list.

#3

Updated by Serghei Samsi over 6 years ago

Yes, I will post ASAP.

#5

Updated by Robert Mustacchi over 6 years ago

  • Status changed from New to Resolved
  • % Done changed from 10 to 100
  • Tags deleted (needs-triage)
#6

Updated by Electric Monk over 6 years ago

git commit 56b8f71e3a910fbd2820f6841b40bfd85f9673c2

Author: Serghei Samsi <sscdvp@gmail.com>

4601 memory leak in ILB daemon on startup
4602 memory leak in ILB daemon on import-config
4668 Memory leak in ilbd' new_req: getpeerucred() allocation isn't released at all
Reviewed by: Dan McDonald <danmcd@omniti.com>
Reviewed by: Marcel Telka <marcel.telka@nexenta.com>
Approved by: Robert Mustacchi <rm@joyent.com>

Also available in: Atom PDF