Project

General

Profile

Actions

Bug #5376

closed

arc_kmem_reap_now() should not result in clearing arc_no_grow

Added by Matthew Ahrens over 7 years ago. Updated over 7 years ago.

Status:
Closed
Priority:
Normal
Category:
zfs - Zettabyte File System
Start date:
2014-11-30
Due date:
% Done:

100%

Estimated time:
Difficulty:
Medium
Tags:
needs-triage
Gerrit CR:

Description

I observed a machine writing data at 200-500MB/s.
mpstat showed that periodically one CPU would become 100% busy for a few
seconds, with heavy crosscall activity. Sometimes the other CPUs would become
idle during this time. I discovered that this was due primarily to
arc_kmem_reap_now()'s calls to kmem_cache_reap_now(), and resulting TLB
shootdowns to remove mappings for freed slabs. There are many opportunities to
improve the kmem/vmem/hat code. However, upon reproducing the issue, I found that ZFS's
ARC management can also be improved to reduce the frequency and duration of
these xcall storms. The primary observation is that we should not clear
arc_no_grow after calling arc_kmem_reap_now(). One way to prevent this would
be to have an idea of when we are getting low on memory before we actually have
to call arc_kmem_reap_now(), to not allow arc_no_grow to be cleared when we are
low on memory, and to ensure that arc_shrink() will keep us in the "low on
memory" region in the absence of other interactions. Implementing this fix
resulted in much less frequent calls to arc_kmem_reap_now() under a synthetic
workload.

Actions

Also available in: Atom PDF