Bug #175
closed
zfs vdev cache consumes excessive memory
0%
Description
See OpenSolaris CR 6684116
but note that the work-around suggested there is less than ideal.
Better to set zfs_vdev_cache_size to zero, i.e. in /etc/system
set zfs:zfs_vdev_cache_size = 0
After some performance verification, it might make sense to
change the default to zero in uts/common/fs/zfs/vdev_cache.c
Updated by Garrett D'Amore over 12 years ago
- Category set to kernel
- Assignee set to Garrett D'Amore
Solaris 11 has been running with the tunable set to zero for a while now. Eventually we ought to just remove the code.
Updated by Garrett D'Amore over 12 years ago
We end up storing 10MB of readahead per vdev, and so if you have many
vdevs (say, if you sell storage systems), that really sucks on main
memory.
We find that the cache is usually very under utilized in the field. (30%-70%)
Updated by Garrett D'Amore over 12 years ago
- Status changed from New to Resolved
Resolved in:
garrett@thinkpad{15}> hg export tip- HG changeset patch
- User Garrett D'Amore <garrett@nexenta.com>
- Date 1303458581 25200
- Node ID f3ce1af7c12d88b7e9513aad2e7e1b20cae5bab0
- Parent 20c193a013b852a0435f528a95759f9684d9115a
175 zfs vdev cache consumes excessive memory
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Eric Schrock <eric.schrock@delphix.com>
Approved by: Richard Lowe <richlowe@richlowe.net>
Updated by Jim Klimov over 11 years ago
- Difficulty set to Medium
- Tags set to needs-triage
I object to removal of the code for everybody based on its inefficiency for somebody. For example, vdev prefetch was very efficient during a scrub on my systems and boosted the scrub speed (minus several hours off the total count for two scrubs).
In fact, I propose to expand the feature by making a separate rolling cache for non-metadata sectors which are currently discarded from prefetched data.
Tuning the two cache sizes is the end-user's adventure, and zero-size defaults can protect the multiple-drive systems with miniature RAM. illumos's task is to provide the mechanisms that can help it suit everyone - from single-spindle laptops to petabyte arrays ;)
Details are tracked here: https://www.illumos.org/issues/2017