zfs loading and unloading metaslabs at high frequency
This is Joyent bug OS-7151 (https://smartos.org/bugview/OS-7151) There is a lot of detail in that bug report. To summarize here, when we are experiencing this performance problem, there are some set of metaslabs that are being loaded and unloaded constantly. Not all systems will experience this problem and we were unable to recreate this scenario on a lab machine, despite trying for several months. However, when a system does experience this problem, what we observe using 'zdb -mm' is that the metaslab fragmentation profile is tightly clustered. That is, all of the metaslabs in the top-level vdevs have very similar fragmentation as shown by zfs. Machine which don't experience this problem have varying degrees of fragmentation across their metaslabs.
Updated by Jerry Jelinek over 2 years ago
- Status changed from New to Closed
When this was first proposed, after discussion on the zfs developer's list, Matt Ahrens suggested the following fix that had in the pipeline would address this issue, albeit in a different way. Concurrent small allocation defeats large allocation That was recently mearged, along with several other fixes in commit: af1d63aba5 11918 metaslab improvements For this reason, I'm closing this as "will not fix".