ZFS support for vectorized algorithms on x86 (initial support)
This tracks preliminary work so that we can eventually add SIMD HW support to ZFS raidz.
We start with a port from openzfs for:
fc0c72b16 Support for vectorized algorithms on x86 (initial support)
and a partial port of:
ab9f4b0b824 SIMD implementation of vdev_raidz generate and reconstruct routines
This initial phase does not add any code which uses the FPU in the kernel. All that this
first phase is doing is porting the basic code refactoring so that we could (in the future)
add the SIMD modules which use FPU features such as SSE or AVX2. That work
will require some additional prerequisite work to allow FPU usage in the kernel.
At this time, there is no intention to port the openzfs code which uses other x86
SIMD features, such as any of the AVX512 support, since there is known overhead
associated with using AVX512.
Updated by Jerry Jelinek almost 2 years ago
To test this I ran a zfs test run which includes running the new "raidz_test" program
to test the algorithms and compare the results to the original raidz algorithm.
I also ran this on a system where I had a copy of a set of files in a raidz2.
I first failed one disk (using zinject) and diff-verified the files against a
golden copy in a separate file system. I then repeated this test (after
clearing the ARC) with 2 failed disks. The files were correctly reconstructed
from parity in both cases.
For performance, the "raidz_test" program includes a benchmark mode. I have
attached a complete set of output. Since this code runs through all of the
algorithms and all of the permutations, there are a lot of results.
Here is a summary of some key comparisons at 4K, 128K and 1M recordsize.
In general the scalar algorithm is much better on generation, except for the
small recordsize (4k). The scalar algorithm is always the same or much better
In these results, the columns are the following and the 'total_bw' column
(2nd to last) provides the relevant comparison.
impl, math, dcols, iosize, disk_bw, total_bw, iter
raidz1 generation original, gen_p, 8, 4096, 108.050526, 972.454730, 1048576 scalar, gen_p, 8, 4096, 106.141851, 955.276656, 1048576 original, gen_p, 8, 131072, 248.547621, 2236.928588, 32768 scalar, gen_p, 8, 131072, 355.345384, 3198.108457, 32768 original, gen_p, 8, 1048576, 478.934748, 4310.412734, 4096 scalar, gen_p, 8, 1048576, 1099.841124, 9898.570117, 4096
raidz2 generation original, gen_pq, 8, 4096, 106.083159, 1060.831590, 1048576 scalar, gen_pq, 8, 4096, 53.017868, 530.178685, 1048576 original, gen_pq, 8, 131072, 191.733953, 1917.339534, 32768 scalar, gen_pq, 8, 131072, 291.722009, 2917.220091, 32768 original, gen_pq, 8, 1048576, 312.419193, 3124.191926, 4096 scalar, gen_pq, 8, 1048576, 455.338371, 4553.383707, 4096
raidz3 generation original, gen_pqr, 8, 4096, 102.868121, 1131.549328, 1048576 scalar, gen_pqr, 8, 4096, 35.402424, 389.426664, 1048576 original, gen_pqr, 8, 131072, 103.808905, 1141.897953, 32768 scalar, gen_pqr, 8, 131072, 160.601484, 1766.616326, 32768 original, gen_pqr, 8, 1048576, 133.020122, 1463.221346, 4096 scalar, gen_pqr, 8, 1048576, 210.622504, 2316.847542, 4096
Updated by Electric Monk almost 2 years ago
- Status changed from New to Closed
- % Done changed from 0 to 100
commit e86372a01d2d16a5dd4a64e144ed978ba17fe7dd Author: Gvozden Neskovic <email@example.com> Date: 2020-06-01T21:05:58.000Z 12668 ZFS support for vectorized algorithms on x86 (initial support) Portions contributed by: Jerry Jelinek <firstname.lastname@example.org> Reviewed by: Brian Behlendorf <email@example.com> Reviewed by: Jason King <firstname.lastname@example.org> Approved by: Dan McDonald <email@example.com>