Bug #11305
closed
add smartos runfile to zfs test suite
100%
Description
Downstream smartos has been maintaining its own zfs test suite runfile while testing kinks have been getting ironed out. Now that the tests run more reliably on smartos, it's time to add the runfile to illumos-gate so it can be updated in-sync with the rest of the downstream platform runfiles.
Updated by Yuri Pankov about 3 years ago
May be illumos-gate should have just one runfile that represents the tests that are in illumos-gate, and downstreams could maintain their own based on what tests they have/run?
Updated by Kody Kantor about 3 years ago
Yuri Pankov wrote:
May be illumos-gate should have just one runfile that represents the tests that are in illumos-gate, and downstreams could maintain their own based on what tests they have/run?
That's a good question. There's a risk that if we have the per-distribution runfiles maintained downstream that they will quickly get out of date as new tests are added. That could lead to lurking distribution-specific problems. For that reason it's my opinion that we should continue to maintain these files in illumos-gate.
A future improvement I've been thinking of is to rework this testing system to use a blacklist of tests instead of the current whitelist system. For example:
- There is a master list of all tests (per-component (e.g. zfs-tests, os-tests, etc., just like today), maintained in illumos-gate)
- Each distribution maintains a blacklist of tests to skip (maintained in downstream repositories)
- By default the master runfile is run, sans the distribution-specific blacklist
This would fix a few test maintenance problems. However since this doesn't exist I think we should continue with the current pattern of per-distribution runfiles in illumos-gate.
Updated by Electric Monk almost 3 years ago
- Status changed from New to Closed
- % Done changed from 0 to 100
git commit 4c6a557ef3ee09ce3af0b3773cf03d745a0bdcb6
commit 4c6a557ef3ee09ce3af0b3773cf03d745a0bdcb6
Author: Kody Kantor <kody@kkantor.com>
Date: 2019-07-18T15:04:09.000Z
11305 add smartos runfile to zfs test suite
Reviewed by: Toomas Soome <tsoome@me.com>
Reviewed by: Andy Stormont <astormont@racktopsystems.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Updated by Kody Kantor almost 3 years ago
For the record, this is what the zfs-test output looks like on smartos with this runfile:
root@coke:/opt/zfs-tests/bin# ./zfstest ... Results Summary PASS 873 Running Time: 01:39:32 Percent passed: 100.0% Log directory: /var/tmp/test_results/20190719T161940 you have mail in /var/mail/root