Project

General

Profile

Bug #11305

add smartos runfile to zfs test suite

Added by Kody Kantor over 1 year ago. Updated over 1 year ago.

Status:
Closed
Priority:
Normal
Assignee:
Kody Kantor
Category:
tests
Start date:
Due date:
% Done:

100%

Estimated time:
Difficulty:
Bite-size
Tags:
Gerrit CR:

Description

Downstream smartos has been maintaining its own zfs test suite runfile while testing kinks have been getting ironed out. Now that the tests run more reliably on smartos, it's time to add the runfile to illumos-gate so it can be updated in-sync with the rest of the downstream platform runfiles.

#1

Updated by Yuri Pankov over 1 year ago

May be illumos-gate should have just one runfile that represents the tests that are in illumos-gate, and downstreams could maintain their own based on what tests they have/run?

#2

Updated by Kody Kantor over 1 year ago

Yuri Pankov wrote:

May be illumos-gate should have just one runfile that represents the tests that are in illumos-gate, and downstreams could maintain their own based on what tests they have/run?

That's a good question. There's a risk that if we have the per-distribution runfiles maintained downstream that they will quickly get out of date as new tests are added. That could lead to lurking distribution-specific problems. For that reason it's my opinion that we should continue to maintain these files in illumos-gate.

A future improvement I've been thinking of is to rework this testing system to use a blacklist of tests instead of the current whitelist system. For example:
- There is a master list of all tests (per-component (e.g. zfs-tests, os-tests, etc., just like today), maintained in illumos-gate)
- Each distribution maintains a blacklist of tests to skip (maintained in downstream repositories)
- By default the master runfile is run, sans the distribution-specific blacklist

This would fix a few test maintenance problems. However since this doesn't exist I think we should continue with the current pattern of per-distribution runfiles in illumos-gate.

#3

Updated by Electric Monk over 1 year ago

  • Status changed from New to Closed
  • % Done changed from 0 to 100

git commit 4c6a557ef3ee09ce3af0b3773cf03d745a0bdcb6

commit  4c6a557ef3ee09ce3af0b3773cf03d745a0bdcb6
Author: Kody Kantor <kody@kkantor.com>
Date:   2019-07-18T15:04:09.000Z

    11305 add smartos runfile to zfs test suite
    Reviewed by: Toomas Soome <tsoome@me.com>
    Reviewed by: Andy Stormont <astormont@racktopsystems.com>
    Approved by: Dan McDonald <danmcd@joyent.com>

#4

Updated by Kody Kantor over 1 year ago

For the record, this is what the zfs-test output looks like on smartos with this runfile:

root@coke:/opt/zfs-tests/bin# ./zfstest
...
Results Summary
PASS     873

Running Time:    01:39:32
Percent passed:    100.0%
Log directory:    /var/tmp/test_results/20190719T161940
you have mail in /var/mail/root

Also available in: Atom PDF