Project

General

Profile

Actions

Feature #13036

closed

grep tests could be easier to extend

Added by Robert Mustacchi about 1 year ago. Updated about 1 year ago.

Status:
Closed
Priority:
Normal
Category:
tests
Start date:
Due date:
% Done:

100%

Estimated time:
Difficulty:
Medium
Tags:
Gerrit CR:

Description

The grep test suite is a little challenging to work with right now. In particular, it's designed so that there are multiple phases of tests. However, the set of output file names is used across all phases. Adding an additional test an earlier phase would mean renaming everything. Instead, we should just make these phases a part of the script. In addition there are a bunch of other small things that could be improved about the test suite:

  • Adding new phases shouldn't require duplicating the same bulk of logic every time
  • It could run all tests to completion, rather than stopping at the first failure.
  • It could actually tests non-zero cases
  • The temporary ends up trying to be in the same directory as the tests, when it could just be in /tmp.
  • When something is different or there's a missing output file, it could tell you the name of the file it expected
Actions #1

Updated by Electric Monk about 1 year ago

  • Gerrit CR set to 841
Actions #2

Updated by Robert Mustacchi about 1 year ago

I tested this by running the grep test suite and verifying that everything passed. I also forced a number of failures along the way to verify that we were still able to detect failures. Most of the improvements there were in response to things I hit adding the other features. In addition, I verified that in cases that used to require pfexec to successfully operate due to the location of the temporary file, that was no longer necessary.

rm@beowulf:~$ pfexec /opt/util-tests/bin/utiltest 
Test: /opt/util-tests/tests/allowed-ips (run as root)             [00:00] [PASS]
Test: /opt/util-tests/tests/chown_test (run as root)              [00:00] [PASS]
Test: /opt/util-tests/tests/date_test (run as root)               [00:00] [PASS]
Test: /opt/util-tests/tests/find/findtest (run as root)           [00:00] [PASS]
Test: /opt/util-tests/tests/grep_test (run as root)               [00:00] [PASS]
Test: /opt/util-tests/tests/libjedec_test (run as root)           [00:00] [PASS]
Test: /opt/util-tests/tests/libsff/libsff (run as root)           [00:00] [PASS]
Test: /opt/util-tests/tests/make_test (run as root)               [00:00] [PASS]
Test: /opt/util-tests/tests/mdb/mdbtest (run as root)             [00:00] [PASS]
Test: /opt/util-tests/tests/mergeq/mqt (run as root)              [00:00] [PASS]
Test: /opt/util-tests/tests/mergeq/wqt (run as root)              [00:00] [PASS]
Test: /opt/util-tests/tests/printf_test (run as root)             [00:00] [PASS]
Test: /opt/util-tests/tests/set-linkprop (run as root)            [00:00] [PASS]
Test: /opt/util-tests/tests/sleep/sleeptest (run as root)         [00:21] [PASS]
Test: /opt/util-tests/tests/smbios (run as root)                  [00:00] [PASS]
Test: /opt/util-tests/tests/xargs_test (run as root)              [00:00] [PASS]
Test: /opt/util-tests/tests/awk/runtests.sh (run as nobody)       [02:46] [PASS]
Test: /opt/util-tests/tests/ctf/precheck (run as root)            [00:00] [PASS]
Test: /opt/util-tests/tests/ctf/ctftest (run as root)             [00:05] [PASS]
Test: /opt/util-tests/tests/demangle/afl-fast (run as root)       [00:01] [PASS]
Test: /opt/util-tests/tests/demangle/gcc-libstdc++ (run as root)  [00:00] [PASS]
Test: /opt/util-tests/tests/demangle/llvm-stdcxxabi (run as root) [00:00] [PASS]
Test: /opt/util-tests/tests/libcustr/custr_remove (run as root)   [00:00] [PASS]
Test: /opt/util-tests/tests/libcustr/custr_trunc (run as root)    [00:00] [PASS]
Test: /opt/util-tests/tests/libnvpair_json/json_00_blank (run as root) [00:00] [PASS]
Test: /opt/util-tests/tests/libnvpair_json/json_01_boolean (run as root) [00:00] [PASS]
Test: /opt/util-tests/tests/libnvpair_json/json_02_numbers (run as root) [00:00] [PASS]
Test: /opt/util-tests/tests/libnvpair_json/json_03_empty_arrays (run as root) [00:00] [PASS]
Test: /opt/util-tests/tests/libnvpair_json/json_04_number_arrays (run as root) [00:00] [PASS]
Test: /opt/util-tests/tests/libnvpair_json/json_05_strings (run as root) [00:00] [PASS]
Test: /opt/util-tests/tests/libnvpair_json/json_06_nested (run as root) [00:00] [PASS]
Test: /opt/util-tests/tests/libnvpair_json/json_07_nested_arrays (run as root) [00:00] [PASS]

Results Summary
PASS      32

Running Time:   00:03:20
Percent passed: 100.0%
Log directory:  /var/tmp/test_results/20200819T133603
Actions #3

Updated by Electric Monk about 1 year ago

  • Status changed from New to Closed
  • % Done changed from 90 to 100

git commit f415aa8f1340638644c6cf63e9d11d3767f4cbba

commit  f415aa8f1340638644c6cf63e9d11d3767f4cbba
Author: Robert Mustacchi <rm@fingolfin.org>
Date:   2020-08-25T23:00:10.000Z

    13036 grep tests could be easier to extend
    Reviewed by: Andy Fiddaman <andy@omniosce.org>
    Reviewed by: Toomas Soome <tsoome@me.com>
    Approved by: Gordon Ross <gordon.w.ross@gmail.com>

Actions

Also available in: Atom PDF