[LU-1779] "skip" inside a test does not report SKIP in Maloo

• Status: Resolved
• Project: Lustre
• Fix Version/s: Lustre 2.3.0, Lustre 2.4.0, Lustre 2.1.4, Lustre 1.8.9
• Type: Bug
• Priority: Minor
• Reporter: Andreas Dilger
• Assignee: Jian Yu
• Resolution: Fixed
• Story Points: 1
• Severity: 3

Description

It seems that tests in ALWAYS_EXCEPT are reported in maloo as "skip ALWAYS excluded test NN". However, if some test is skipped due to an environmental condition (e.g. sanity.sh test_27m, which is testing out-of-space conditions and is skipped during review runs when the filesystem is too large, as in https://maloo.whamcloud.com/sub_tests/78d60ce0-9dc0-11e1-a1d8-52540035b04c) this will print a "SKIP" message in the test, but the test itself is reported as "PASS" Maloo.

It would be more useful if the test were reported in Maloo as "SKIP: sanity test_27m 892015396 > 1400000 skipping out-of-space test on OST0". What does auster/Maloo key on to determine whether a test is SKIP or PASS, and what needs to be done to make the "skip" directive inside a test cause Maloo to report the test as skipped?

The reason this is of interest is because I want the tests skipped because of LU-482 to be clearly flagged as not having been run, rather than pretending the test has passed. The patch in http://review.whamcloud.com/2731 will mark the test with "skip", but until this issue is fixed then the whole replay-dual will appear to be a PASS, even though nothing was run.

Issue Links

• is related to: LU-482 Test failure on test suite replay-dua...

Comments

[Chris Gearing (Inactive)]

The problem here is that the test framework is reporting PASS in the yaml. We need to make the test framework behave properly.

[Jian Yu]

Patch for master branch: http://review.whamcloud.com/3810

[Jian Yu]

Patch for b2_1 branch: http://review.whamcloud.com/3899

The one for master and b2_3 branches has been landed.

[Andreas Dilger]

Patch landed on b2_1, b2_3, master.

[Jian Yu]

Patch for b1_8 branch: http://review.whamcloud.com/4955