Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-13773

sanity-dom test failure not triggering test suite marked as FAIL


    • Type: Bug
    • Status: In Progress
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: Lustre 2.13.0, Lustre 2.14.0, Lustre 2.12.5
    • Fix Version/s: None
    • Labels:
    • Severity:
    • Rank (Obsolete):


      We see, in LU-13759, that a sanityn test that is run as part of sanity-dom can fail and the failure is not propagated to a level that Maloo recognizes it as a failure. There are several problems that are also seen with sanity-dom logs; logs for the failed tests are not collected and client log for successful tests are not collected. Note that this seems like an issue with the test framework and not with Maloo.

      For failure at https://testing.whamcloud.com/test_sets/08c6fa9d-e2a3-457b-a1ed-b4318dbf166a, we can see that the sanity-dom/sanityn test 20 failure is recognized, but this does not lead to the sanity-dom test suite being marked at failed. In the results file at https://testing.whamcloud.com/test_sessions/b31ca578-dd4f-4725-9a0c-5e19f3031c69/show_results, we see that the fail is registered, but

          -   name: test_sanityn
          -   name: test_1
              status: PASS
              duration: 4
              return_code: 0
          -   name: test_19
              status: SKIP
              duration: 2
              return_code: 0
              error: not cache-capable obdfilter
          -   name: test_20
              status: FAIL
              duration: 5
              return_code: 1
              error: 1 page left in cache after lock cancel
          -   name: test_23
              status: PASS
              duration: 65
              return_code: 0
          -   name: test_51d
              status: PASS
              duration: 435
              return_code: 0
          duration: 1129
          status: PASS

      We had a related issue where, when a sanity-dom/sanityn test failed, the failure would trigger the whole test suite to fail, but Maloo thinks the last test in sanityn failed which is false; see LU-10589 and one example failure at https://testing.whamcloud.com/test_sets/870c7a78-467f-11e9-9646-52540065bddc. In this case it looks like all logs are not collected.

      Patrick Farrell produced a patch which clears the problem with a sub test of a sub suite failure not triggering the suite to be marked as fail at https://review.whamcloud.com/#/c/34186/ . I think the no log collected/displayed issue still exists with this proposed solution.




            • Assignee:
              jamesanunez James Nunez
              jamesanunez James Nunez
            • Votes:
              0 Vote for this issue
              2 Start watching this issue


              • Created: