[LU-9495] Improve recovery-double-scale.sh with sub-tests

• Type: Improvement
• Resolution: Unresolved
• Priority: Minor
• Affects Version/s: Lustre 2.10.4
• Labels: test
• Assignee: WC Triage
• Reporter: James Nunez (Inactive)

Description

Currently, recovery-double-scale contains a single sub-test called pairwise_fail(). pairwise_fail is composed of nine sub-tests. When one of pairwise_fail subtests is skipped or failed, the whole test suite is marked as SKIP or FAIL. Each of the nine subtests of pairwise_fail should be broken out into separate sub-tests. For example, for test 1

test_1() {
    # CMD_TEST_NUM=17.1
    failover_pair MDS OST "test 1: failover MDS, then OST =========="
    sleep $FAILOVER_PERIOD
}
run_test 1 " failover MDS, then OST”

With whatever additional variables and modifications needed to make sure each test runs correctly in a separate sub-test