[LU-614] Speed up test scripts

• Status: Resolved
• Project: Lustre
• Type: Improvement
• Priority: Minor
• Reporter: Andreas Dilger
• Assignee: Andreas Dilger
• Resolution: Fixed

Description

Fixes to speed up the Lustre internal test scripts in order improve testing efficiency.

As a starter, conf-sanity test_54[ab] are taking 30 minutes together to do very low-value testing of the llverdev and llverfs scripts (e.g. https://maloo.whamcloud.com/test_sets/ff31e77a-ca79-11e0-8d02-52540025f9af). It is important to verify those scripts work, but they should only ever do partial testing in conf-sanity, and full runs are usually only done in a separate "large-LUN" test.

Issue Links

• is related to: LU-1791 sanity.sh test_224b takes too long to...

Comments

[Andreas Dilger]

Also fixing sanity test_220 to use unlinkmany instead of "rm -r" to hopefully save 15 minutes of wasted time running "rm" in hash order. https://maloo.whamcloud.com/test_logs/bd3870f0-ca79-11e0-8d02-52540025f9af

[Andreas Dilger]

Change http://review.whamcloud.com/1266

[Andreas Dilger]

Original test (https://maloo.whamcloud.com/test_sets/ff31e77a-ca79-11e0-8d02-52540025f9af):

05:29:27:== conf-sanity test 54a: llverdev ========================= 05:29:25 (1313756965)
05:35:09:PASS 54a (343s)
05:35:10:== conf-sanity test 54b: llverfs ========================== 05:35:08 (1313757308)
05:59:41:PASS 54b (1472s)

New test (https://maloo.whamcloud.com/test_logs/82ced8f8-cbc0-11e0-8d02-52540025f9af):
21:02:59:== conf-sanity test 54a: test llverdev and partial verify of device = 21:02:57 (1313899377)
21:03:39:PASS 54a (41s)
21:03:40:== conf-sanity test 54b: test llverfs and partial verify of filesystem 21:03:38 (1313899418)
21:04:27:PASS 54b (48s)

Net savings (343 + 1472 - 41 - 48) = 1726s = 28m

[Build Master (Inactive)]

Integrated in lustre-master » x86_64,client,el5,inkernel #272
LU-614 test: speed up needlessly long tests

Oleg Drokin : f426fb920e3b97298a603a142d56be89c12f2006
Files :

• lustre/tests/conf-sanity.sh
• lustre/tests/test-framework.sh
• lustre/tests/sanity.sh

[Build Master (Inactive)]

Integrated in lustre-master » i686,client,el6,inkernel #272
LU-614 test: speed up needlessly long tests

Oleg Drokin : f426fb920e3b97298a603a142d56be89c12f2006
Files :

• lustre/tests/test-framework.sh
• lustre/tests/conf-sanity.sh
• lustre/tests/sanity.sh

[Build Master (Inactive)]

Integrated in lustre-master » x86_64,server,el5,inkernel #272
LU-614 test: speed up needlessly long tests

Oleg Drokin : f426fb920e3b97298a603a142d56be89c12f2006
Files :

• lustre/tests/sanity.sh
• lustre/tests/test-framework.sh
• lustre/tests/conf-sanity.sh

[Build Master (Inactive)]

Integrated in lustre-master » x86_64,server,el6,inkernel #272
LU-614 test: speed up needlessly long tests

Oleg Drokin : f426fb920e3b97298a603a142d56be89c12f2006
Files :

• lustre/tests/test-framework.sh
• lustre/tests/conf-sanity.sh
• lustre/tests/sanity.sh

[Build Master (Inactive)]

Integrated in lustre-master » i686,client,el5,ofa #272
LU-614 test: speed up needlessly long tests

Oleg Drokin : f426fb920e3b97298a603a142d56be89c12f2006
Files :

• lustre/tests/test-framework.sh
• lustre/tests/sanity.sh
• lustre/tests/conf-sanity.sh

[Build Master (Inactive)]

Integrated in lustre-master » x86_64,server,el5,ofa #272
LU-614 test: speed up needlessly long tests

Oleg Drokin : f426fb920e3b97298a603a142d56be89c12f2006
Files :

• lustre/tests/test-framework.sh
• lustre/tests/conf-sanity.sh
• lustre/tests/sanity.sh

[Build Master (Inactive)]

Integrated in lustre-master » x86_64,client,el6,inkernel #272
LU-614 test: speed up needlessly long tests

Oleg Drokin : f426fb920e3b97298a603a142d56be89c12f2006
Files :

• lustre/tests/test-framework.sh
• lustre/tests/sanity.sh
• lustre/tests/conf-sanity.sh

[Build Master (Inactive)]

Integrated in lustre-master » x86_64,client,el5,ofa #272
LU-614 test: speed up needlessly long tests

Oleg Drokin : f426fb920e3b97298a603a142d56be89c12f2006
Files :

• lustre/tests/sanity.sh
• lustre/tests/conf-sanity.sh
• lustre/tests/test-framework.sh

[Build Master (Inactive)]

Integrated in lustre-master » i686,server,el6,inkernel #272
LU-614 test: speed up needlessly long tests

Oleg Drokin : f426fb920e3b97298a603a142d56be89c12f2006
Files :

• lustre/tests/conf-sanity.sh
• lustre/tests/test-framework.sh
• lustre/tests/sanity.sh

[Build Master (Inactive)]

Integrated in lustre-master » i686,server,el5,inkernel #272
LU-614 test: speed up needlessly long tests

Oleg Drokin : f426fb920e3b97298a603a142d56be89c12f2006
Files :

• lustre/tests/test-framework.sh
• lustre/tests/sanity.sh
• lustre/tests/conf-sanity.sh

[Build Master (Inactive)]

Integrated in lustre-master » x86_64,client,sles11,inkernel #272
LU-614 test: speed up needlessly long tests

Oleg Drokin : f426fb920e3b97298a603a142d56be89c12f2006
Files :

• lustre/tests/conf-sanity.sh
• lustre/tests/sanity.sh
• lustre/tests/test-framework.sh

[Build Master (Inactive)]

Integrated in lustre-master » x86_64,client,ubuntu1004,inkernel #272
LU-614 test: speed up needlessly long tests

Oleg Drokin : f426fb920e3b97298a603a142d56be89c12f2006
Files :

• lustre/tests/sanity.sh
• lustre/tests/test-framework.sh
• lustre/tests/conf-sanity.sh

[Build Master (Inactive)]

Integrated in lustre-master » i686,client,el5,inkernel #272
LU-614 test: speed up needlessly long tests

Oleg Drokin : f426fb920e3b97298a603a142d56be89c12f2006
Files :

• lustre/tests/conf-sanity.sh
• lustre/tests/sanity.sh
• lustre/tests/test-framework.sh

[Build Master (Inactive)]

Integrated in lustre-master » i686,server,el5,ofa #272
LU-614 test: speed up needlessly long tests

Oleg Drokin : f426fb920e3b97298a603a142d56be89c12f2006
Files :

• lustre/tests/conf-sanity.sh
• lustre/tests/test-framework.sh
• lustre/tests/sanity.sh

[Andreas Dilger]

Old sanity test_220: 2567s
https://maloo.whamcloud.com/test_logs/bd3870f0-ca79-11e0-8d02-52540025f9af
New sanity test_220: 2231s
https://maloo.whamcloud.com/test_sessions/7b5eb708-d13f-11e0-8d02-52540025f9af

Savings: (2567s - 2231s) = 345s per test run

[Oleg Drokin]

the patch for test 220 actually introduced a regression and I am not even sure how was it able to complete properly before.

The problem is we overwrite next_id before the final unlink, so nothing could be unlinked

[Build Master (Inactive)]

Integrated in lustre-master » x86_64,client,el5,inkernel #276
LU-614 tests: speed up needlessly long tests

Oleg Drokin : becbc7263598e10212f1dcb8eb2cc9446c3c01bb
Files :

• lustre/tests/sanity.sh

[Build Master (Inactive)]

Integrated in lustre-master » i686,client,el6,inkernel #276
LU-614 tests: speed up needlessly long tests

Oleg Drokin : becbc7263598e10212f1dcb8eb2cc9446c3c01bb
Files :

• lustre/tests/sanity.sh

[Build Master (Inactive)]

Integrated in lustre-master » x86_64,client,el6,inkernel #276
LU-614 tests: speed up needlessly long tests

Oleg Drokin : becbc7263598e10212f1dcb8eb2cc9446c3c01bb
Files :

• lustre/tests/sanity.sh

[Build Master (Inactive)]

Integrated in lustre-master » i686,client,el5,ofa #276
LU-614 tests: speed up needlessly long tests

Oleg Drokin : becbc7263598e10212f1dcb8eb2cc9446c3c01bb
Files :

• lustre/tests/sanity.sh

[Build Master (Inactive)]

Integrated in lustre-master » x86_64,server,el5,ofa #276
LU-614 tests: speed up needlessly long tests

Oleg Drokin : becbc7263598e10212f1dcb8eb2cc9446c3c01bb
Files :

• lustre/tests/sanity.sh

[Build Master (Inactive)]

Integrated in lustre-master » x86_64,server,el5,inkernel #276
LU-614 tests: speed up needlessly long tests

Oleg Drokin : becbc7263598e10212f1dcb8eb2cc9446c3c01bb
Files :

• lustre/tests/sanity.sh

[Build Master (Inactive)]

Integrated in lustre-master » i686,server,el5,inkernel #276
LU-614 tests: speed up needlessly long tests

Oleg Drokin : becbc7263598e10212f1dcb8eb2cc9446c3c01bb
Files :

• lustre/tests/sanity.sh

[Build Master (Inactive)]

Integrated in lustre-master » i686,server,el6,inkernel #276
LU-614 tests: speed up needlessly long tests

Oleg Drokin : becbc7263598e10212f1dcb8eb2cc9446c3c01bb
Files :

• lustre/tests/sanity.sh

[Build Master (Inactive)]

Integrated in lustre-master » x86_64,server,el6,inkernel #276
LU-614 tests: speed up needlessly long tests

Oleg Drokin : becbc7263598e10212f1dcb8eb2cc9446c3c01bb
Files :

• lustre/tests/sanity.sh

[Build Master (Inactive)]

Integrated in lustre-master » x86_64,client,sles11,inkernel #276
LU-614 tests: speed up needlessly long tests

Oleg Drokin : becbc7263598e10212f1dcb8eb2cc9446c3c01bb
Files :

• lustre/tests/sanity.sh

[Build Master (Inactive)]

Integrated in lustre-master » x86_64,client,ubuntu1004,inkernel #276
LU-614 tests: speed up needlessly long tests

Oleg Drokin : becbc7263598e10212f1dcb8eb2cc9446c3c01bb
Files :

• lustre/tests/sanity.sh

[Build Master (Inactive)]

Integrated in lustre-master » i686,client,el5,inkernel #276
LU-614 tests: speed up needlessly long tests

Oleg Drokin : becbc7263598e10212f1dcb8eb2cc9446c3c01bb
Files :

• lustre/tests/sanity.sh

[Build Master (Inactive)]

Integrated in lustre-master » x86_64,client,el5,ofa #276
LU-614 tests: speed up needlessly long tests

Oleg Drokin : becbc7263598e10212f1dcb8eb2cc9446c3c01bb
Files :

• lustre/tests/sanity.sh

[Build Master (Inactive)]

Integrated in lustre-master » i686,server,el5,ofa #276
LU-614 tests: speed up needlessly long tests

Oleg Drokin : becbc7263598e10212f1dcb8eb2cc9446c3c01bb
Files :

• lustre/tests/sanity.sh

[Andreas Dilger]

Close old issue.