[LU-11607] Reduce repeated function calls in Lustre test suites Created: 02/Nov/18  Updated: 17/Feb/21  Resolved: 17/Feb/21

Status: Resolved
Project: Lustre
Component/s: None
Affects Version/s: Lustre 2.12.0
Fix Version/s: Lustre 2.14.0

Type: Improvement Priority: Minor
Reporter: James Nunez (Inactive) Assignee: James Nunez (Inactive)
Resolution: Fixed Votes: 0
Labels: easy, tests

Issue Links:
Related
is related to LU-11889 sanity test 156 fails on ZFS: roc_hit... Open
is related to LU-5375 Failure on test suite sanity test_151... Resolved
is related to LU-11965 interop: sanity test 311 fails with '... Resolved
Rank (Obsolete): 9223372036854775807

 Description   

There are many repeated calls to the same function with the same input in each of the Lustre test suites. One example of this is the call to
“$(lustre_version_code $SINGLEMDS)”. All test suites combined call “$(lustre_version_code $SINGLEMDS)” 269 times; sanity.sh 135 times, conf-sanity 27 times, sanityn 16 times, etc. There are 44 calls to “$(lustre_version_code ost1)” and 28 calls to “$(lustre_version_code mgs)”. There are 113 calls to "facet_fstype $SINGLEMDS" with 34 in conf-sanity alone.

Let's get the environment variables that are stable during each test suite in a routine, call it at the beginning of each test suite and export it for the test suites to use. The routine must be called after we source the configuration file

. $LUSTRE/tests/test-framework.sh
init_test_env $@
. ${CONFIG:=$LUSTRE/tests/cfg/$NAME.sh}

since SINGLEMDS and other variables are defined in the configuration file.

The patch for LU-11564, https://review.whamcloud.com/#/c/33463, started this move by calling lustre_version_code() at the beginning of the test suite and using the value for each test that needs it.

Please list any other test suite routine calls that can be reduced in this way.



 Comments   
Comment by Gerrit Updater [ 28/Dec/18 ]

James Nunez (jnunez@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/33938
Subject: LU-11607 tests: create get_env() to get Lustre env
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: c51e83fc8be49929e785d3fe24fd3335552efa42

Comment by Gerrit Updater [ 24/Jan/19 ]

Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/33938/
Subject: LU-11607 tests: create routine to get Lustre env
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 4eb4479b0ea050d99033a9bac9994d2f1509200c

Comment by James A Simmons [ 25/Jan/19 ]

I'm seeing:

----============= acceptance-small: sanity-quota ============---- Thu Jan 24 22:04:03 UTC 2019 Running: bash /usr/lib64/lustre/tests/sanity-quota.sh Skipping tests: 55 == sanity-quota test complete, duration -o sec ======================================================= 22:04:04 (1548367444) /usr/lib64/lustre/tests/sanity-quota.sh: line 37: get_lustre_env: command not found Trace dump: = /usr/lib64/lustre/tests/sanity-quota.sh:37:main() sanity-quota: FAIL: test-framework exiting on error

Comment by James Nunez (Inactive) [ 25/Jan/19 ]

James - a couple of patches were landed out of order. The error that you reported should be cleaned up now. Please rebase any patches that are in flight.

Comment by Andreas Dilger [ 27/Jan/19 ]

Janes N, it makes sense to push out a patch that removes usage of SINGLEMDS, but leaves the definition of SINGLEMDS in test-framework.sh for a few weeks in case it is used in another patch in flight. They can both be submitted at the same time, but mark the second one -1 until we are ready to land it.

Comment by Gerrit Updater [ 25/Feb/19 ]

James Nunez (jnunez@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/34319
Subject: LU-11607 tests: create routine to get Lustre env
Project: fs/lustre-release
Branch: b2_12
Current Patch Set: 1
Commit: a0563c775921bbc8399ba9e9998632e132c90fae

Comment by Gerrit Updater [ 01/Apr/19 ]

Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/34319/
Subject: LU-11607 tests: create routine to get Lustre env
Project: fs/lustre-release
Branch: b2_12
Current Patch Set:
Commit: 08badb42a5040bd56d3dc27fac40bc3831c15f93

Comment by James Nunez (Inactive) [ 23/Apr/19 ]

With the landing of https://review.whamcloud.com/33611/, get_lustre_env() was moved into init_logging() in test-framework.sh and, thus, is available to all test scripts from Lustre 2.12.53 on.

We've already cleaned up sanity.sh by replacing calls to lustre_version_code() with MGS_VERSION, MDS1_VERSION or CLIENT_VERSION and replacing facet_fstype() with mds1_FSTYPE or ost1_FSTYPE. What's left to do for this ticket is to replace these calls in all the other Lustre test suites.

Comment by Gerrit Updater [ 07/Aug/19 ]

James Nunez (jnunez@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/35719
Subject: LU-11607 tests: replace version/fstype calls in sanity/n
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 668ca3817b19082f59bf8776d535e3012f5cd264

Comment by Gerrit Updater [ 07/Aug/19 ]

James Nunez (jnunez@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/35721
Subject: LU-11607 tests: replace version/fstype calls in conf-sanity
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: e2d2025d06d326ebf1f9ff7c1b96442bbdc0e1d3

Comment by Gerrit Updater [ 07/Aug/19 ]

James Nunez (jnunez@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/35723
Subject: LU-11607 tests: replace version/fstype calls in recov-small
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 7018501158f2d3aaa7659b22b382602681e4f6e5

Comment by Gerrit Updater [ 07/Aug/19 ]

James Nunez (jnunez@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/35724
Subject: LU-11607 tests: replace version/fstype calls in replay-sing
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 7c37142616d935c6279cbc7bd27d9fca4f8b5df2

Comment by Gerrit Updater [ 10/Aug/19 ]

James Nunez (jnunez@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/35760
Subject: LU-11607 tests: replace lustre_version calls in sanity-flr/pfl
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 8c7c81492559f352a87315eba5bfb9c45efcfc10

Comment by Gerrit Updater [ 11/Aug/19 ]

James Nunez (jnunez@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/35766
Subject: LU-11607 tests: replace version/fstype in sanity-dom/quota
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: c9f52e570abc25ab112d1729dccafccf69d425fc

Comment by Gerrit Updater [ 15/Aug/19 ]

Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/35721/
Subject: LU-11607 tests: replace version/fstype calls in conf-sanity
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: b4c955fe72d8598c4eaf98b809ae42be94f8c40b

Comment by Gerrit Updater [ 26/Aug/19 ]

James Nunez (jnunez@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/35928
Subject: LU-11607 tests: replace lustre_version in mds-survey/pcc/sec
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 8c0de7a025308c88492950d30ed028ba6ce09069

Comment by Gerrit Updater [ 26/Aug/19 ]

James Nunez (jnunez@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/35929
Subject: LU-11607 tests: replace lustre_version/fstype in sanity-scrub
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: ddaeeaa4d5ea75591811c94a373eeccaedbb059e

Comment by Gerrit Updater [ 27/Aug/19 ]

James Nunez (jnunez@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/35934
Subject: LU-11607 tests: replace lustre_version/fstype in posix/perf
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 398772e0a406aa5ca19650548f706fa6dbd84b08

Comment by Gerrit Updater [ 03/Sep/19 ]

Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/35724/
Subject: LU-11607 tests: replace version/fstype calls in replay-single
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 771630a718aa33d03879fefa17242c61df530f74

Comment by Gerrit Updater [ 06/Sep/19 ]

Andreas Dilger (adilger@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/36093
Subject: LU-11607 tests: add a default definition for SINGLEMDS
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 1371a18669bf0b075f3c0beda79d38e26989ebc0

Comment by Gerrit Updater [ 07/Sep/19 ]

Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/35760/
Subject: LU-11607 tests: replace lustre_version calls in sanity-flr/pfl
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 56b8c3d8d1acf2f6c491ed64126ce72199ace276

Comment by Gerrit Updater [ 07/Sep/19 ]

Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/35766/
Subject: LU-11607 tests: replace version/fstype in sanity-dom/quota
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 9d5d2714c074a51c9a348b917f3cf32c5005b3b8

Comment by Gerrit Updater [ 16/Sep/19 ]

Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/35723/
Subject: LU-11607 tests: replace version/fstype calls in recovery-small
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 82430bdc5a042a0cf74965e0bb975eef8316c0d7

Comment by Gerrit Updater [ 16/Sep/19 ]

Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/36093/
Subject: LU-11607 tests: add a default definition for SINGLEMDS
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: cf2e838320f50a1fc320e74d4c051772b8742742

Comment by Gerrit Updater [ 04/Oct/19 ]

Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/35934/
Subject: LU-11607 tests: replace lustre_version/fstype in posix/perf
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 73cbcdfe749edd3a441084b90cb2b53f73fa86e4

Comment by Gerrit Updater [ 04/Oct/19 ]

James Nunez (jnunez@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/36375
Subject: LU-11607 tests: replace lustre_version/fstype - full
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 8a8d3817d90073b2c366d73e408c7465031b5a11

Comment by Gerrit Updater [ 04/Oct/19 ]

James Nunez (jnunez@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/36380
Subject: LU-11607 tests: replace lustre_version/fstype - large-lun
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 608a4317a215c9b93d01bf8f81f739fccd51f719

Comment by Gerrit Updater [ 27/Oct/19 ]

Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/36380/
Subject: LU-11607 tests: replace lustre_version/fstype - large-lun
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: d2fff2c2e49526f1bcbdb1f63ed20aff558b3836

Comment by Gerrit Updater [ 12/Nov/19 ]

Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/35928/
Subject: LU-11607 tests: replace lustre_version in mds-survey/pcc/sec
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 3643e27f7a97dac195a102db4cf7f895e019159c

Comment by Gerrit Updater [ 05/Dec/19 ]

Minh Diep (mdiep@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/36942
Subject: LU-11607 tests: replace version/fstype calls in conf-sanity
Project: fs/lustre-release
Branch: b2_12
Current Patch Set: 1
Commit: 240bc8adf88a46880c37e09987edb01d043e22c5

Comment by Gerrit Updater [ 09/Dec/19 ]

James Nunez (jnunez@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/36965
Subject: LU-11607 tests: remove duplicate code lnet-selftest
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: b2e63c0e881d4e4370d717d3cbbc3294cc809575

Comment by Gerrit Updater [ 14/Dec/19 ]

Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/36375/
Subject: LU-11607 tests: replace lustre_version/fstype - full
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 782b3533edb419374b7357b67ba4f9960d7e96cf

Comment by Gerrit Updater [ 20/Dec/19 ]

Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/36965/
Subject: LU-11607 tests: remove duplicate code lnet-selftest
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: cbdceec741f5fb782b6a7f86d30daed9f5ed562a

Comment by Gerrit Updater [ 08/Feb/20 ]

Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/35719/
Subject: LU-11607 tests: replace version/fstype calls in sanity/n
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 6ee31faf3970d47a79f538712382073b3833a06c

Comment by Gerrit Updater [ 25/Feb/20 ]

Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/36942/
Subject: LU-11607 tests: replace version/fstype calls in conf-sanity
Project: fs/lustre-release
Branch: b2_12
Current Patch Set:
Commit: 70a42ae7880e137267dcb0dc00b4a38fbae0fe37

Comment by Gerrit Updater [ 20/Oct/20 ]

James Nunez (jnunez@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/40313
Subject: LU-11607 tests: Reproducer sanity
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: ca7547335efff5b72d235904cd128ed05fde9bfc

Comment by Gerrit Updater [ 08/Feb/21 ]

Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/35929/
Subject: LU-11607 tests: replace lustre_version/fstype in sanity-scrub
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: e4799a53832bd1278b766ed2e0a261281b10f28c

Comment by Peter Jones [ 17/Feb/21 ]

Looks like everything has landed for 2.14

Generated at Sat Feb 10 02:45:21 UTC 2024 using Jira 9.4.14#940014-sha1:734e6822bbf0d45eff9af51f82432957f73aa32c.