[LU-13194] Interop: sanityn test 104 fails with '/mnt/lustre/f104.sanityn mtime (1580286855:1580286853) diff' Created: 03/Feb/20  Updated: 08/Feb/20  Resolved: 07/Feb/20

Status: Resolved
Project: Lustre
Component/s: None
Affects Version/s: Lustre 2.10.8, Lustre 2.12.3
Fix Version/s: Lustre 2.14.0, Lustre 2.12.4

Type: Bug Priority: Minor
Reporter: James Nunez (Inactive) Assignee: James Nunez (Inactive)
Resolution: Fixed Votes: 0
Labels: interop

Severity: 3
Rank (Obsolete): 9223372036854775807

 Description   

sanityn test_104 fails for clients with Lustre versions that are newer than the client version starting with Lustre 2.12.3 servers and 2.12.58.171 clients starting on 24 OCT 2019; see https://testing.whamcloud.com/test_sets/591b7932-f793-11e9-a197-52540065bddc and https://testing.whamcloud.com/test_sets/d58f7e54-f767-11e9-add9-52540065bddc.

Looking at a recent failure, https://testing.whamcloud.com/test_sets/8d9bdf58-4333-11ea-86c7-52540065bddc, for Lustre servers 2.11.0 and clients 2.12.3.109, we see the mtimes produced

CMD: trevis-51vm12 debugfs -c -R 'stat ROOT/f104.sanityn' /dev/mapper/mds1_flakey
trevis-51vm12: debugfs 1.45.2.wc1 (27-May-2019)
trevis-51vm12: /dev/mapper/mds1_flakey: catastrophic mode - not reading inode or group bitmaps
STAT a|m|ctime 1580286853 1580286853 1580286853
MDT a|m|ctime 1580286853 1580286853 1580286853
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 0.00156288 s, 655 kB/s
CMD: trevis-51vm12 debugfs -c -R 'stat ROOT/f104.sanityn' /dev/mapper/mds1_flakey
trevis-51vm12: debugfs 1.45.2.wc1 (27-May-2019)
trevis-51vm12: /dev/mapper/mds1_flakey: catastrophic mode - not reading inode or group bitmaps
STAT a|m|ctime 1580286853 1580286855 1580286855
MDT a|m|ctime 1580286853 1580286855 1580286855
 sanityn test_104: @@@@@@ FAIL: /mnt/lustre/f104.sanityn mtime (1580286855:1580286853) diff 

The patch for LU-12026, https://review.whamcloud.com/36286/, added test 104 to sanityn. It landed to master on October 2019 and landed to b2_12 on 17 JAN 2020.

sanityn test 104 does not have a Lustre version check, but needs one. Master, future 2.14.0, interop testing runs with 2.12.3 and 2.13.0. 2.13.0 already has this patch and 2.12.4 has this patch. So, when we move interop testing for master to use 2.12.4, we don’t have to worry about master interop testing hitting this issue. The only other release we need to worry about is the LTS branch b2_12. The 2.12 branch runs interop testing with 2.11.0 and 2.10.8. Thus, if we land a patch to b2_12 that checks the server version to make sure it is 2.12.3.110 or later, then all b2_12 interop testing should skip this test when appropriate.



 Comments   
Comment by Gerrit Updater [ 04/Feb/20 ]

James Nunez (jnunez@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/37420
Subject: LU-13194 tests: check server version sanityn 104
Project: fs/lustre-release
Branch: b2_12
Current Patch Set: 1
Commit: 67dbba5b73fc148596e81f801d3d92a9f5d38dbe

Comment by Andreas Dilger [ 04/Feb/20 ]

What is a bit confusing is that it looks like the test should be passing:

STAT a|m|ctime 1580286853 1580286855 1580286855
MDT a|m|ctime 1580286853 1580286855 1580286855
 sanityn test_104: @@@@@@ FAIL: /mnt/lustre/f104.sanityn mtime (1580286855:1580286853) diff 

The respective atimes, mtimes, and ctimes are the same between the client and MDT, even if they are not the same as each other.  Either the STAT or MDT lines are being printed incorrectly, or the test is checking the wrong values?

Comment by Gerrit Updater [ 06/Feb/20 ]

James Nunez (jnunez@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/37461
Subject: LU-13194 tests: check server version sanityn 104
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 6e64326566cc6963bc2896183fe84ec6b21d302a

Comment by Gerrit Updater [ 06/Feb/20 ]

Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/37461/
Subject: LU-13194 tests: check server version sanityn 104
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: a8b9a123fea3762b999e80f56fdbbdf2ea10e280

Comment by Peter Jones [ 07/Feb/20 ]

Landed for 2.14

Comment by Gerrit Updater [ 08/Feb/20 ]

Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/37420/
Subject: LU-13194 tests: check server version sanityn 104
Project: fs/lustre-release
Branch: b2_12
Current Patch Set:
Commit: dca549213447dc3958ca225aae386a48c9757618

Generated at Sat Feb 10 02:59:11 UTC 2024 using Jira 9.4.14#940014-sha1:734e6822bbf0d45eff9af51f82432957f73aa32c.