[LU-13194] Interop: sanityn test 104 fails with '/mnt/lustre/f104.sanityn mtime (1580286855:1580286853) diff' - Whamcloud Community JIRA

Details

Type: Bug
Resolution: Fixed
Priority: Minor
Fix Version/s: Lustre 2.14.0, Lustre 2.12.4
Affects Version/s: Lustre 2.10.8, Lustre 2.12.3
Labels:
- interop

Severity:
3
Rank (Obsolete):
9223372036854775807

Description

sanityn test_104 fails for clients with Lustre versions that are newer than the client version starting with Lustre 2.12.3 servers and 2.12.58.171 clients starting on 24 OCT 2019; see https://testing.whamcloud.com/test_sets/591b7932-f793-11e9-a197-52540065bddc and https://testing.whamcloud.com/test_sets/d58f7e54-f767-11e9-add9-52540065bddc.

Looking at a recent failure, https://testing.whamcloud.com/test_sets/8d9bdf58-4333-11ea-86c7-52540065bddc, for Lustre servers 2.11.0 and clients 2.12.3.109, we see the mtimes produced

CMD: trevis-51vm12 debugfs -c -R 'stat ROOT/f104.sanityn' /dev/mapper/mds1_flakey
trevis-51vm12: debugfs 1.45.2.wc1 (27-May-2019)
trevis-51vm12: /dev/mapper/mds1_flakey: catastrophic mode - not reading inode or group bitmaps
STAT a|m|ctime 1580286853 1580286853 1580286853
MDT a|m|ctime 1580286853 1580286853 1580286853
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 0.00156288 s, 655 kB/s
CMD: trevis-51vm12 debugfs -c -R 'stat ROOT/f104.sanityn' /dev/mapper/mds1_flakey
trevis-51vm12: debugfs 1.45.2.wc1 (27-May-2019)
trevis-51vm12: /dev/mapper/mds1_flakey: catastrophic mode - not reading inode or group bitmaps
STAT a|m|ctime 1580286853 1580286855 1580286855
MDT a|m|ctime 1580286853 1580286855 1580286855
 sanityn test_104: @@@@@@ FAIL: /mnt/lustre/f104.sanityn mtime (1580286855:1580286853) diff

The patch for ~~LU-12026~~, https://review.whamcloud.com/36286/, added test 104 to sanityn. It landed to master on October 2019 and landed to b2_12 on 17 JAN 2020.

sanityn test 104 does not have a Lustre version check, but needs one. Master, future 2.14.0, interop testing runs with 2.12.3 and 2.13.0. 2.13.0 already has this patch and 2.12.4 has this patch. So, when we move interop testing for master to use 2.12.4, we don’t have to worry about master interop testing hitting this issue. The only other release we need to worry about is the LTS branch b2_12. The 2.12 branch runs interop testing with 2.11.0 and 2.10.8. Thus, if we land a patch to b2_12 that checks the server version to make sure it is 2.12.3.110 or later, then all b2_12 interop testing should skip this test when appropriate.

Attachments

Issue Links

mentioned in: Page No Confluence page found with the given URL.

Activity

[LU-13194] Interop: sanityn test 104 fails with '/mnt/lustre/f104.sanityn mtime (1580286855:1580286853) diff'

Peter Jones made changes - 08/Feb/20 5:50 AM

Fix Version/s

New: Lustre 2.12.4 [ 14690 ]

Peter Jones made changes - 07/Feb/20 4:01 AM

Fix Version/s		New: Lustre 2.14.0 [ 14490 ]
Resolution		New: Fixed [ 1 ]
Status	Original: Open [ 1 ]	New: Resolved [ 5 ]

James Nunez (Inactive) made changes - 05/Feb/20 12:09 AM

Remote Link

New: This issue links to "Page (Whamcloud Community Wiki)" [ 24480 ]

James Nunez (Inactive) made changes - 04/Feb/20 4:09 AM

Description

Original: sanityn test_104 fails for clients with Lustre versions that are newer than the client version starting with Lustre 2.12.3 servers and 2.12.58.171 clients starting on 24 OCT 2019; see https://testing.whamcloud.com/test_sets/591b7932-f793-11e9-a197-52540065bddc and https://testing.whamcloud.com/test_sets/d58f7e54-f767-11e9-add9-52540065bddc.

Looking at a recent failure, https://testing.whamcloud.com/test_sets/8d9bdf58-4333-11ea-86c7-52540065bddc, for Lustre servers 2.11.0 and clients 2.12.3.109, we see the mtimes produced
{noformat}
CMD: trevis-51vm12 debugfs -c -R 'stat ROOT/f104.sanityn' /dev/mapper/mds1_flakey
trevis-51vm12: debugfs 1.45.2.wc1 (27-May-2019)
trevis-51vm12: /dev/mapper/mds1_flakey: catastrophic mode - not reading inode or group bitmaps
STAT a|m|ctime 1580286853 1580286853 1580286853
MDT a|m|ctime 1580286853 1580286853 1580286853
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 0.00156288 s, 655 kB/s
CMD: trevis-51vm12 debugfs -c -R 'stat ROOT/f104.sanityn' /dev/mapper/mds1_flakey
trevis-51vm12: debugfs 1.45.2.wc1 (27-May-2019)
trevis-51vm12: /dev/mapper/mds1_flakey: catastrophic mode - not reading inode or group bitmaps
STAT a|m|ctime 1580286853 1580286855 1580286855
MDT a|m|ctime 1580286853 1580286855 1580286855
sanityn test_104: @@@@@@ FAIL: /mnt/lustre/f104.sanityn mtime (1580286855:1580286853) diff
{noformat}

The patch for ~~LU-12026~~, https://review.whamcloud.com/#/c/36286/, added test 104 to sanityn. It landed to master on October 2019 and landed to b2_12 on 17 JAN 2020.

sanityn test 104 does not have a Lustre version check, but needs one. Master, future 2.14.0, interop testing runs with 2.12.3 and 2.13.0. 2.13.0 already has this patch and 2.12.4 has this patch. So, when we move interop testing for master to use 2.12.4, we don’t have to worry about master interop testing hitting this issue. The only other release we need to worry about is the LTS branch b2_12. The 2.12 branch runs interop testing with 2.11.0 and 2.10.8. Thus, if we land a patch to b2_12 that checks the server version to make sure it is 2.12.3.110 or later, then all b2_12 interop testing should skip this test when appropriate.

New: sanityn test_104 fails for clients with Lustre versions that are newer than the client version starting with Lustre 2.12.3 servers and 2.12.58.171 clients starting on 24 OCT 2019; see https://testing.whamcloud.com/test_sets/591b7932-f793-11e9-a197-52540065bddc and https://testing.whamcloud.com/test_sets/d58f7e54-f767-11e9-add9-52540065bddc.

Looking at a recent failure, https://testing.whamcloud.com/test_sets/8d9bdf58-4333-11ea-86c7-52540065bddc, for Lustre servers 2.11.0 and clients 2.12.3.109, we see the mtimes produced
{noformat}
CMD: trevis-51vm12 debugfs -c -R 'stat ROOT/f104.sanityn' /dev/mapper/mds1_flakey
trevis-51vm12: debugfs 1.45.2.wc1 (27-May-2019)
trevis-51vm12: /dev/mapper/mds1_flakey: catastrophic mode - not reading inode or group bitmaps
STAT a|m|ctime 1580286853 1580286853 1580286853
MDT a|m|ctime 1580286853 1580286853 1580286853
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 0.00156288 s, 655 kB/s
CMD: trevis-51vm12 debugfs -c -R 'stat ROOT/f104.sanityn' /dev/mapper/mds1_flakey
trevis-51vm12: debugfs 1.45.2.wc1 (27-May-2019)
trevis-51vm12: /dev/mapper/mds1_flakey: catastrophic mode - not reading inode or group bitmaps
STAT a|m|ctime 1580286853 1580286855 1580286855
MDT a|m|ctime 1580286853 1580286855 1580286855
sanityn test_104: @@@@@@ FAIL: /mnt/lustre/f104.sanityn mtime (1580286855:1580286853) diff
{noformat}

The patch for ~~LU-12026~~, https://review.whamcloud.com/36286/, added test 104 to sanityn. It landed to master on October 2019 and landed to b2_12 on 17 JAN 2020.

sanityn test 104 does not have a Lustre version check, but needs one. Master, future 2.14.0, interop testing runs with 2.12.3 and 2.13.0. 2.13.0 already has this patch and 2.12.4 has this patch. So, when we move interop testing for master to use 2.12.4, we don’t have to worry about master interop testing hitting this issue. The only other release we need to worry about is the LTS branch b2_12. The 2.12 branch runs interop testing with 2.11.0 and 2.10.8. Thus, if we land a patch to b2_12 that checks the server version to make sure it is 2.12.3.110 or later, then all b2_12 interop testing should skip this test when appropriate.

James Nunez (Inactive) made changes - 04/Feb/20 4:08 AM

Assignee

Original: WC Triage [ wc-triage ]

New: James Nunez [ jamesanunez ]

James Nunez (Inactive) created issue - 03/Feb/20 11:42 PM

People

Assignee:: James Nunez (Inactive)

Reporter:: James Nunez (Inactive)

Votes:: 0 Vote for this issue

Watchers:: 4 Start watching this issue

Dates

Created:: 03/Feb/20 11:42 PM

Updated:: 08/Feb/20 5:50 AM

Resolved:: 07/Feb/20 4:00 AM