[LU-7481] Failover: recovery-mds-scale test_failover_mds: /dev/lvm-Role_MDS/P1 failed to initialize!

• Type: Bug
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.10.0
• Affects Version/s: Lustre 2.8.0
• Environment: EL7 Server/SLES11 SP3 Client, Build# 3251
• Severity: 3
• Assignee: Hongchao Zhang
• Reporter: Maloo

Description

This issue was created by maloo for Saurabh Tandan <[e-mail]>

This issue relates to the following test suite run: https://testing.hpdd.intel.com/test_sets/c010ef0a-906d-11e5-a833-5254006e85c2.

The sub-test test_failover_mds failed with the following error:

/dev/lvm-Role_MDS/P1 failed to initialize!

Test log:

Update not seen after 90s: wanted '' got 'lustre:MDT0000'
 recovery-mds-scale test_failover_mds: @@@@@@ FAIL: /dev/lvm-Role_MDS/P1 failed to initialize! 

Looks similar to TEI-3682

Issue Links

• duplicates: LU-8729 conf-sanity test_84: FAIL: /dev/mapper/mds1_flakey failed to initialize!
• duplicates: LU-7428 conf-sanity test_84, replay-dual 0a: /dev/lvm-Role_MDS/P1 failed to initialize!
• is related to: LU-9059 mount.lustre FATAL: unhandled/unloaded fs type 0 'ext3'

Comments

[Peter Jones]

Landed for 2.10

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/24845/
Subject: LU-7481 utils: label lustre device correctly
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: a6e430e81669a6ab40ecae9b416dd2cdee45908c

[Gerrit Updater]

Hongchao Zhang ([e-mail]) uploaded a new patch: https://review.whamcloud.com/24845
Subject: LU-7481 utils: label lustre device correctly
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 1264e511ddf2229a3feaeda52323720176254be7

[Hongchao Zhang]

This should be similar problem, but not the same problem described in LU-7428, which was caused in "osd_ro". recovery-mds-scale has no such operation,
it only failover the MDS(or OST), which could cause such problem.

I have investigated the code lines of "e2fsprogs" and found it only read the buffer of the super_block of the device and write it back after some modification.
if there are no super_block related operations in the corresponding EXT4 filesystem, the buffer won't be written back to disk along with the journal, which can
answer why the freeze operation added in the patch http://review.whamcloud.com/20586/ in LU-7428 can fix the problem (ext4_freeze commits super_block).

I created a tentative patch at http://review.whamcloud.com/23935

[Gerrit Updater]

Hongchao Zhang ([e-mail]) uploaded a new patch: http://review.whamcloud.com/23935
Subject: LU-7481 utils: commit device label
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 27d82e7f30b105781565915a289545e222a7c5cb

[Saurabh Tandan (Inactive)]

Still hitting on master (2.8.57 commit 9f36c9a2fc6f8cfc99):
https://testing.hpdd.intel.com/test_sessions/a3cd7814-72ac-11e6-8afd-5254006e85c2

[Sarah Liu]

Hi Saurabh,

the patch for LU-7428 was landed on 13/Jul/16, while you reopened this ticket on 04/May/16, I think when you reopened the ticket, the patch didn't land at that time. close for now as a dup of LU-7428

[Saurabh Tandan (Inactive)]

Reopening the issue as even after LU-7428 been resolved this issue still persists.

[Andreas Dilger]

I suspect this is a duplicate of LU-7428.