[LU-6200] Failover recovery-mds-scale test_failover_ost: test_failover_ost returned 1

• Type: Bug
• Resolution: Duplicate
• Priority: Major
• Affects Version/s: Lustre 2.7.0, Lustre 2.8.0, Lustre 2.10.0, Lustre 2.11.0, Lustre 2.10.4
• Labels: p4hc
• Environment: client and server: lustre-master build # 2835 RHEL6
• Severity: 3
• Assignee: Hongchao Zhang
• Reporter: Maloo

Description

This issue was created by maloo for sarah <[e-mail]>

This issue relates to the following test suite run: https://testing.hpdd.intel.com/test_sets/be3ebe76-a817-11e4-93dd-5254006e85c2.

The sub-test test_failover_ost failed with the following error:

test_failover_ost returned 1

client 3 shows

tar: etc/sysconfig/quota_nld: Cannot write: No such file or directory
tar: etc/sysconfig/quota_nld: Cannot utime: No such file or directory
tar: etc/sysconfig/sandbox: Cannot write: No such file or directory
tar: etc/sysconfig/nfs: Cannot write: No such file or directory
tar: Exiting with failure status due to previous errors

Issue Links

• is duplicated by: LU-463 orphan recovery happens too late, causing writes to fail with ENOENT after recovery
• is related to: LU-5483 recovery-mds-scale test failover_mds: oom failure on client
• is related to: LU-5526 recovery-mds-scale test failover_mds: dd: No space left on device

Comments

[Hongchao Zhang]

Resolved as duplicate of LU-11765

[Hongchao Zhang]

Hi Sergey,

Thanks!
This should be the same as LU-11765, only the fixing ways are different, the patch in this ticket recreates the object
if it doesn't exist and the patch in LU-11765 returns -EAGAIN to notify the caller to retry.

[Sergey Cheremencev]

Hi,

Description of the problem looks similar with I've already fixed in https://review.whamcloud.com/#/c/33836/ .
Please look carefully and If I am right this could be resolved as a dup of LU-11765.

[Hongchao Zhang]

the patch https://review.whamcloud.com/#/c/13668/ has been updated

[James Casper (Inactive)]

2.11.0:
https://testing.hpdd.intel.com/test_sessions/e6578085-2eed-486d-8601-e5214bac4bb0

[Hongchao Zhang]

the patch https://review.whamcloud.com/#/c/13668/ has been updated.

[Hongchao Zhang]

the patch has been updated as per the review feedback.

[Alexey Lyashkov]

Hongchao,

i don't have an access to gerrit now, but you patch
https://git.hpdd.intel.com/?p=fs/lustre-release.git;a=commitdiff;h=daa98c46817c98d6fbf70dafa9fbdde678f8b9ba;hp=32d1a1c5d610d054ad4609c1cf332172e8310805
is bad.

Looks, You can't use a

+ /* Do sync create if the seq is about to used up */
+ if (fid_seq_is_idif(seq) || fid_seq_is_mdt0(seq)) {
+ if (unlikely(oid >= IDIF_MAX_OID - 1))
+ sync = 1;

because ost id in this case need to account lower 16 bits from seq, please look to the ost id macros.

[Hongchao Zhang]

the patch https://review.whamcloud.com/#/c/13668/ has been updated.

[Saurabh Tandan (Inactive)]

Another instance found on b2_8 for failover testing , build# 6.
https://testing.hpdd.intel.com/test_sessions/0aed3028-da39-11e5-a8a6-5254006e85c2
https://testing.hpdd.intel.com/test_sessions/eaf85780-d65e-11e5-afe8-5254006e85c2
https://testing.hpdd.intel.com/test_sessions/54ec62da-d99d-11e5-9ebe-5254006e85c2
https://testing.hpdd.intel.com/test_sessions/eb9f29ec-d8da-11e5-83e2-5254006e85c2
https://testing.hpdd.intel.com/test_sessions/2f0aa9f6-d5a5-11e5-9cc2-5254006e85c2
https://testing.hpdd.intel.com/test_sessions/c5a8e44c-d9c7-11e5-85dd-5254006e85c2