[LU-4690] sanity test_4: Expect error removing in-use dir /mnt/lustre/remote_dir

• Status: Resolved
• Project: Lustre
• Affects Version/s: Lustre 2.6.0, Lustre 2.5.1, Lustre 2.5.2
• Fix Version/s: Lustre 2.6.0, Lustre 2.5.2
• Type: Bug
• Priority: Blocker
• Reporter: Maloo
• Assignee: Di Wang
• Resolution: Fixed
• Labels: dne2
• Severity: 3

Description

This issue was created by maloo for Bob Glossman <[e-mail]>

This issue relates to the following test suite run: http://maloo.whamcloud.com/test_sets/0dd7df32-a0b5-11e3-9f3a-52540035b04c.

The sub-test test_4 failed with the following error:

Expect error removing in-use dir /mnt/lustre/remote_dir

There have been many of these lately. maloo reports:

Failure Rate: 31.00% of last 100 executions [all branches]

A search shows failures like this started happening 2/26, none earlier.
Suspect something landed just recently causing these failures.

Info required for matching: sanity 4

Issue Links

• is related to: LU-4471 Failed 'rmdir' on remote directories ...
• is related to: LU-4215 Some expected improvements for OUT
• is related to: LU-3696 sanity test_17m, test_17n: e2fsck una...
• is related to: LU-5296 lod_attr_set() skips attr_set on osp ...
• is related to: LU-5675 qsd_reint_index(): II_FL_NONUNQ is se...
• is related to: LU-5016 clean up use of la_attr for non-attri...

Comments

[Bob Glossman (Inactive)]

I see that test_4 was only added to sanity.sh very recently, in commit 1f55f2a9071d5e7db4042b959723086dee1c379a, LU-4471 mdd: mdd_unlink: do trans_start after sanity check.

That would explain why all failures are recent. Strongly suspect something is very wrong with this new subtest.

[Bob Glossman (Inactive)]

I note that this change was back ported into b2_5 at nearly the same time. I'll bet it's causing test failures there too.

[Di Wang]

Hmm, right now we will check whether the object is empty during getattr. But it is missing for striped dir. And also current empty check for remote directory is kind of jacky, given that we already have remote object iteration from LFSCK. I will check what I can do here. And in the mean time, I would suggest to disable the test.

[Di Wang]

Disable LU-4690 temporarily. http://review.whamcloud.com/#/c/9440/

[Di Wang]

http://review.whamcloud.com/9511

[Jodi Levi (Inactive)]

I have retriggered Change, 9511 test.

[Jodi Levi (Inactive)]

Does this test need to be re-enabled now that Change, 9511 has landed? Or can this ticket be closed?

[Di Wang]

There is another patch http://review.whamcloud.com/#/c/10261/ to cleanup the 9511 a bit. Probably close the ticket after 10261 is landed?

[Jodi Levi (Inactive)]

Patches landed to Master.

[Niu Yawei (Inactive)]

Looks the patch caused serious problem with quota:

In lod_attr_set():

        for (i = 0; i < lo->ldo_stripenr; i++) {
                LASSERT(lo->ldo_stripe[i]);
+               if (dt_object_exists(lo->ldo_stripe[i]) == 0)
+                       continue;
                rc = dt_attr_set(env, lo->ldo_stripe[i], attr, handle, capa);

I think dt_object_exists() will always return false for osp object, then any chown/chgrp will never be applied to OST objects. Any thoughts?

Because test_34 of s-q was disabled for LU-4515, this problem wasn't found by maloo test, I've submit a patch to re-enable test_34 in LU-4515.

[Di Wang]

I think dt_object_exists() will always return false for osp object, then any chown/chgrp will never be applied to OST objects. Any thoughts?

Hmm, we actually separate the remote and exist flag for MD OSP object, but for OST object, this check might have problem. The easiest fix might be add S_ISDIR(dt->do_lu.lo_header->loh_attr) check here?

[Niu Yawei (Inactive)]

Thank you, Di. I'll try to compose a patch according to your comment and track the fix in LU-5296.