[LU-12209] cannot create stripe dir: Stale file handle

• Type: Bug
• Resolution: Duplicate
• Priority: Critical
• Affects Version/s: Lustre 2.10.7
• Environment: CentOS 7.6, servers 2.10.7, clients 2.12 or 2.10
• Severity: 3
• Assignee: Lai Siyao
• Reporter: Stephane Thiell

Description

I'm facing a new issue on Oak (2.10.7 servers), tried with both 2.10 and 2.12 clients:

As root:

# cd /oak/stanford/groups/
# lfs mkdir -i 1 caiwei
lfs mkdir: dirstripe error on 'caiwei': Stale file handle
lfs setdirstripe: cannot create stripe dir 'caiwei': Stale file handle

# lfs getdirstripe .
lmv_stripe_count: 0 lmv_stripe_offset: 0 lmv_hash_type: none

does that ring a bell? Oak is only using DNE v1 with statically striped directories. Never seen that before 2.10.7 (we recently upgraded Oak).

a basic lctl dk doesn't show anything on the MDS but I may have to enable specific debug flags to see more. No other traces found so far.

Tried with 2.10 and 2.12 clients, with or without idmap.

Thanks!

Issue Links

• duplicates: LU-11418 hung threads on MDT and MDT won't umount

Comments

[Peter Jones]

Stephane

You are correct. Yet another illustration as to why it is confusing to having multiple patches tracked under the same Jira ticket spanning release boundaries

Peter

[Stephane Thiell]

Peter, this patch (https://review.whamcloud.com/#/c/33401/ - LU-11418 llog: refresh remote llog upon -ESTALE) is already available in 2.12.0:

 

commit 71f409c9b31b90fa432f1f46ad4e612fb65c7fcc
Author: Lai Siyao <[e-mail]>
Date:   Wed Oct 17 13:29:53 2018 +0800

    LU-11418 llog: refresh remote llog upon -ESTALE

But it's not included in 2.10.7 (that we're running on our Oak servers).

[Peter Jones]

Nice! Thanks all. sthiell note that this fix is included in the upcoming 2.12.1

[Stephane Thiell]

Hi Lai,

We restarted the servers with the patch this morning and the problem is now gone. Thanks!

[Lai Siyao]

This looks to be the same issue which was fixed by https://review.whamcloud.com/#/c/33401/, can you apply this patch on all MDS's and try again?

[Stephane Thiell]

Thanks Patrick for this analysis. I see that obj->opo_stale = 1; only in osp_invalidate()...

Because it's not impacting production, but just new group creation, we won't failover the MDT today (new groups can wait a bit ). We have some interactive jobs running. But I'll try to find a good time during the weekend to do so. Let me know if you want me to grab more debug info before then.