[LU-16806] mkdir: cannot create directory ‘test5’: Object is remote

• Type: Bug
• Resolution: Fixed
• Priority: Major
• Affects Version/s: Lustre 2.14.0, Lustre 2.15.2
• Labels: llnl
• Environment: server (garter): ; toss 4.5-6rc6 ; 4.18.0-425.19.2.1toss.t4.x86_64 ; 2.14.0_19.llnl ; ; client (mutt): ; toss 4.5-6 ; 4.18.0-425.19.2.1toss.t4.x86_64 ; 2.15.2_5.llnl
• Severity: 3
• Assignee: Lai Siyao
• Reporter: Gian-Carlo Defazio

Description

Attempts to create directories are intermittently failing when using "mkdir".

So far, "lfs mkdir" succeeds every time, so does file creation using "touch".

 

Comments

[Gian-Carlo Defazio]

The patch fixed the issue on 2.14.

I'm a bit confused by what the patch is doing.

Is the idea that when (lds->lds_dir_def_striping_set == 0) that means that the parent directory doesn't have default striping set for its sub-directories, so when the sub-directory is created it should instead inherit from the root of the whole filesystem?

 

[Lai Siyao]

Yes, upgrading to 2.15.x can get around this issue since related code has been removed.

[Peter Jones]

So LLNL's exposure to this issue will disappear once they upgrade to 2.15.x?

[Lai Siyao]

This is caused by a bug in 2.14 code, but it's not an issue in the following release because related code has been removed.

[Gerrit Updater]

"Lai Siyao <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/51045
Subject: LU-16806 lod: access DMV when it's valid
Project: fs/lustre-release
Branch: b2_14
Current Patch Set: 1
Commit: 65887f6728d68d8b2e10969df25e5303603a4797

[Gian-Carlo Defazio]

I've added mkdir-attempt-client-and-mds.tar.gz which has debug logs for the client and mds's with debug=-1.

[Lai Siyao]

Could you collect MDS debug logs upon failure? I'm afraid some patch for backward compatibility is missing, maybe it's on server side.

[Olaf Faaland]

Hi Serguei or Peter,

Can you ask an appropriate Whamcloud-er to advise?  The two test clusters involved will be taken away from us for other testing on Tuesday or Wednesday of next week.

thanks

[Serguei Smirnov]

It looks like this one needs attention of experts in higher-level Lustre. 

For example, is LUDOC-289 relevant here?

[Peter Jones]

Serguei

Can you please advise?

Thanks

Peter

[Gian-Carlo Defazio]

I've uploaded debug logs for the attempted creation of the directory junk8

debug log which contains failed attempt for junk8 is
dk.mutt12.1683589208

It then succeeded (after failing about 10 times) in
dk.mutt12.1683589684