[LU-7660] FS default striping settings only honored on MDT 0

• Type: Bug
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.9.0
• Affects Version/s: Lustre 2.5.0, Lustre 2.7.0, Lustre 2.8.0
• Labels: dne
• Severity: 3
• Assignee: Lai Siyao
• Reporter: John Hammond

Description

From the manual "Setting the striping specification on the root directory determines the striping for all new files created in the file system unless an overriding striping specification takes precedence (such as a striping layout specified by the application, or set using lfs setstripe, or specified for the parent directory)." In a DNE setup, setting the default striping on the root directory only affects files created on MDT 0. See ll_dir_setstripe():

                ...
                buf = param;
                /* Get fsname and assume devname to be -MDT0000. */
                ll_get_fsname(inode->i_sb, buf, MTI_NAME_MAXLEN);
                strcat(buf, "-MDT0000.lov");
		buf += strlen(buf);

                /* Set root stripesize */
                sprintf(buf, ".stripesize=%u",
			lump ? le32_to_cpu(lump->lmm_stripe_size) : 0);
                rc = ll_send_mgc_param(mgc->u.cli.cl_mgc_mgsexp, param);
                if (rc)
		        GOTO(end, rc);
                ...

Issue Links

• has to be done before: LU-8159 cache xattr in ldiskfs OSD
• is duplicated by: LU-8264 lfs setstripe without -p pool_name doesn't inherit pool from parent/ROOT directory
• is related to: LU-8092 racy striping & default striping cache in LOD
• is related to: LU-7661 MGS_SET_INFO handler is too permissive
• is related to: LU-8454 non-root user is able to change stripe size for mount point
• is related to: LU-5676 DNE 2: cache LMV EA in LOD
• is related to: LU-7813 default pool not inherited when specifying any layout parameter
• is related to: LU-8653 broken inheritance of default striping
• is related to: LU-7335 store default filesystem layout directly on root inode

Comments

[Joseph Gmitter (Inactive)]

Patch has landed to master for 2.9.0

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch http://review.whamcloud.com/19041/
Subject: LU-7660 dne: support fs default stripe
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 226fd401f9d8bfcd1a71bf264d9baef1e0842441

[Lai Siyao]

LU-8159 has been created for it.

[Andreas Dilger]

Lai, can you please also file a new LU ticket about caching the xattrs in osd-ldiskfs. It would be useful to do a before/after mdtest to see if this is needed or not.

[Gerrit Updater]

Lai Siyao ([e-mail]) uploaded a new patch: http://review.whamcloud.com/19041
Subject: LU-7660 dne: support fs default stripe
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: aeda66065b36048147c969aac4b53cb28dfaf1f4

[Andreas Dilger]

Ideally, the fetching of the default layout xattr from the root directory can be handled (mostly?) transparently via LOD, and the only real change that is needed is for the other MDTs to know the root FID of the filesystem. Ideally, this would be handled by replicating the root directory (and the default layout xattr) to all MDTs so that it can be load balanced and MDT0000 is not a single point of failure, essentially a form of DNE2 striped directory that is instead (or in addition) mirrored. However, I expect that may be too much work to fix this problem. In the short term, it seems that the only information needed is the root directory FID, and sufficient handling in the LOD/OSP layers (DLM locking and xattr caching in particular) that will allow the MDT to fetch this xattr from MDT0000.

[Niu Yawei (Inactive)]

I vote for the second method (set default striping in root inode, and let slave MDTs to fetch it on-demand), since I think config log isn't a perfect place to store a composite layout template and treating root inode setting as fs-wide setting looks natural to me.

[Di Wang]

I am not sure what you mean two different mechanisms here? TBH, I do not see any problem to regard default FS striping as the configuration of the filesystem, that said I do not see the problem to spread this by config log. Plus this will only need a few lines of changes. i.e. add config records for all MDTs in ll_dir_setstripe().

The concern I have for extending such changes by the ldlm lock of a special FID on MDT0 is that it would make MDT0 more unique, which might be a problem when we want FS to be functional without MDT0. Though I do not hold my strong opinion here.

[Alex Zhuravlev]

I mean that having two different mechanism to store essentially same information doesn't really help. we check one FID to take defaults, just add another FID (predefined) ? the approach with LDLM and regular storage looks great to me. the same way we could store/cache other things (like FLDB).

[Di Wang]

you mean inherit fs-default layout on each directory creation? why would we do that?