[LU-5916] OST pool inheritence is no longer persistent (no LOV EA for child directory)

• Status: Resolved
• Project: Lustre
• Affects Version/s: Lustre 2.6.0, Lustre 2.7.0, Lustre 2.5.2, Lustre 2.4.3
• Fix Version/s: Lustre 2.7.0, Lustre 2.5.4
• Type: Bug
• Priority: Critical
• Reporter: Thomas LEIBOVICI - CEA (Inactive)
• Assignee: Niu Yawei (Inactive)
• Resolution: Fixed
• Labels: HB

Environment

el6
Lustre 2.4 to Lustre 2.7 (master).

• Severity: 3

Description

A child directory should inherit from the OST pool of its parent.

In Lustre 2.4 to master (2.7), this is no longer the case: the child directory has no LOV EA stored on the MDS, so the pool inheritance is lost when the MDS restarts.

lfs getstripe indicates the child directory doesn't inherit from its parent pool.

mkdir /mnt/lustre/dir
lfs setstripe -p lustre.pool1 /mnt/lustre/dir
lfs getstripe -p /mnt/lustre/dir
returns "lustre.pool1" (correct)
mkdir /mnt/lustre/dir/subdir
lfs getstripe -p /mnt/lustre/dir/subdir
-> empty (should return lustre.pool1)
getfattr -n trusted.lov -d /mnt/lustre/dir/subdir
/mnt/lustre/dir/subdir: trusted.lov: No such attribute
touch /mnt/lustre/dir/subdir/file
lfs getstripe -p /mnt/lustre/dir/subdir/file
returns "lustre.pool1" (correct). Its seams the MDS inherited the pool somehow.

Then, after restarting the MDS:
touch /mnt/lustre/dir/subdir/file2
lfs getstripe -p /mnt/lustre/dir/subdir/file2
-> empty (should be lustre.pool1)

Comments

[Peter Jones]

Niu

Could you please advise?

Thanks

Peter

[Niu Yawei (Inactive)]

I checked the code, looks we lost this when integrating LOD into the stack (that's from 2.3.51), because the default trusted.lov can still being inherited from parent and cached in memory, so creating file before restarting MDS should be fine.

See the comment from mdd_object_create():

        /* During creation, there are only a few cases we need do xattr_set to
         * create stripes.
         * 1. regular file: see comments above.
         * 2. create striped directory with provided stripeEA.
         * 3. create striped directory because inherit default layout from the
         * parent. */

Obviously we missed the 4th case that creating dir which inherits default striping (XATTR_NAME_LOV) from parent.

And seems there are some other problems in this code path:

In declare stage, see mdd_declare_object_create():

        /* replay case, create LOV EA from client data */
        if (spec->no_create ||
            (spec->sp_cr_flags & MDS_OPEN_HAS_EA && S_ISREG(attr->la_mode))) {
                const struct lu_buf *buf;

                buf = mdd_buf_get_const(env, spec->u.sp_ea.eadata,
                                        spec->u.sp_ea.eadatalen);
                rc = mdo_declare_xattr_set(env, c, buf, XATTR_NAME_LOV, 0,
                                           handle);
                if (rc)
                        GOTO(out, rc);
        }

It just missed declare for the case of inheriting parent's XATTR_NAME_LMV or XATTR_NAME_LOV.

In create stage, see mdd_object_create():

        if (spec->no_create ||
            (S_ISREG(attr->la_mode) && spec->sp_cr_flags & MDS_OPEN_HAS_EA) ||
            S_ISDIR(attr->la_mode)) {
                const struct lu_buf *buf;

                buf = mdd_buf_get_const(env, spec->u.sp_ea.eadata,
                                        spec->u.sp_ea.eadatalen);
                rc = mdo_xattr_set(env, son, buf,
                                   S_ISDIR(attr->la_mode) ? XATTR_NAME_LMV :
                                                            XATTR_NAME_LOV, 0,
                                   handle, BYPASS_CAPA);
                if (rc != 0)
                        GOTO(err_destroy, rc);
        }

It seems to me that when inheriting parent's XATTR_NAME_LMV, the passed spec->u.sp_ea.eadata should be NULL, I'm not sure how it worked. Di, could take a look to see if I missed anytying here?

[Gerrit Updater]

Niu Yawei ([e-mail]) uploaded a new patch: http://review.whamcloud.com/12746
Subject: LU-5916 lod: inherit default pool setting properly
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 2b237b11932af2b8dcf4a0401fc8e6c7cee37fbf

[Niu Yawei (Inactive)]

I looked into the code closer and realized that the NAME_LOV for default stripping/pool is declared in lod_declare_object_create() and the NAME_LOV is set in mdd_object_create() -> lod_xattr_set() -> lod_dir_striping_create(), but for the case of parent has only pool setting (no default stripping), inheriting NAME_LOV is skipped mistakenly, see lod_dir_striping_create_internal():

        /* Transfer default LOV striping from the parent */
        if (lo->ldo_striping_cached &&
            !LOVEA_DELETE_VALUES(lo->ldo_def_stripe_size,
                                 lo->ldo_def_stripenr,
                                 lo->ldo_def_stripe_offset)){

This patch fix the problem: http://review.whamcloud.com/#/c/12746

[Gerrit Updater]

Niu Yawei ([e-mail]) uploaded a new patch: http://review.whamcloud.com/12746
Subject: LU-5916 lod: inherit default pool setting properly
Project: fs/lustre-release
Branch: master
Current Patch Set: 2
Commit: 4c11c85c283bb200cadc492be42b298aefea1177

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch http://review.whamcloud.com/12746/
Subject: LU-5916 lod: inherit default pool setting properly
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: e8214d6bd4ccc3ff386c77904664c60f1848ef11

[Jodi Levi (Inactive)]

Patch landed to Master.

[Henri Doreau (Inactive)]

Could a backport to b2_5 be considered? The patch fixes a critical regression.

[Niu Yawei (Inactive)]

Sure, I'm going to port it to b2_5.

[Gerrit Updater]

Niu Yawei ([e-mail]) uploaded a new patch: http://review.whamcloud.com/13411
Subject: LU-5916 lod: inherit default pool setting properly
Project: fs/lustre-release
Branch: b2_5
Current Patch Set: 1
Commit: 840c3fa79c535578b17037ae9f3f6a4838255531

[Niu Yawei (Inactive)]

port to b2_5: http://review.whamcloud.com/#/c/13411/

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch http://review.whamcloud.com/13411/
Subject: LU-5916 lod: inherit default pool setting properly
Project: fs/lustre-release
Branch: b2_5
Current Patch Set:
Commit: 56211bf18fbca9930d77cea1ee0e3f2d83d59ec4