[LU-15640] Setting project ID for existing directory fails after update from 2.12 to 2.14

• Status: Open
• Project: Lustre
• Affects Version/s: Lustre 2.14.0, Lustre 2.15.0
• Type: Bug
• Priority: Major
• Reporter: Robert Redl
• Assignee: WC Triage
• Resolution: Unresolved
• Labels: ZFS, quota

Environment

Kernel: 4.18.0-240.1.1.el8_lustre.x86_64
Lustre: 2.14.0-1.el8
ZFS: 2.0.0-1.el8

• Severity: 3

Description

MDS and OSS are installed with the 2.14 packages from the official repository. Project quotas are enabled following the documentation and they are working as expected for directories created after the upgrade from 2.12 to 2.14.

Setting a project-ID for an existing directory fails with the following error:

lfs project -s -p 1 -r /mnt/directory-created-before-upgrade
lfs: failed to set xattr for '/mnt/directory-created-before-upgrad': No such device or address

 
Is there any specific procedure required after an upgrade from 2.12 to 2.14 to make project quotas work on old directories?

Issue Links

• is related to: LU-13189 ASSERTION( obj->oo_with_projid ) fail...

Comments

[Etienne Aujames]

From the Lustre sources:

static int osd_declare_attr_set(const struct lu_env *env,
....
#ifdef ZFS_PROJINHERIT
        if (attr && attr->la_valid & LA_PROJID) {
                /* quota enforcement for project */
                if (attr->la_projid != obj->oo_attr.la_projid) {
                        if (!osd->od_projectused_dn)
                                GOTO(out, rc = -EOPNOTSUPP);

                        /* Usually, if project quota is upgradable for the
                         * device, then the upgrade will be done before or when
                         * mount the device. So when we come here, this project
                         * should have project ID attribute already (that is
                         * zero by default).  Otherwise, there was something
                         * wrong during the former upgrade, let's return failure
                         * to report that.
                         *
                         * Please note that, different from other attributes,
                         * you can NOT simply set the project ID attribute under
                         * such case, because adding (NOT change) project ID
                         * attribute needs to change the object's attribute
                         * layout to match zfs backend quota accounting
                         * requirement. */
                        if (unlikely(!obj->oo_with_projid))
                                GOTO(out, rc = -ENXIO);

                        rc = qsd_transfer(env, osd_def_qsd(osd),
                                          &oh->ot_quota_trans, PRJQUOTA,
                                          obj->oo_attr.la_projid,
                                          attr->la_projid, bspace,
                                          &info->oti_qi, true);
                        if (rc)
                                GOTO(out, rc);
                }
        }
#endif
....
static int __osd_object_attr_get(const struct lu_env *env, struct osd_device *o,
....
#ifdef ZFS_PROJINHERIT
        if (o->od_projectused_dn && osa->flags & ZFS_PROJID) {
                rc = -sa_lookup(obj->oo_sa_hdl, SA_ZPL_PROJID(o),
                                &osa->projid, 8);
                if (rc)
                        GOTO(out_sa, rc);
        
                la->la_projid = osa->projid;
                la->la_valid |= LA_PROJID;
                obj->oo_with_projid = 1;
        } else {
                la->la_projid = ZFS_DEFAULT_PROJID;
                la->la_valid &= ~LA_PROJID;
        }
#else           

[Etienne Aujames]

The corresponding commit:

commit 291e7196d39365739f9daa02efd25535b5415174
Author: Nathaniel Clark <[e-mail]>
Date:   Thu May 16 13:18:04 2019 -0400

    LU-12309 osd-zfs: Support disabled project quotas
    
    Allow project quotas to be compiled in but disabled in the zpool.
    This would be the case for zpools created by pre-0.8.0 ZFS, but then
    used with newer ZFS.

I am not sure if this code is involved but it seems likely.

[Robert Redl]

All pools of MDTs and OSTs have been upgraded and the feature is shown as active:

zpool get feature@project_quota pool
NAME  PROPERTY               VALUE                  SOURCE
pool  feature@project_quota  active                 local

Project quotas are also working as expected for directories that have been created after the upgrade. The problem is only, that it is not possible to assign a project ID to an old directory.

Is it possible that a specific upgrade procedure is required? I already tried lfsck, which did not change the situation. What helps is to migrate the directory using lfs migrate -m 0.

[Robert Redl]

The problem unfortunately persists with Lustre 2.15.0-RC5 and ZFS 2.0.7.

[Robert Redl]

The patch for https://jira.whamcloud.com/browse/LU-13189 is solving this issue as well.