[LU-3059] shrink lod_object to 128 bytes

• Type: Bug
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.4.0, Lustre 2.5.0, Lustre 2.6.0
• Affects Version/s: Lustre 2.4.0
• Labels: lod
• Business Value: -7
• Severity: 3
• Assignee: John Hammond
• Reporter: John Hammond

Description

Before:

struct lod_object {
        struct dt_object           ldo_obj;              /*     0    88 */
        /* --- cacheline 1 boundary (64 bytes) was 24 bytes ago --- */
        __u16                      ldo_stripenr;         /*    88     2 */
        __u16                      ldo_layout_gen;       /*    90     2 */
        __u32                      ldo_stripe_size;      /*    92     4 */
        char *                     ldo_pool;             /*    96     8 */
        struct dt_object * *       ldo_stripe;           /*   104     8 */
        int                        ldo_stripes_allocated; /*   112     4 */
        unsigned int               ldo_striping_cached:1; /*   116:31  4 */
        unsigned int               ldo_def_striping_set:1; /*   116:30  4 */

        /* XXX 30 bits hole, try to pack */

        __u32                      ldo_def_stripe_size;  /*   120     4 */
        __u16                      ldo_def_stripenr;     /*   124     2 */
        __u16                      ldo_def_stripe_offset; /*   126     2 */
        /* --- cacheline 2 boundary (128 bytes) --- */
        mdsno_t                    ldo_mds_num;          /*   128     4 */

        /* size: 136, cachelines: 3, members: 13 */
        /* bit holes: 1, sum bit holes: 30 bits */
        /* padding: 4 */
        /* last cacheline: 8 bytes */
};

After:

struct lod_object {
        struct dt_object           ldo_obj;              /*     0    88 */
        /* --- cacheline 1 boundary (64 bytes) was 24 bytes ago --- */
        __u16                      ldo_stripenr;         /*    88     2 */
        __u16                      ldo_layout_gen;       /*    90     2 */
        __u32                      ldo_stripe_size;      /*    92     4 */
        char *                     ldo_pool;             /*    96     8 */
        struct dt_object * *       ldo_stripe;           /*   104     8 */
        unsigned int               ldo_stripes_allocated:16; /*   112:16  4 */
        unsigned int               ldo_striping_cached:1; /*   112:15  4 */
        unsigned int               ldo_def_striping_set:1; /*   112:14  4 */

        /* XXX 14 bits hole, try to pack */

        __u32                      ldo_def_stripe_size;  /*   116     4 */
        __u16                      ldo_def_stripenr;     /*   120     2 */
        __u16                      ldo_def_stripe_offset; /*   122     2 */
        mdsno_t                    ldo_mds_num;          /*   124     4 */
        /* --- cacheline 2 boundary (128 bytes) --- */

        /* size: 128, cachelines: 2, members: 13 */
        /* bit holes: 1, sum bit holes: 14 bits */
};

Issue Links

• is related to: LU-2789 lod_load_striping()) ASSERTION( lo->ldo_stripenr == 0 ) failed

Comments

[John Hammond]

All four patches landed to master. On 2.4.52-37-g6350af1 compiled for x86_64 we have the following object sizes:

lu_object_header  88
cl_object_header 152

lu                40
cl                56
md                56
dt                64
mdd               80
lod              112
osd_ldiskfs      176 
osp              200
lov              224
ccc              248
mdt              256
osc              272

[John Hammond]

http://review.whamcloud.com/#change,5920 obdclass: use a dummy structure for lu_ref_link

[John Hammond]

http://review.whamcloud.com/5892 shrink dt_object by 8 bytes on x86_64

[John Hammond]

http://review.whamcloud.com/5890 shrink lu_object by 16 bytes on x86_64

[John Hammond]

Please see http://review.whamcloud.com/5878.

But then there definitely needs to be a lock in lod_object.

[Alex Zhuravlev]

we could also turn ldo_stripe + ldo_stripes_allocated + ldo_striping_cached + ldo_def_striping_set (probably more?) a separately allocated structure.
because most of time we don't use those fields at all.

then use SLAB to pack ldo_object structures better in a page.