[LU-17985] replay-single test_80b: (osd_io.c:2208:osd_write()) ASSERTION( inode != ((void *)0) )

• Type: Bug
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.16.0, Lustre 2.15.6
• Affects Version/s: Lustre 2.15.5
• Severity: 3
• Assignee: Hongchao Zhang
• Reporter: Maloo

Description

This issue was created by maloo for Minh Diep <[e-mail]>

This issue relates to the following test suite run: https://testing.whamcloud.com/test_sets/e443cbe4-4ba4-47d1-abec-03d27dec4726

test_80b failed with the following error:

rundbench load on trevis-62vm1.trevis.whamcloud.com,trevis-62vm2 failed!

Test session details:
clients: https://build.whamcloud.com/job/lustre-b2_15/93 - 4.18.0-477.27.1.el8_8.x86_64
servers: https://build.whamcloud.com/job/lustre-b2_15/93 - 4.18.0-477.27.1.el8_lustre.x86_64

<<Please provide additional information about the failure here>>

[ 8587.295279] LustreError: 700920:0:(osd_io.c:2208:osd_write()) ASSERTION( inode != ((void *)0) ) failed:
[ 8587.297122] LustreError: 700920:0:(osd_io.c:2208:osd_write()) LBUG
[ 8587.298329] Pid: 700920, comm: mdt_out00_003 4.18.0-477.27.1.el8_lustre.x86_64 #1 SMP Wed Jun 26 19:11:29 UTC 2024
[ 8587.300274] Call Trace TBD:
[ 8587.301039] [<0>] libcfs_call_trace+0x6f/0xa0 [libcfs]
[ 8587.302157] [<0>] lbug_with_loc+0x3f/0x70 [libcfs]
[ 8587.303121] [<0>] osd_write+0x89f/0xcd0 [osd_ldiskfs]
[ 8587.304240] [<0>] dt_record_write+0x32/0x110 [obdclass]
[ 8587.305550] [<0>] out_tx_write_exec+0x92/0x390 [ptlrpc]
[ 8587.307034] [<0>] out_tx_end+0x166/0x5f0 [ptlrpc]
[ 8587.308070] [<0>] out_handle+0x17e6/0x2180 [ptlrpc]
[ 8587.309130] [<0>] tgt_request_handle+0xccd/0x1b10 [ptlrpc]
[ 8587.310297] [<0>] ptlrpc_server_handle_request+0x323/0xbe0 [ptlrpc]
[ 8587.311603] [<0>] ptlrpc_main+0xc0f/0x1560 [ptlrpc]
[ 8587.312682] [<0>] kthread+0x134/0x150
[ 8587.313500] [<0>] ret_from_fork+0x35/0x40
[ 8587.314337] Kernel panic - not syncing: LBUG
[ 8587.315185] CPU: 1 PID: 700920 Comm: mdt_out00_003 Kdump: loaded Tainted: G OE --------- - - 4.18.0-477.27.1.el8_lustre.x86_64 #1
[ 8587.317539] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
[ 8587.318671] Call Trace:
[ 8587.319212] dump_stack+0x41/0x60
[ 8587.319900] panic+0xe7/0x2ac
[ 8587.320538] ? ret_from_fork+0x35/0x40
[ 8587.321290] lbug_with_loc.cold.8+0x18/0x18 [libcfs]
[ 8587.322275] osd_write+0x89f/0xcd0 [osd_ldiskfs]
[ 8587.323207] ? __ldiskfs_journal_stop+0x36/0xb0 [ldiskfs]
[ 8587.324380] ? osd_trans_exec_check+0x18/0x160 [osd_ldiskfs]
[ 8587.325504] ? osd_trans_exec_check+0x18/0x160 [osd_ldiskfs]
[ 8587.326610] dt_record_write+0x32/0x110 [obdclass]
[ 8587.327597] out_tx_write_exec+0x92/0x390 [ptlrpc]
[ 8587.328621] out_tx_end+0x166/0x5f0 [ptlrpc]
[ 8587.329553] out_handle+0x17e6/0x2180 [ptlrpc]
[ 8587.330513] ? lustre_msg_get_opc+0x7b/0xf0 [ptlrpc]
[ 8587.331586] tgt_request_handle+0xccd/0x1b10 [ptlrpc]
[ 8587.332662] ptlrpc_server_handle_request+0x323/0xbe0 [ptlrpc]
[ 8587.333866] ptlrpc_main+0xc0f/0x1560 [ptlrpc]
[ 8587.334826] ? ptlrpc_wait_event+0x590/0x590 [ptlrpc]
[ 8587.335894] kthread+0x134/0x150
[ 8587.336561] ? set_kthread_struct+0x50/0x50
[ 8587.337391] ret_from_fork+0x35/0x40

VVVVVVV DO NOT REMOVE LINES BELOW, Added by Maloo for auto-association VVVVVVV
replay-single test_70b - rundbench load on trevis-62vm1.trevis.whamcloud.com,trevis-62vm2 failed!

Comments

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/56940/
Subject: LU-17985 osd-ldiskfs: drop osd object if failed to create
Project: fs/lustre-release
Branch: b2_15
Current Patch Set:
Commit: 6b5bd48a7a0896e6eea00dd0b982ea2a08040f03

[Gerrit Updater]

"Jian Yu <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/56940
Subject: LU-17985 osd-ldiskfs: drop osd object if failed to create
Project: fs/lustre-release
Branch: b2_15
Current Patch Set: 1
Commit: 50c26cc6dd18927c801044f45ab8b2d72c813b86

[Peter Jones]

Merged for 2.16

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/55571/
Subject: LU-17985 osd-ldiskfs: drop osd object if failed to create
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 40e27b4251bec6d60ce0a6310a5ac7094980f9a3

[Gerrit Updater]

"Hongchao Zhang <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/55571
Subject: LU-17985 osd-ldiskfs: drop osd object if failed to create
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 5ad441850aa83f9df9b345cc780a3609f624a196

[Hongchao Zhang]

As per the current codes, the osd_object->oo_inode is set as NULL in 3 cases
1, osd_fid_lookup
the object existing flag LOHA_EXISTS will be set if the inode exists after calling osd_fid_lookup

2, osd_object_delete
it will be called after the lu_object is being deleted

3, osd_create
If the newly created inode had already contained correct XATTR_NAME_LMA but the OI insert was failed,
it will set osd_object->oo_inode = NULL while LOHA_EXISTS was still set

static int osd_create(const struct lu_env *env, struct dt_object *dt,
                      struct lu_attr *attr, struct dt_allocation_hint *hint,
                      struct dt_object_format *dof, struct thandle *th)
{
        ...
        result = __osd_create(info, obj, attr, hint, dof, th);
        if (result == 0) {
                if (fid_is_idif(fid) &&
                    !osd_dev(dt->do_lu.lo_dev)->od_index_in_idif) {
                        ...
                } else {
                        on_ost = fid_is_on_ost(info, osd_obj2dev(obj),
                                               fid, OI_CHECK_FLD);
                        result = osd_ea_fid_set(info, obj->oo_inode, fid,
                                                on_ost ? LMAC_FID_ON_OST : 0,
                                                0);
                }
                if (obj->oo_dt.do_body_ops == &osd_body_ops_new)
                        obj->oo_dt.do_body_ops = &osd_body_ops;
        }

        if (!result && !CFS_FAIL_CHECK(OBD_FAIL_OSD_NO_OI_ENTRY)) {
                struct inode *inode = obj->oo_inode;

                result = __osd_oi_insert(env, obj, fid, th);
                if (result && inode) {
                        ...                        
                        obj->oo_inode = NULL;
                }
        }
        ...
}

Is the flag LOHA_EXISTS needed to be cleared here?

[Peter Jones]

Hongchao

Could you please investigate?

Thanks

Peter