[LU-5416] (ofd_objects.c:256:ofd_precreate_objects()) ASSERTION( nr > 0 ) failed

• Status: Closed
• Project: Lustre
• Affects Version/s: Lustre 2.7.0
• Fix Version/s: Lustre 2.7.0
• Type: Bug
• Priority: Minor
• Reporter: John Hammond
• Assignee: John Hammond
• Resolution: Fixed
• Labels: fault, ofd
• Severity: 3

Description

In ofd_precreate_objects() if dt_declare_create() fails in the first loop iteration then the assertion that nr > 0 will fail.

        for (i = 0; i < nr; i++) {
                fo = batch[i];
                LASSERT(fo);

                if (unlikely(ofd_object_exists(fo))) {
                        /* object may exist being re-created by write replay */
                        CDEBUG(D_INODE, "object "LPX64"/"LPX64" exists: "
                               DFID"\n", ostid_seq(&oseq->os_oi), id,
                               PFID(lu_object_fid(&fo->ofo_obj.do_lu)));
                        continue;
                }

                next = ofd_object_child(fo);
                LASSERT(next != NULL);

                rc = dt_declare_create(env, next, &info->fti_attr, NULL,
                                       &info->fti_dof, th);
                if (rc) {
                        nr = i;
                        break;
                }
        }

        rc = dt_trans_start_local(env, ofd->ofd_osd, th);
        if (rc)
                GOTO(trans_stop, rc);

        CDEBUG(D_OTHER, "%s: create new object "DFID" nr %d\n",
               ofd_name(ofd), PFID(fid), nr);

        LASSERT(nr > 0);

[62687.401108] LustreError: 28373:0:(ofd_objects.c:256:ofd_precreate_objects()) ASSERTION\
( nr > 0 ) failed:
[62687.404317] LustreError: 28373:0:(ofd_objects.c:256:ofd_precreate_objects()) LBUG
[62687.406798] Pid: 28373, comm: ll_ost01_007
[62687.407690]
[62687.407691] Call Trace:
[62687.408622]  [<ffffffffa0f068c5>] libcfs_debug_dumpstack+0x55/0x80 [libcfs]
[62687.410080]  [<ffffffffa0f06ec7>] lbug_with_loc+0x47/0xb0 [libcfs]
[62687.411414]  [<ffffffffa0c9322b>] ofd_precreate_objects+0x13fb/0x19c0 [ofd]
[62687.412881]  [<ffffffffa0c8d80e>] ? ofd_precreate_batch+0x2e/0x60 [ofd]
[62687.414264]  [<ffffffffa0c82024>] ofd_create_hdl+0x524/0x2400 [ofd]
[62687.415682]  [<ffffffffa0591952>] ? lustre_pack_reply_v2+0x232/0x2a0 [ptlrpc]
[62687.417219]  [<ffffffffa05f2445>] tgt_request_handle+0x245/0xad0 [ptlrpc]
[62687.418708]  [<ffffffffa05a2e01>] ptlrpc_main+0xce1/0x1960 [ptlrpc]
[62687.420057]  [<ffffffffa05a2120>] ? ptlrpc_main+0x0/0x1960 [ptlrpc]
[62687.421406]  [<ffffffff8109eab6>] kthread+0x96/0xa0
[62687.422451]  [<ffffffff8100c30a>] child_rip+0xa/0x20
[62687.423494]  [<ffffffff81554710>] ? _spin_unlock_irq+0x30/0x40
[62687.424698]  [<ffffffff8100bb10>] ? restore_args+0x0/0x30
[62687.425844]  [<ffffffff8109ea20>] ? kthread+0x0/0xa0
[62687.426921]  [<ffffffff8100c300>] ? child_rip+0x0/0x20
[62687.427987]
[62687.428673] Kernel panic - not syncing: LBUG

This issue was found through DT API fault injection.

Comments

[John Hammond]

The second for loop in ofd_precreate_objects() has the same type of issue. If dt_create() fails in the first iteration then we get

Message from syslogd@q at Jul 29 14:11:34 ...
 kernel:[  835.526433] LustreError: 6084:0:(ofd_objects.c:406:ofd_precreate_objects()) ASSERTION( (!(objects == 0) || (rc < 0)) ) failed: 

Message from syslogd@q at Jul 29 14:11:34 ...
 kernel:[  835.529381] LustreError: 6084:0:(ofd_objects.c:406:ofd_precreate_objects()) LBUG
Write failed: Broken pipe

[John Hammond]

Please see http://review.whamcloud.com/11370.

[John Hammond]

Patch landed to master.