[LU-14457] racer test_1: crash with directory migration

• Status: Open
• Project: Lustre
• Affects Version/s: Lustre 2.14.0
• Type: Bug
• Priority: Minor
• Reporter: Maloo
• Assignee: WC Triage
• Resolution: Unresolved
• Severity: 3

Description

This issue was created by maloo for Andreas Dilger <[e-mail]>

This issue relates to the following test suite run: https://testing.whamcloud.com/test_sets/07632945-d143-4352-9ed9-297c98f1fdce

test_1 crashed after running for 3100s with the following error:

[ 3557.862509] LustreError: 31802:0:(lu_object.h:885:lu_object_attr()) ASSERTION( ((o)->lo_header->loh_attr & LOHA_EXISTS) != 0 ) failed: 
[ 3557.866245] LustreError: 31802:0:(lu_object.h:885:lu_object_attr()) LBUG
[ 3557.888908] Kernel panic - not syncing: LBUG
[ 3557.889650] CPU: 1 PID: 31802 Comm: mdt00_075 Kdump: loaded  3.10.0-1127.19.1.el7_lustre.x86_64 #1
[ 3557.891877] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
[ 3557.892830] Call Trace:
[ 3557.893247]  [<ffffffffb877ffa5>] dump_stack+0x19/0x1b
[ 3557.894137]  [<ffffffffb8779541>] panic+0xe8/0x21f
[ 3557.894929]  [<ffffffffc097d9eb>] lbug_with_loc+0x9b/0xa0 [libcfs]
[ 3557.895951]  [<ffffffffc131e7a0>] lu_object_attr.isra.21.part.22+0x36/0x36 [mdt]
[ 3557.897156]  [<ffffffffc12e0bb3>] mdt_link_parents_lock.isra.53+0x14c3/0x15a0 [mdt]
[ 3557.898477]  [<ffffffffc145e19b>] ? __mdd_lookup.isra.17+0x28b/0x430 [mdd]
[ 3557.899810]  [<ffffffffc0bbe3e9>] ? lu_object_find+0x19/0x20 [obdclass]
[ 3557.900900]  [<ffffffffc12d565a>] ? mdt_migrate_lookup.isra.42+0x47a/0x870 [mdt]
[ 3557.902171]  [<ffffffffc12be2ca>] ? mdt_remote_object_lock+0x2a/0x30 [mdt]
[ 3557.903298]  [<ffffffffc12e1545>] mdt_reint_migrate+0x8b5/0xfb0 [mdt]
[ 3557.905607]  [<ffffffffc12e1cc3>] mdt_reint_rec+0x83/0x210 [mdt]
[ 3557.906597]  [<ffffffffc12b9a30>] mdt_reint_internal+0x720/0xaf0 [mdt]
[ 3557.907667]  [<ffffffffc12c55c7>] mdt_reint+0x67/0x140 [mdt]
[ 3557.908671]  [<ffffffffc0f136fa>] tgt_request_handle+0x7ea/0x1750 [ptlrpc]
[ 3557.911001]  [<ffffffffc0eb31a6>] ptlrpc_server_handle_request+0x256/0xb10 [ptlrpc]
[ 3557.912283]  [<ffffffffc0eb7cfc>] ptlrpc_main+0xb3c/0x14e0 [ptlrpc]
[ 3557.914543]  [<ffffffffb80c6691>] kthread+0xd1/0xe0

Since this is during racer, it is entirely possible that the source or target file was removed at the same time.

VVVVVVV DO NOT REMOVE LINES BELOW, Added by Maloo for auto-association VVVVVVV
racer test_1 - trevis-55vm4 crashed during racer test_1

Issue Links

• is related to: LU-6963 racer test_1: lu_object_attr() ASSERT...
• is related to: LU-7145 mdd_object_type() uses need audit

Comments

[Alex Zhuravlev]

hitting this as well:

 LustreError: 9866:0:(lu_object.h:883:lu_object_attr()) ASSERTION( ((o)->lo_header->loh_attr & LOHA_EXISTS) != 0 ) failed: in racer / 1

Trace:

PID: 9866   TASK: ffff8800da804000  CPU: 1   COMMAND: "mdt00_012"
 #0 [ffff8800da827a78] panic at ffffffff810af881
    /tmp/kernel/kernel/panic.c: 265
 #1 [ffff8800da827af8] mdt_link_parents_lock at ffffffffa0c91ade [mdt]
    /home/lustre/master-mine/lustre/include/lu_object.h: 883
 #2 [ffff8800da827be0] mdt_reint_migrate at ffffffffa0c93735 [mdt]
    /home/lustre/master-mine/lustre/mdt/mdt_reint.c: 2258
 #3 [ffff8800da827d18] mdt_reint_rec at ffffffffa0c95eb7 [mdt]
    /home/lustre/master-mine/lustre/mdt/mdt_reint.c: 3100
 #4 [ffff8800da827d48] mdt_reint_internal at ffffffffa0c6b24c [mdt]
    /home/lustre/master-mine/lustre/mdt/mdt_handler.c: 2760
 #5 [ffff8800da827d80] mdt_reint at ffffffffa0c76d37 [mdt]
    /home/lustre/master-mine/lustre/mdt/mdt_handler.c: 2834
 #6 [ffff8800da827da8] tgt_request_handle at ffffffffa05904b0 [ptlrpc]
    /home/lustre/master-mine/lustre/include/lu_target.h: 618
 #7 [ffff8800da827e20] ptlrpc_main at ffffffffa05469b0 [ptlrpc]
    /home/lustre/master-mine/lustre/include/lustre_net.h: 2379
 #8 [ffff8800da827f10] kthread at ffffffff810d02aa
    /tmp/kernel/kernel/kthread.c: 246
 #9 [ffff8800da827f50] ret_from_fork at ffffffff818001c4
    /tmp/kernel/arch/x86/entry/entry_64.S: 422