[LU-5388] Interop 2.5.2<->2.6 failure on test suite replay-dual test_24: (lu_object.h:855:lu_object_attr()) ASSERTION( ((o)->lo_header->loh_attr & LOHA_EXISTS) != 0 ) failed

• Type: Bug
• Resolution: Duplicate
• Priority: Critical
• Fix Version/s: Lustre 2.5.3
• Affects Version/s: Lustre 2.5.2
• Environment: server: 2.5.2; client: lustre-b2_6-rc2 ldiskfs
• Severity: 3
• Assignee: WC Triage
• Reporter: Maloo

Description

This issue was created by maloo for sarah <[e-mail]>

This issue relates to the following test suite run: https://testing.hpdd.intel.com/test_sets/512f0146-0d81-11e4-b3f5-5254006e85c2.

The sub-test test_24 failed with the following error:

test failed to respond and timed out

Not sure but this may related with LU-5163
MDS console

04:50:42:Lustre: DEBUG MARKER: == replay-dual test 24: reconstruct on non-existing object == 04:50:00 (1405511400)
04:50:42:Lustre: DEBUG MARKER: /usr/sbin/lctl set_param fail_loc=0x119
04:50:42:Lustre: *** cfs_fail_loc=119, val=2147483648***
04:50:43:LustreError: 1776:0:(ldlm_lib.c:2415:target_send_reply_msg()) @@@ dropping reply  req@ffff880079979800 x1473783347046016/t545460846596(0) o36->3d5f7619-adde-ff24-3759-10f38dee94cd@10.2.4.188@tcp:0/0 lens 488/456 e 0 to 0 dl 1405511406 ref 1 fl Interpret:/0/0 rc 0/0
04:50:43:Lustre: DEBUG MARKER: lctl set_param fail_loc=0
04:50:43:Lustre: DEBUG MARKER: lctl set_param -n osd*.*MDT*.force_sync 1
04:50:43:LustreError: 1777:0:(lu_object.h:855:lu_object_attr()) ASSERTION( ((o)->lo_header->loh_attr & LOHA_EXISTS) != 0 ) failed: 
04:50:43:LustreError: 1777:0:(lu_object.h:855:lu_object_attr()) LBUG
04:50:43:Pid: 1777, comm: mdt00_001
04:50:43:
04:50:43:Call Trace:
04:50:43: [<ffffffffa0a17895>] libcfs_debug_dumpstack+0x55/0x80 [libcfs]
04:50:43: [<ffffffffa0a17e97>] lbug_with_loc+0x47/0xb0 [libcfs]
04:50:44: [<ffffffffa041680a>] mdt_attr_get_complex+0x6da/0x770 [mdt]
04:50:44: [<ffffffffa0a22308>] ? libcfs_log_return+0x28/0x40 [libcfs]
04:50:45: [<ffffffffa042fb95>] mdt_reconstruct_setattr+0xd5/0x3f0 [mdt]
04:50:46: [<ffffffffa0a27901>] ? libcfs_debug_msg+0x41/0x50 [libcfs]
04:50:46: [<ffffffffa042f565>] mdt_reconstruct+0x45/0x120 [mdt]
04:50:47: [<ffffffffa040aeab>] mdt_reint_internal+0x6bb/0x780 [mdt]
04:50:47: [<ffffffffa040afb4>] mdt_reint+0x44/0xe0 [mdt]
04:50:47: [<ffffffffa040e58a>] mdt_handle_common+0x52a/0x1470 [mdt]
04:50:47: [<ffffffffa044a755>] mds_regular_handle+0x15/0x20 [mdt]
04:50:47: [<ffffffffa13acbc5>] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc]
04:50:47: [<ffffffffa0a293cf>] ? lc_watchdog_touch+0x6f/0x170 [libcfs]
04:50:47: [<ffffffffa13a42a9>] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc]
04:50:49: [<ffffffffa0a27901>] ? libcfs_debug_msg+0x41/0x50 [libcfs]
04:50:49: [<ffffffffa13adf2d>] ptlrpc_main+0xaed/0x1740 [ptlrpc]
04:50:49: [<ffffffffa13ad440>] ? ptlrpc_main+0x0/0x1740 [ptlrpc]
04:50:49: [<ffffffff8109ab56>] kthread+0x96/0xa0
04:50:50: [<ffffffff8100c20a>] child_rip+0xa/0x20
04:50:50: [<ffffffff8109aac0>] ? kthread+0x0/0xa0
04:50:50: [<ffffffff8100c200>] ? child_rip+0x0/0x20
04:50:51:
04:50:51:Kernel panic - not syncing: LBUG
04:50:51:Pid: 1777, comm: mdt00_001 Not tainted 2.6.32-431.17.1.el6_lustre.x86_64 #1
04:50:52:Call Trace:
04:50:52: [<ffffffff8152795f>] ? panic+0xa7/0x16f
04:50:52: [<ffffffffa0a17eeb>] ? lbug_with_loc+0x9b/0xb0 [libcfs]
04:50:52: [<ffffffffa041680a>] ? mdt_attr_get_complex+0x6da/0x770 [mdt]
04:50:52: [<ffffffffa0a22308>] ? libcfs_log_return+0x28/0x40 [libcfs]
04:50:52: [<ffffffffa042fb95>] ? mdt_reconstruct_setattr+0xd5/0x3f0 [mdt]
04:50:53: [<ffffffffa0a27901>] ? libcfs_debug_msg+0x41/0x50 [libcfs]
04:50:53: [<ffffffffa042f565>] ? mdt_reconstruct+0x45/0x120 [mdt]
04:50:54: [<ffffffffa040aeab>] ? mdt_reint_internal+0x6bb/0x780 [mdt]
04:50:55: [<ffffffffa040afb4>] ? mdt_reint+0x44/0xe0 [mdt]
04:50:56: [<ffffffffa040e58a>] ? mdt_handle_common+0x52a/0x1470 [mdt]
04:50:56: [<ffffffffa044a755>] ? mds_regular_handle+0x15/0x20 [mdt]
04:50:58: [<ffffffffa13acbc5>] ? ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc]
04:50:59: [<ffffffffa0a293cf>] ? lc_watchdog_touch+0x6f/0x170 [libcfs]
04:51:00: [<ffffffffa13a42a9>] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc]
04:51:00: [<ffffffffa0a27901>] ? libcfs_debug_msg+0x41/0x50 [libcfs]
04:51:00: [<ffffffffa13adf2d>] ? ptlrpc_main+0xaed/0x1740 [ptlrpc]
04:51:02: [<ffffffffa13ad440>] ? ptlrpc_main+0x0/0x1740 [ptlrpc]
04:51:02: [<ffffffff8109ab56>] ? kthread+0x96/0xa0
04:51:02: [<ffffffff8100c20a>] ? child_rip+0xa/0x20
04:51:02: [<ffffffff8109aac0>] ? kthread+0x0/0xa0
04:51:02: [<ffffffff8100c200>] ? child_rip+0x0/0x20
04:51:04:Initializing cgroup subsys cpuset

Issue Links

• is related to: LU-5163 (lu_object.h:852:lu_object_attr()) ASSERTION( ((o)->lo_header->loh_attr & LOHA_EXISTS) != 0 ) failed
• is related to: LU-5285 mdt_reconstruct_setattr() calls mdt_attr_get_complex() without checking that object exists
• is related to: LU-5339 replay-dual defines test_24() twice