[LU-7440] llog_osd_write_rec()) LBUG

Tue, 17 Nov 2015 22:28:28 +0000

Error occurred during soak testing of build '20151116' (see https://wiki.hpdd.intel.com/pages/viewpage.action?title=Soak+Testing+on+Lola&spaceKey=Releases#SoakTestingonLola-2015116). DNE is enabled, MDTs had been formatted with ldiskfs, OST with zfs. MDSes are configured in active - active HA failover configuration.

MDS crashed with:

Nov 16 15:13:33 lola-11 kernel: LustreError: 5489:0:(llog_osd.c:601:llog_osd_write_rec()) soaked-MDT0004-osp-MDT0007: [0x30006f946:0x1:0x0]index 1 already set in log bitmap
Nov 16 15:13:33 lola-11 kernel: LustreError: 5489:0:(llog_osd.c:603:llog_osd_write_rec()) LBUG
Nov 16 15:13:33 lola-11 kernel: Pid: 5489, comm: mdt00_008
Nov 16 15:13:33 lola-11 kernel: 
Nov 16 15:13:33 lola-11 kernel: Call Trace:
Nov 16 15:13:33 lola-11 kernel: [<ffffffffa0876875>] libcfs_debug_dumpstack+0x55/0x80 [libcfs]
Nov 16 15:13:33 lola-11 kernel: [<ffffffffa0876e77>] lbug_with_loc+0x47/0xb0 [libcfs]
Nov 16 15:13:33 lola-11 kernel: [<ffffffffa09839f3>] llog_osd_write_rec+0x1bb3/0x1c60 [obdclass]
Nov 16 15:13:33 lola-11 kernel: [<ffffffffa08826c1>] ? libcfs_debug_msg+0x41/0x50 [libcfs]
Nov 16 15:13:33 lola-11 kernel: [<ffffffffa0970416>] llog_write_rec+0xb6/0x270 [obdclass]
Nov 16 15:13:33 lola-11 kernel: [<ffffffffa0978962>] llog_cat_new_log+0x452/0xed0 [obdclass]
Nov 16 15:13:33 lola-11 kernel: [<ffffffffa09797f1>] llog_cat_declare_add_rec+0x411/0x430 [obdclass]
Nov 16 15:13:33 lola-11 kernel: [<ffffffffa097006f>] llog_declare_add+0x7f/0x1b0 [obdclass]
Nov 16 15:13:33 lola-11 kernel: [<ffffffffa0c6b2ac>] top_trans_start+0x17c/0x920 [ptlrpc]
Nov 16 15:13:33 lola-11 kernel: [<ffffffffa133ce11>] lod_trans_start+0x61/0x70 [lod]
Nov 16 15:13:33 lola-11 kernel: [<ffffffffa13e7ff4>] mdd_trans_start+0x14/0x20 [mdd]
Nov 16 15:13:33 lola-11 kernel: [<ffffffffa13d6a8a>] mdd_create+0x9aa/0x1600 [mdd]
Nov 16 15:13:33 lola-11 kernel: [<ffffffffa1285824>] ? mdt_version_save+0x84/0x1a0 [mdt]
Nov 16 15:13:33 lola-11 kernel: [<ffffffffa1287fe6>] mdt_reint_create+0xbb6/0xcc0 [mdt]
Nov 16 15:13:33 lola-11 kernel: [<ffffffffa09df690>] ? lu_ucred+0x20/0x30 [obdclass]
Nov 16 15:13:33 lola-11 kernel: [<ffffffffa1267675>] ? mdt_ucred+0x15/0x20 [mdt]
Nov 16 15:13:33 lola-11 kernel: [<ffffffffa12808dc>] ? mdt_root_squash+0x2c/0x3f0 [mdt]
Nov 16 15:13:33 lola-11 kernel: [<ffffffff81064c00>] ? default_wake_function+0x0/0x20
Nov 16 15:13:33 lola-11 kernel: [<ffffffffa1284a1d>] mdt_reint_rec+0x5d/0x200 [mdt]
Nov 16 15:13:33 lola-11 kernel: [<ffffffffa127077b>] mdt_reint_internal+0x62b/0xb80 [mdt]
Nov 16 15:13:33 lola-11 kernel: [<ffffffffa127116b>] mdt_reint+0x6b/0x120 [mdt]
Nov 16 15:13:33 lola-11 kernel: [<ffffffffa0c54dfc>] tgt_request_handle+0x8bc/0x12e0 [ptlrpc]
Nov 16 15:13:33 lola-11 kernel: [<ffffffffa0bfc6f1>] ptlrpc_main+0xe41/0x1910 [ptlrpc]
Nov 16 15:13:33 lola-11 kernel: [<ffffffff8152a39e>] ? thread_return+0x4e/0x7d0
Nov 16 15:13:33 lola-11 kernel: [<ffffffffa0bfb8b0>] ? ptlrpc_main+0x0/0x1910 [ptlrpc]
Nov 16 15:13:33 lola-11 kernel: [<ffffffff8109e78e>] kthread+0x9e/0xc0
Nov 16 15:13:33 lola-11 kernel: [<ffffffff8100c28a>] child_rip+0xa/0x20
Nov 16 15:13:33 lola-11 kernel: [<ffffffff8109e6f0>] ? kthread+0x0/0xc0
Nov 16 15:13:33 lola-11 kernel: [<ffffffff8100c280>] ? child_rip+0x0/0x20
Nov 16 15:13:33 lola-11 kernel:

Attached console and syslog file of affected MDS.

Whamcloud Community JIRA

[LU-7440] llog_osd_write_rec()) LBUG