[LU-15139] sanity test_160h: dt_record_write() ASSERTION( dt->do_body_ops->dbo_write ) failed

• Type: Bug
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.16.0
• Affects Version/s: Lustre 2.14.0, Lustre 2.15.0
• Severity: 3
• Assignee: Alex Zhuravlev
• Reporter: Maloo

Description

This issue was created by maloo for Andreas Dilger <[e-mail]>

This issue relates to the following test suite run: https://testing.whamcloud.com/test_sets/a9a49ddc-ce2e-4d36-a040-a2522bc7bfaa

test_160h failed with the following error:

[ 7154.952072] LustreError: 27898:0:(dt_object.c:503:dt_record_write()) ASSERTION( dt->do_body_ops->dbo_write ) failed: 
[ 7154.954429] LustreError: 27898:0:(dt_object.c:503:dt_record_write()) LBUG
[ 7154.955686] Pid: 27898, comm: mdt_out00_000 3.10.0-1160.15.2.el7_lustre.ddn12.x86_64 #1 SMP Tue Feb 23 18:25:44 UTC 2021
[ 7154.957907] Call Trace:
[ 7154.958488]  [<ffffffffc08197cc>] libcfs_call_trace+0x8c/0xc0 [libcfs]
[ 7154.960155]  [<ffffffffc081987c>] lbug_with_loc+0x4c/0xa0 [libcfs]
[ 7154.961662]  [<ffffffffc0a50272>] dt_record_write+0xb2/0x120 [obdclass]
[ 7154.963451]  [<ffffffffc0da7b69>] out_tx_write_exec+0x169/0x2f0 [ptlrpc]
[ 7154.965451]  [<ffffffffc0d9ff51>] out_tx_end+0xe1/0x5c0 [ptlrpc]
[ 7154.967053]  [<ffffffffc0da4172>] out_handle+0x1452/0x1bc0 [ptlrpc]
[ 7154.968687]  [<ffffffffc0d98a3e>] tgt_request_handle+0xaee/0x15f0 [ptlrpc]
[ 7154.970322]  [<ffffffffc0d3f40b>] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc]
[ 7154.972268]  [<ffffffffc0d42d74>] ptlrpc_main+0xb34/0x1470 [ptlrpc]

Another test had a slightly different crash that is also new that may be related:
https://testing.whamcloud.com/test_sets/d3ede9b0-ecaa-484e-8840-891e62bdb314

[86376.168111] LustreError: 31898:0:(mdt_handler.c:5316:mdt_fini()) ASSERTION( atomic_read(&d->ld_ref) == 0 ) failed: 
[86376.169205] LustreError: 31898:0:(mdt_handler.c:5316:mdt_fini()) LBUG
[86376.169811] Pid: 31898, comm: umount 3.10.0-1160.15.2.el7_lustre.ddn12.x86_64 #1 SMP Tue Feb 23 03:03:21 UTC 2021
[86376.170763] Call Trace:
[86376.171066]  [<ffffffffc0bed7cc>] libcfs_call_trace+0x8c/0xc0 [libcfs]
[86376.171857]  [<ffffffffc0bed87c>] lbug_with_loc+0x4c/0xa0 [libcfs]
[86376.172537]  [<ffffffffc148b5ea>] mdt_device_fini+0x8fa/0x930 [mdt]
[86376.173227]  [<ffffffffc0e0ea68>] class_cleanup+0x9a8/0xc40 [obdclass]
[86376.174052]  [<ffffffffc0e0f98c>] class_process_config+0x65c/0x2830 [obdclass]
[86376.174793]  [<ffffffffc0e11d26>] class_manual_cleanup+0x1c6/0x710 [obdclass]
[86376.175574]  [<ffffffffc0e446a5>] server_put_super+0xa35/0x1150 [obdclass]
[86376.176295]  [<ffffffff8c4506fd>] generic_shutdown_super+0x6d/0x100
[86376.177103]  [<ffffffff8c450b02>] kill_anon_super+0x12/0x20
[86376.177684]  [<ffffffffc0e14932>] lustre_kill_super+0x32/0x50 [obdclass]
[86376.178360]  [<ffffffff8c450ede>] deactivate_locked_super+0x4e/0x70
[86376.179043]  [<ffffffff8c451666>] deactivate_super+0x46/0x60
[86376.179630]  [<ffffffff8c470cff>] cleanup_mnt+0x3f/0x80
[86376.180163]  [<ffffffff8c470d92>] __cleanup_mnt+0x12/0x20
[86376.180745]  [<ffffffff8c2c294b>] task_work_run+0xbb/0xe0
[86376.181296]  [<ffffffff8c22cc65>] do_notify_resume+0xa5/0xc0

VVVVVVV DO NOT REMOVE LINES BELOW, Added by Maloo for auto-association VVVVVVV
sanity test_160h - onyx-44vm4 crashed during sanity test_160h

Issue Links

• is duplicated by: LU-15849 replay-dual: test_26 hit dt_declare_record_write()) ASSERTION( dt->do_body_ops )
• is related to: LU-15809 replay-dual test_29: timeout llog_verify_record() lustre-MDT0000-osp-MDT0001: record is too large: 0 > 32768
• is related to: LU-17255 dt_record_write()) ASSERTION( dt->do_body_ops->dbo_write )
• is related to: LU-15769 llog_osd_next_block()): invalid llog tail at log id ...
• is related to: LU-14932 runtests: test_1 llog_cat_cleanup()) ASSERTION( index ) on MDS
• is related to: LU-15695 dt_ladvise(): ASSERTION( dt->do_body_ops->dbo_ladvise )
• is related to: LU-15938 MDT recovery did not finish due to corrupt llog record
• is related to: LU-13195 replay-single test_118: dt_declare_record_write() ASSERTION( dt->do_body_ops ) failed
• is related to: LU-16398 ost-pools: FAIL: remove sub-test dirs failed