[LU-10863] sanity test_160f: FAIL: User cl7 still found in changelog_users; mds stucks

Thu, 21 Nov 2019 21:51:44 +0000

== sanity test 160f: changelog garbage collect (timestamped users) =================================== 02:42:04 (1522291324)

mdd.lustre-MDT0000.changelog_mask=+hsm

mdd.lustre-MDT0001.changelog_mask=+hsm

Registered 2 changelog users: 'cl6 cl6'

mdd.lustre-MDT0000.changelog_mask=+hsm

mdd.lustre-MDT0001.changelog_mask=+hsm

Registered 2 changelog users: 'cl6 cl7 cl6 cl7'

striped dir -i0 -c2 /mnt/lustre/d160f.sanity

total: 4 create in 0.01 seconds: 720.08 ops/second

mdd.lustre-MDT0000.changelog_max_idle_time=10

mdd.lustre-MDT0001.changelog_max_idle_time=10

mdd.lustre-MDT0000.changelog_min_gc_interval=2

mdd.lustre-MDT0001.changelog_min_gc_interval=2

mdd.lustre-MDT0000.changelog_min_free_cat_entries=3

mdd.lustre-MDT0001.changelog_min_free_cat_entries=3

fail_loc=0x1313

fail_val=3

lustre-MDT0000: clear the changelog for cl6 to record #22

verifying user clear: 20 + 2 == 22

pdsh@fre1239: fre1237: ssh exited with exit code 1

pdsh@fre1239: fre1237: ssh exited with exit code 1

 sanity test_160f: @@@@@@ FAIL: User cl7 still found in changelog_users

mds console:

[ 9826.567243] Lustre: DEBUG MARKER: == sanity test 160f: changelog garbage collect (timestamped users) =================================== 02:42:04 (1522291324)
[ 9826.929631] Lustre: lustre-MDD0000: changelog on
[ 9826.932217] Lustre: Skipped 1 previous similar message
[ 9827.571088] Lustre: 22694:0:(osd_handler.c:1810:osd_trans_start()) lustre-MDT0000: credits 838 > trans_max 704
[ 9827.576272] Lustre: 22694:0:(osd_handler.c:1741:osd_trans_dump_creds())   create: 4/32/0, destroy: 0/0/0
[ 9827.579979] Lustre: 22694:0:(osd_handler.c:1748:osd_trans_dump_creds())   attr_set: 1/1/0, xattr_set: 7/403/0
[ 9827.583774] Lustre: 22694:0:(osd_handler.c:1758:osd_trans_dump_creds())   write: 10/179/0, punch: 0/0/0, quota 2/2/0
[ 9827.587870] Lustre: 22694:0:(osd_handler.c:1765:osd_trans_dump_creds())   insert: 12/216/0, delete: 0/0/0
[ 9827.591557] Lustre: 22694:0:(osd_handler.c:1772:osd_trans_dump_creds())   ref_add: 5/5/0, ref_del: 0/0/0
[ 9827.595250] Pid: 22694, comm: mdt00_000
[ 9827.597383] 
[ 9827.597383] Call Trace:
[ 9827.597383] Call Trace:
[ 9827.600733]  [<ffffffffa03f07ee>] libcfs_call_trace+0x4e/0x60 [libcfs]
[ 9827.603622]  [<ffffffffa03f0826>] libcfs_debug_dumpstack+0x26/0x30 [libcfs]
[ 9827.606588]  [<ffffffffa0a5c17c>] osd_trans_start+0x43c/0x460 [osd_ldiskfs]
[ 9827.609589]  [<ffffffffa08a1fea>] top_trans_start+0x33a/0x950 [ptlrpc]
[ 9827.612612]  [<ffffffffa0d003e1>] lod_trans_start+0x31/0x40 [lod]
[ 9827.615426]  [<ffffffffa0daf5b4>] mdd_trans_start+0x14/0x20 [mdd]
[ 9827.618115]  [<ffffffffa0d95427>] mdd_create+0xb77/0x13a0 [mdd]
[ 9827.620666]  [<ffffffffa0c36d86>] mdt_create+0x846/0xbb0 [mdt]
[ 9827.623306]  [<ffffffffa05d1f29>] ? lprocfs_counter_add+0xf9/0x160 [obdclass]
[ 9827.626289]  [<ffffffffa06076f1>] ? lprocfs_job_stats_log+0xd1/0x630 [obdclass]
[ 9827.629262]  [<ffffffffa0c3725b>] mdt_reint_create+0x16b/0x350 [mdt]
[ 9827.631944]  [<ffffffffa0c387b0>] mdt_reint_rec+0x80/0x210 [mdt]
[ 9827.634321]  [<ffffffffa0c1814b>] mdt_reint_internal+0x5fb/0x9c0 [mdt]
[ 9827.636684]  [<ffffffffa0c230c7>] mdt_reint+0x67/0x140 [mdt]
[ 9827.638827]  [<ffffffffa08900d5>] tgt_request_handle+0x925/0x13b0 [ptlrpc]
[ 9827.641196]  [<ffffffffa0835c9e>] ptlrpc_server_handle_request+0x24e/0xab0 [ptlrpc]
[ 9827.643664]  [<ffffffff810af0b8>] ? __wake_up_common+0x58/0x90
[ 9827.645780]  [<ffffffffa0839cf0>] ptlrpc_main+0xaa0/0x1de0 [ptlrpc]
[ 9827.647977]  [<ffffffffa0839250>] ? ptlrpc_main+0x0/0x1de0 [ptlrpc]
[ 9827.650246]  [<ffffffff810a5b8f>] kthread+0xcf/0xe0
[ 9827.652096]  [<ffffffff810a5ac0>] ? kthread+0x0/0xe0
[ 9827.654038]  [<ffffffff81644fd8>] ret_from_fork+0x58/0x90
[ 9827.656005]  [<ffffffff810a5ac0>] ? kthread+0x0/0xe0
[ 9827.657919]
[ 9840.693340] Lustre: DEBUG MARKER: sanity test_160f: @@@@@@ FAIL: User cl7 still found in changelog_users

Whamcloud Community JIRA

[LU-10863] sanity test_160f: FAIL: User cl7 still found in changelog_users; mds stucks