[LU-11933] sanity-hsm: test_406 sanity: test_230b osd_declare_dir_delete() ASSERTION( dt_object_exists(dt) ) Created: 06/Feb/19 Updated: 15/Jan/20 Resolved: 28/Sep/19 |
|
| Status: | Resolved |
| Project: | Lustre |
| Component/s: | None |
| Affects Version/s: | Lustre 2.13.0, Lustre 2.12.3 |
| Fix Version/s: | Lustre 2.13.0, Lustre 2.12.4 |
| Type: | Bug | Priority: | Major |
| Reporter: | Alexander Boyko | Assignee: | Lai Siyao |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Issue Links: |
|
||||||||
| Severity: | 3 | ||||||||
| Rank (Obsolete): | 9223372036854775807 | ||||||||
| Description |
|
This issue was created by maloo for Alexander Boyko <c17825@cray.com> This issue relates to the following test suite run: https://testing.whamcloud.com/test_sets/3f081d66-2971-11e9-a318-52540065bddc [ 6056.684483] Lustre: DEBUG MARKER: == sanity-hsm test 406: attempting to migrate HSM archived files is safe ============================= 14:13:40 (1549376020) [ 6060.313122] LustreError: 4422:0:(osd_index.c:1117:osd_declare_dir_delete()) ASSERTION( dt_object_exists(dt) ) failed: [ 6060.315139] LustreError: 4422:0:(osd_index.c:1117:osd_declare_dir_delete()) LBUG [ 6060.316456] Pid: 4422, comm: mdt00_000 3.10.0-957.1.3.el7_lustre.x86_64 #1 SMP Fri Jan 11 05:58:47 UTC 2019 [ 6060.318137] Call Trace: [ 6060.318641] [<ffffffffc0a127cc>] libcfs_call_trace+0x8c/0xc0 [libcfs] [ 6060.319891] [<ffffffffc0a1287c>] lbug_with_loc+0x4c/0xa0 [libcfs] [ 6060.321802] [<ffffffffc1012acd>] osd_declare_dir_delete+0x27d/0x2b0 [osd_zfs] [ 6060.323583] [<ffffffffc129870e>] lod_sub_declare_delete+0xbe/0x1f0 [lod] [ 6060.324964] [<ffffffffc1273be4>] lod_declare_delete+0x24/0x30 [lod] [ 6060.326378] [<ffffffffc130964f>] mdd_orphan_declare_insert+0x33f/0x690 [mdd] [ 6060.327797] [<ffffffffc12f0fe8>] mdd_declare_create+0xa48/0xcb0 [mdd] [ 6060.329315] [<ffffffffc1310d8d>] mdd_declare_migrate_create+0x159/0x3ad [mdd] [ 6060.330679] [<ffffffffc12f61bb>] mdd_migrate+0xe9b/0x1810 [mdd] [ 6060.332092] [<ffffffffc1187786>] mdt_reint_migrate_internal+0xf56/0x14e0 [mdt] [ 6060.333573] [<ffffffffc1187f75>] mdt_reint_rename_or_migrate.isra.51+0x265/0x860 [mdt] [ 6060.335338] [<ffffffffc1188580>] mdt_reint_migrate+0x10/0x20 [mdt] [ 6060.336542] [<ffffffffc1188633>] mdt_reint_rec+0x83/0x210 [mdt] [ 6060.337928] [<ffffffffc1165183>] mdt_reint_internal+0x6e3/0xaf0 [mdt] [ 6060.339487] [<ffffffffc11704e7>] mdt_reint+0x67/0x140 [mdt] [ 6060.340551] [<ffffffffc0e1740a>] tgt_request_handle+0xafa/0x1590 [ptlrpc] [ 6060.342292] [<ffffffffc0dba99e>] ptlrpc_server_handle_request+0x24e/0xab0 [ptlrpc] [ 6060.343705] [<ffffffffc0dbe47c>] ptlrpc_main+0xbbc/0x2090 [ptlrpc] [ 6060.344944] [<ffffffffa52c1c31>] kthread+0xd1/0xe0 [ 6060.345964] [<ffffffffa5974c37>] ret_from_fork_nospec_end+0x0/0x39 [ 6060.347201] [<ffffffffffffffff>] 0xffffffffffffffff VVVVVVV DO NOT REMOVE LINES BELOW, Added by Maloo for auto-association VVVVVVV |
| Comments |
| Comment by Sebastien Buisson [ 21/Mar/19 ] |
|
Another occurence: |
| Comment by Amir Shehata (Inactive) [ 04/Apr/19 ] |
|
https://testing.whamcloud.com/test_sessions/d5ae56c7-e1fa-4073-9937-dec69a43fb8f |
| Comment by Vladimir Saveliev [ 08/Apr/19 ] |
|
https://testing.whamcloud.com/test_sets/e307dc3a-44ec-11e9-a256-52540065bddc |
| Comment by Bruno Faccini (Inactive) [ 10/Apr/19 ] |
|
Just got one more, https://testing.whamcloud.com/test_sets/09b1fc0a-5b9d-11e9-9720-52540065bddc . |
| Comment by Jian Yu [ 22/Apr/19 ] |
|
sanity test 230b hit the same assertion on master branch: |
| Comment by Oleg Drokin [ 09/Jul/19 ] |
|
this issue is still causing these kind of crashes. |
| Comment by Jian Yu [ 03/Aug/19 ] |
|
+1 on master branch: https://testing.whamcloud.com/test_sets/e02cf1b6-b603-11e9-b753-52540065bddc |
| Comment by Wang Shilong (Inactive) [ 08/Aug/19 ] |
|
https://testing.whamcloud.com/test_sets/81a7f204-b9a9-11e9-a2b6-52540065bddc |
| Comment by Patrick Farrell (Inactive) [ 08/Aug/19 ] |
|
Do you think this crash is related to https://review.whamcloud.com/#/c/35360/ / |
| Comment by Lai Siyao [ 09/Aug/19 ] |
|
No, it's related with volatile file migration, I'll check details later. |
| Comment by Andreas Dilger [ 26/Aug/19 ] |
|
+1 on master https://testing.whamcloud.com/test_sets/149b75f8-c5a4-11e9-a2b6-52540065bddc |
| Comment by Minh Diep [ 28/Aug/19 ] |
|
+1 on b2_12 https://testing.whamcloud.com/test_sessions/26c1e599-2cb2-4191-9492-37ca65e906d9 |
| Comment by Andreas Dilger [ 29/Aug/19 ] |
|
+1 on master https://testing.whamcloud.com/test_sets/3c2314e6-ca9a-11e9-90ad-52540065bddc sanity test_230b |
| Comment by Andreas Dilger [ 29/Aug/19 ] |
|
It looks like this is only crashing during directory migration, mostly in sanity test_230b and much less often in sanity-hsm test_406: It's hit about once a day - not terrible, but probably deserves some attention. |
| Comment by Gerrit Updater [ 11/Sep/19 ] |
|
Lai Siyao (lai.siyao@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/36154 |
| Comment by Gerrit Updater [ 27/Sep/19 ] |
|
Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/36154/ |
| Comment by Peter Jones [ 28/Sep/19 ] |
|
Landed for 2.13 |
| Comment by Gerrit Updater [ 08/Oct/19 ] |
|
Li Xi (lixi@ddn.com) uploaded a new patch: https://review.whamcloud.com/36398 |
| Comment by Gerrit Updater [ 08/Oct/19 ] |
|
Andreas Dilger (adilger@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/36399 |
| Comment by Gerrit Updater [ 21/Nov/19 ] |
|
Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/36399/ |