[LU-9231] cleanup test env when mdd-survey exit abnormally

• Type: Bug
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.10.0
• Severity: 3
• Assignee: WC Triage
• Reporter: Minh Diep

Description

If mdd-survey failed and did not clean up properly, we might hit

12:03:27:[53931.651543] LustreError: 6647:0:(hash.c:1138:cfs_hash_destroy()) ASSERTION( !cfs_hash_with_assert_empty(hs) ) failed: hash lu_site_osd-ldi bucket 3(318) is not  empty: 1 items left
12:03:27:[53931.656508] LustreError: 6647:0:(hash.c:1138:cfs_hash_destroy()) LBUG
12:03:27:[53931.658856] Pid: 6647, comm: obd_zombid
12:03:27:[53931.661008] 
12:03:27:[53931.661008] Call Trace:
12:03:27:[53931.665075]  [<ffffffffa06fe853>] libcfs_debug_dumpstack+0x53/0x80 [libcfs]
12:03:27:[53931.667470]  [<ffffffffa06fedf5>] lbug_with_loc+0x45/0xc0 [libcfs]
12:03:27:[53931.669791]  [<ffffffffa071612c>] cfs_hash_putref+0x38c/0x4e0 [libcfs]
12:03:27:[53931.672205]  [<ffffffffa0850d18>] lu_site_fini+0x48/0x90 [obdclass]
12:03:27:[53931.674491]  [<ffffffffa0cb9687>] osd_device_free+0xa7/0x4a0 [osd_ldiskfs]
12:03:27:[53931.676781]  [<ffffffffa0836b66>] class_decref+0x3f6/0x4e0 [obdclass]
12:03:27:[53931.679009]  [<ffffffffa0821fae>] obd_zombie_impexp_cull+0x5fe/0xa80 [obdclass]
12:03:27:[53931.681302]  [<ffffffffa082249d>] obd_zombie_impexp_thread+0x6d/0x1b0 [obdclass]
12:03:27:[53931.683610]  [<ffffffff810c4ec0>] ? default_wake_function+0x0/0x20
12:03:38:[53931.685808]  [<ffffffffa0822430>] ? obd_zombie_impexp_thread+0x0/0x1b0 [obdclass]
12:03:38:[53931.688082]  [<ffffffff810b052f>] kthread+0xcf/0xe0
12:03:38:[53931.690100]  [<ffffffff810b0460>] ? kthread+0x0/0xe0
12:03:38:[53931.692131]  [<ffffffff81696658>] ret_from_fork+0x58/0x90
12:03:38:[53931.694157]  [<ffffffff810b0460>] ? kthread+0x0/0xe0
12:03:38:[53931.696192] 
12:03:38:[53931.698000] Kernel panic - not syncing: LBUG