[LU-5242] Test hang sanity test_132, test_133: umount ost

• Type: Bug
• Resolution: Fixed
• Priority: Critical
• Fix Version/s: Lustre 2.8.0
• Affects Version/s: Lustre 2.6.0, Lustre 2.7.0, Lustre 2.5.3
• Labels: prz
• Severity: 3
• Assignee: Isaac Huang (Inactive)
• Reporter: Maloo

Description

This issue was created by maloo for Nathaniel Clark <[e-mail]>

This issue relates to the following test suite run:

http://maloo.whamcloud.com/test_sets/e5783778-f887-11e3-b13a-52540035b04c.

The sub-test test_132 failed with the following error:

test failed to respond and timed out

Info required for matching: sanity 132

Issue Links

• duplicates: LU-5163 (lu_object.h:852:lu_object_attr()) ASSERTION( ((o)->lo_header->loh_attr & LOHA_EXISTS) != 0 ) failed
• duplicates: LU-6105 Update ZFS/SPL version to 0.6.3-1.2
• is duplicated by: LU-4716 replay-ost-single test_5: stuck in dbuf_read->zio_wait
• is related to: LU-6089 qsd_handler.c:1139:qsd_op_adjust()) ASSERTION( qqi ) failed
• is related to: LU-6155 osd_count_not_mapped() calls dbuf_hold_impl() without the lock
• is related to: LU-5277 sanity test_132: mdt_build_target_list(), unable to handle kernel NULL pointer dereference
• is related to: LU-5737 sanity test_132: client NULL req->rq_import in ptlrpc_request_committed()
• is related to: LU-6008 sanity test_102b: setstripe failed
• is related to: LU-6195 osd-zfs: osd_declare_object_destroy() calls dmu_tx_hold_zap() with wrong keys
• is related to: LU-4950 sanity-benchmark test fsx hung: txg_sync was stuck on OSS
• is related to: LU-7020 OST_DESTROY message times out on MDS repeatedly, indefinitely
• is related to: LU-2160 Implement ZFS dmu_tx_hold_append() declarations for llog
• is related to: LU-4968 Test failure sanity test_132: umount /mnt/ost2
• is related to: LU-3665 obdfilter-survey test_3a: unmount stuck in obd_exports_barrier()

Comments

[Peter Jones]

Landed for 2.8

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch http://review.whamcloud.com/13630/
Subject: LU-5242 osd-zfs: umount hang in sanity 133g
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 9b704e4088d867851cdb011f0a2560b1e622555c

[Gerrit Updater]

Nathaniel Clark ([e-mail]) uploaded a new patch: http://review.whamcloud.com/13805
Subject: LU-5242 osd-zfs: umount hang in sanity 133g
Project: fs/lustre-release
Branch: b2_5
Current Patch Set: 1
Commit: 817cf8a2e781d546508929a9f58b44561ae3361c

[Bob Glossman (Inactive)]

another seen on b2_5 with zfs:
https://testing.hpdd.intel.com/test_sessions/435c3152-b816-11e4-9ecb-5254006e85c2

[Andreas Dilger]

I would agree with Alex on this. By deferring unlink of small files it will probably double or triple the total IO that the MDT is doing because in addition to the actual dnode deletion it also needs to insert the dnode into the deathrow ZAP in one TXG and then delete it from the same ZAP in a different txg. If there are a large number of objects being deleted at once (easily possible on the MDT), then the deathrow ZAP may get quite large (and never shrink) and updates would become less efficient than if it is kept small.

[Isaac Huang (Inactive)]

Thanks all. I'll work on a patch first without the small object optimization to get this bug fixed; then will benchmark to figure out whether to optimize the small object path or not.

[Alex Zhuravlev]

I have no objection to do this as simple as possible. my point was that MDT is known to be CPU-bound and ZAP (even micro ZAP) isn't free.

[Brian Behlendorf]

I could see a case for zero length files as an possible optimization. But I suspect that even for the MDT it would be more efficient to handle the freeing asynchronously outside of any request processing. Even if the file is zero length you're still going to be freeing a spill block for the xattrs and updating a dbuf for the dnode object. Personally I'd keep it as simple and concise as possible until it's clear something more is required. But that's just my preference.

Keep in mind that none of this free space will be available for a couple TXGs anyway.