Details
-
Technical task
-
Resolution: Fixed
-
Critical
-
Lustre 2.5.0
-
9919
Description
In the restore case of hsm_cdt_request_completed(), if the copytool returned success but the layout swap fails then we get an unreadable file with HS_RELEASED clear but LOV_PATTERN_F_RELEASED set.
Perhaps the new HSM attributes should be applied to the volatile object before layout swap, and hsm_swap_layouts() should call mo_swap_layouts() with SWAP_LAYOUTS_MDS_HSM set.
Attachments
Activity
Hello Andreas,
I am sorry if I missed to do something here, to be honest actually I mainly focus to get the patch done for the branch where problem has been reported. But then should I create a new patch version for each affected version listed?
Patch was only landed to master and not b2_5. In the future, this type of patch should be cherry-picked to b2_5 so that it is fixed in the maintenance release.
patch http://review.whamcloud.com/7631 has landed. Closing.
Hehe, finally I found that my fault-injection code itself introduced some problem because being added after the volatile/2nd file layout change and not reverting it to mimic the error !! This caused the restored datas to be available as if restore succeed …
I changed this in patch-set #13, and now new sub-test test_12o runs fine, returning errors on both copytool (ENOTSUPP, injected!) and client (ENODATA) sides with layout-swap fault-injection, and next restore attempt without fault-injection to be successful.
Will run with build+patch locally and see if I can still reproduce the Volatile object leak on MDT, seen as part of this ticket and LU-4293.
Some update, after I added fault-injection (force -ENOENT in the middle of mdd_swap_layouts() to cause layouts swap back) and associated sub-test test_12o within patch-set #8.
test_12o fails due to "diff" command, that caused the implicit restore, to be successful when it is expected to fail because of the fault-injection. Strange is that the Restore operation has been marked as failed, the Copytool received the error, and file still has the "released" flag set!!
I wonder if there could be some issue in mdd_swap_layouts() causing this unexpected behavior ?
I am wondering if I should also add some error injection to simulate SWAP_LAYOUT failure during restore ??
I will also push a new patch-set #8 to address John's last comment and convert to usual error handling style.
I found a possible bug in my original patch version causing layout-lock not to be released when restore is canceled … Just submitted patch-set #8 to fix this, will see if is passes auto-tests (particularly sanity-hsm/test_33 which was timing-out due to md5sum process never ending!!…).
Latest patch-set #6 auto-tests failures appear not related to this patch but to multiple issues already addressed in others tickets (LU-4093, LU-4086, …). Some of them have heir patches that landed until now, so I will rebase it again …
1st patch attempt is at http://review.whamcloud.com/7631.
Unfortunately it failed in multiple auto-tests and needs at least to re-base.
But also failed in several sub-tests of sanity-hsm, which is highly suspect ...
Currently under investigations.
> _ always call mo_swap_layouts() with SWAP_LAYOUTS_MDS_HSM flag from hsm_swap_layouts(), since (actually?) hsm_swap_layouts() is only called from mdt_hsm_update_request_state() for a RESTORE op.
This seems better to me.
Bruno, the patch was marked as affecting the 2.5.0 release. I'm just going through patches that have landed to master and trying to see which ones need to be landed for 2.5.1 that have not been landed there, since that is the long-term maintenance release. If you are closing a but then you should consider if it is fixing a problem that is serious and may affect earlier versions of Lustre and should land on the maintenance release. In many cases, Oleg can cherry-pick the patch directly to b2_5 without putting it through Gerrit/Jenkins/autotest again, but he needs to know to do this.