if fileA was opened (i.e. step 2 were executed except successful reply), then opencount is non-zero and unlink can't destroy the object.
at the moment I don't understand how opencount can get to 0 unless client2 is evicted.
forgot to mention, that open normally stores FID in last_rcvd so that reconstruct can find the object w/o name. IIRC 
Alex, I think the opened file is still open-unlinked but its FID is FID1 from create at step 1, while resend is made with FID2 from client2. At step 2 it found file by name and open it (its FID is FID1) but during resend name doesn't exist and new file is created with the same name and FID2. It looks like possible case but I don't see why this is critical error after all, that is valid race, how is it different from the case when unlink from client 1 comes earlier than open from client2?
Alex, reconstruct open isn't take any fid's from last_rcvd and don't store anything except a disposition.
struct lsd_reply_data {
__u64 lrd_transno; /* transaction number */
__u64 lrd_xid; /* transmission id */
__u64 lrd_data; /* per-operation data */
__u32 lrd_result; /* request result */
__u32 lrd_client_gen; /* client generation */
};
struct tg_reply_data {
/** chain of reply data anchored in tg_export_data */
struct list_head trd_list;
/** copy of on-disk reply data */
struct lsd_reply_data trd_reply;
/** versions for Version Based Recovery */
__u64 trd_pre_versions[4];
/** slot index in reply_data file */
int trd_index;
/** tag the client used */
__u16 trd_tag;
};
it will be best solution in case if reply cache will store an open handle and file xid, but nothing now.
Mikhail it's problem from recovery view and execute-once semantic. As different result will be in case fail hit and don't hit between unlink and resend.
object will created in case fail don't hit, but if crash it exist we don't have an object created. because create request can removed from replay queue due commit update and don't join into recovery. So it's not a valid race I think.
MDT have create an object when none expect to have this reply.
sure, I don't recall all the details, but we do have XID in mfd:
struct mdt_file_data {
/** portals handle must be first */
struct portals_handle mfd_open_handle;
/** open mode provided by client */
u64 mfd_open_flags;
/** protected by med_open_lock */
struct list_head mfd_list;
/** xid of the open request */
__u64 mfd_xid;
which should be enough to find the object and reconstruct the reply?
Alex, current code have mfd search too later after req->r_fid2 checks. It's problem. As about solution, same way was chosen but several problems exist.
1) mfd was don't exist with resend after recovery. if no open lock provided. client start create before fail, mdt failed before reply send. Client have think this is resend but oops. But say way will be in case client able to close this file in situation.
client doing open, link fail, client have resend a open and send close immediately. But server able to delivery a reply (server don't know about link flap, it's just client side view). Reply arrived to the same open request as XID isn't changed.
Based on LNet ideas - we should don't have assumptions about request order processing - so processing close before open resend is possible. mfd will destroyed in this case.. OOPS.
so we need to separate mfd close vs mfd don't created.
2) mfd list is large in most cases, scanning this list can take a lot time a specially if it twice (second one in mdt_finish_open).
3) I worry about remote create in open(O_CREATE) case - what we should do in this case? did it possible to return different fid as provided as fid2 in client open?
as about 1/2 it looks solution exist. tg_reply_data is completely 'in memory' object so easy to extent to the two additional fid.
first one - mfd_exist, should resolve a problem with (1) and separate resend after replay (should be handled as replay) and clear resend.
second one - open handle - support to fast search mfd via mdt_handle2mfd.
(several hack's / optimisations possible - as open is schedule difficult reply always so we able to separate reply is delivered or not in mdt_steal_locks which scan for outstanding difficult reply's)
cross open(O_CREATE) don't have mfd also, it have create an object on remote mdt but no MFD was created in same time (mdt_finish_open skip).
So good to save a real child fid in this case to avoid lookup by name and side effects of this change.
Alexey Lyashkov (c17817@cray.com) uploaded a new patch:
https://review.whamcloud.com/34266
Subject: LU-11952 mdt: fix reconstruct_open
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 02ff43d8954460d43228fd866c274fdda81bfdc6
Andriy Skulysh (c17819@cray.com) uploaded a new patch: https://review.whamcloud.com/35112
Subject: LU-11952 mdt: fix reconstruct open
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: c0d28df84c5b946e1f8f35134a78b6e567ef5e15
I understand that https://review.whamcloud.com/35112 is moving slowly, but it should be considered for L2.13 if at all possible.
I guess we can shoot for 2.14.0 now.
2.15.0?
"Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/35112/
Subject: LU-11952 mdt: fix reconstruct open
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: cf6ce3329f92f146206be8b63846d2c6e45c92d6
Landed for 2.15