|
[15:32:02] John Hammond: Can I ask you something about LDLM_FL_LVB_READY and layout fetch?
[15:33:31] Jinshan Xiong: yes
[15:35:27] John Hammond: In ll_layout_refresh(), in the "requeue layout lock for file ..." case, I see that the layout is returned in the reply to LDLM_ENQUEUE but we still send a MDS_GETXATTR to fetch the layout.
[15:36:14] Jinshan Xiong: actually not. This is the tricky part.
[15:36:15] John Hammond: It seems because LDLM_FL_LVB_READY isn't set until ll_layout_conf() is called.
[15:37:07] Jinshan Xiong: ah LDLM_ENQUEUE, let me see.
[15:37:40] John Hammond: I'm looking at the debug log and I see two RPCs.
[15:38:26] Jinshan Xiong: right now, I believe the layout is returned by getxattr
[15:39:05] John Hammond: But also by the reply to LDLM_ENQUEUE:
[15:39:06] John Hammond: 00000002:00010000:3.0:1383074475.529239:0:4081:0:(mdc_locks.c:760:mdc_finish_enqueue()) ### layout lock returned by: layout, lvb_len: 56
[15:41:12] Jinshan Xiong: yes, it can be returned by LDLM_ENQUEUE, but not by completion_ast().
[15:42:10] Jinshan Xiong: did you see that the layout is returned but LVB_READY is not set
[15:42:12] Jinshan Xiong: ?
[15:42:23] John Hammond: Right.
[15:43:03] Jinshan Xiong: is that a LAYOUT lock enqueue?
[15:43:58] John Hammond: Yes, from ll_layout_refresh().
[15:46:02] John Hammond: Outside of OSC, only ldlm_lock_allow_match() will set LDLM_FL_LVB_READY and only ll_layout_conf() calls ldlm_lock_allow_match().
[15:46:36] Jinshan Xiong: This is not good.
[15:46:51] Jinshan Xiong: we can only set LVB_READY after layout is applied.
[15:47:54] Jinshan Xiong: seems that we need a new flag to mark that layout has been transferred by DLM ENQUEUE
[15:48:25] Jinshan Xiong: instead of using ldlm_is_lvb_ready() in ll_layout_fetch()
[15:55:58] John Hammond: Maybe I'm missing something but why can't the "layout lock returned by ..." block of mdc_finish_enqueue() set LDLM_FL_LVB_READY?
[15:56:40] Jinshan Xiong: because having a valid layout lock on the client means the layout is correct.
[15:57:02] Jinshan Xiong: So we have to apply the layout before setting the lvb ready flag
[15:57:58] John Hammond: > because having a valid layout lock on the client means the layout is correct.
[15:58:20] John Hammond: Which layout?
[16:00:11] John Hammond: Do you mean that we need separate flags for "LVB ready" and "allow match"?
[16:00:18] Jinshan Xiong: cl_conf_set().
[16:00:42] Jinshan Xiong: LVB ready implies allow match.
[16:00:58] Jinshan Xiong: I mean a new flag to mark layout is returned from DLM ENQ
[16:01:13] Jinshan Xiong: actually in current code, just an extra RPC is needed.
[16:01:30] Jinshan Xiong: based on the fact that layout is rarely changed, maybe not a big deal
[16:02:10] John Hammond: Altought it's rarely changed the lock can be cancelled frequently.
[16:02:47] John Hammond: I'm not changing layouts here.
[16:04:18] Jinshan Xiong: I see. In that case, we can define a new flag for this, or just use l_lvb_data to mark that it's already had a valid layout
[16:05:07] Jinshan Xiong: ah no, we can't use l_lvb_data, because it may be an empty file w/o layout
|