To prevent replay faking.

Created an attachment (id=22098) [details]
replay signature HLD

Created an attachment (id=22181) [details]
replay signature HLD (v2)

(From update of attachment 22181 [details])
Inspection template(s):
Bug:       18657 
Developer: yong.fan@sun.com 
Size:       Lines of Text 
Date:      2009-6-29
Defects:   1
Type:      CODE
Inspector: Nicolas.Williams@sun.com

--------------
 - "Recovery spoofing" might be a better term than "recovery faking".

 - Replay signature keys can be changed very infrequently.  12h, as proposed, is fine.  I don't
have a good source to cite as to usable HMAC key lifetimes; RFC2104 specifically states that there
is insufficient analysis on this matter.

 - However, it'd be a good idea to add a nonce (say, 64-bit) to the replay capability if we can.

 - We should remove the need to renew replay capabilities.  This will significantly reduce
complexity.  To do this two keys must be valid at all times: the current key and the previous key.

 - It's important to note that lock recovery is not covered here.  At DLM recovery time a client
can DoS the cluster by causing a lock conflict, which will in turn cause some or all clients to be
evicted.  This isn't easy to protect against, but at least it's only a DoS; protection against this
should be out of scope in this feature.

(From update of attachment 22181 [details])
Also, we need to do something about opens.  Renewing replay signatures may be one way to do that,
but is there any other way?  Can the client open by fid and use the capability it has to prove that
it should be allowed to replay the open?

Created an attachment (id=24427) [details]
Proposed HLD updates

I've edited the HLD.  I believe we can get by without replay signature renewal.  To do this we need
to recover open state by reconstructing opens as anonymous open by FID, with security provided by
FID capabilities, rather than the current replay of open RPCs scheme.  The fundamental principal
for replay signatures would then be: replay signatures are to be used for transaction RPCs with
normally bounded duration (basically: all filesystem transactions, such as the create component of
an open, but not the stateful parts of open), reconstruction should be used for all state (locks,
open files).

(In reply to comment #5)
> I've edited the HLD.  I believe we can get by without replay signature
> renewal.  To do this we need to recover open state by reconstructing opens
> as anonymous open by FID, with security provided by FID capabilities, rather
> than the current replay of open RPCs scheme.

Note that there is already a desire to do this for another reason - to allow the
Lustre protocol to change beween major versions of Lustre in an incompatible
way.  If the clients don't keep any saved RPCs over an upgrade (by being told
in advance that there is going to be a server upgrade) then the only client
state that needs to be saved on the client are open file handles.  The goal is
that open files are re-opened after server restart based on information in the
file handle itself instead of via saved RPCs.

This project is called "simplified interoperability", and while we have a good
idea of what is involved, there is unfortunately nobody working on it.

(In reply to comment #6)
> This project is called "simplified interoperability", and while we have a good
> idea of what is involved, there is unfortunately nobody working on it.

Oh, wait.  I now recall Eric Mei is going to start working on this, so now would
be an excellent time to discuss with him any other requirements we have for the
open file handling from a recovery POV.

Yes, Oleg just explained that to me on IRC as well.  As a result I just posted on lustre-devel with
a specific proposal for separating open recovery from transaction recovery.

Created an attachment (id=24596) [details]
Latest replay signature HLD

Created an attachment (id=24597) [details]
PDF version

Updating priority to P3 since it is not 2.0 task any longer

HLD for new Lustre Capability finished, no DLD yet.