[LU-2869] extended attribute cache for Lustre

• Status: Resolved
• Project: Lustre
• Fix Version/s: Lustre 2.5.0
• Type: Improvement
• Priority: Minor
• Reporter: Andrew Perepechko
• Assignee: James Nunez (Inactive)
• Resolution: Fixed
• Labels: patch, xattr

Description

The patch implements an extended attribute cache for
a Lustre client. It is organized as a write-through
cache: reads are performed from cache, updates are sent
synchronously to the MDS. An additional inode bit
is added to protect the cache.

The patch itself will be uploaded shortly.

Issue Links

• is related to: LU-3795 replay-single test_58b test_58c: trus...
• is related to: LU-3670 some trusted xattrs are not coherent ...
• is related to: LU-3669 (mdt_xattr.c:178:mdt_getxattr()) ASSE...
• is related to: LU-549 Add xattr list/value cache on client
• is related to: LU-3583 LDLM support for atomic match_or_enqu...
• is related to: LU-10276 Revise what MDS_INODELOCK_XATTR protects
• is related to: LU-17238 negative xattr cache on client

Comments

[Andrew Perepechko]

http://review.whamcloud.com/#change,5537

[Andrew Perepechko]

xattr cache performance for the simplest Lustre configuration (2 nodes), read case is faster for the cached version as compared with the uncached by 4.5 times

[Nathan Rutman]

Xyratex MRP-57

[jacques-charles lafoucriere]

Nathan the xyratex url is broken (server not found error), can you put a valid/public one ?

[Nathan Rutman]

Sorry JC, the Xyratex tickets are really only useful for Xyratex engineers. We have posted the patch in the first comment above.

[Andrew Perepechko]

Will this patch be inspected? It's almost three months from the patch upload. I had to rebase to a recent version.

[James Nunez (Inactive)]

The master branch should open for new features soon.

[Andrew Perepechko]

Thanks, James!

[James Nunez (Inactive)]

Landed for 2.5

[Alex Zhuravlev]

can you explain why mot_xattr_sem is needed, please?

[Andrew Perepechko]

The semaphore was added to protect the case when the client does not take the XATTR lock and requests FLXATTRALL.
FLXATTRALL is implemented as getting the list and then getting the values, so setting/removing an xattr in the middle
can lead to failure. Currently, there is no FLXATTRLS user that does not take the XATTR lock, so there is no use
case for the semaphore being contended.

[Alex Zhuravlev]

well, if this failure is -ENOENT, then it should be easy to check and skip? on the otherhand we carry this structure for every object in MDT cache. a bit expensive for such a case?

[Andrew Perepechko]

Besides -ENOENT, due to races an FLXATTRLS user can get an inconsistent list of attribute names and values, which the object never had from another client viewpoint. But since there is no use case for that, the locking can be simply dropped.

[Alex Zhuravlev]

not sure what kind of inconsistency FLXATTRLS can observe? any plans to make a patch to drop it?

[Andrew Perepechko]

not sure what kind of inconsistency FLXATTRLS can observe?

Like, setxattr "a=x", setxattr "b=y". An unlocked FLXATTRLS can pack "b=y" update without packing "a=x" because of update races.

any plans to make a patch to drop it?

I'll update http://review.whamcloud.com/#/c/7208/ with this fix in a while.

[Alex Zhuravlev]

great. thanks!