[LU-12499] improve the file path layout of HSM posix copytool

• Status: Open
• Project: Lustre
• Type: Improvement
• Priority: Minor
• Reporter: Li Xi
• Assignee: WC Triage
• Resolution: Unresolved

Description

Andreas's comment on the current layout:

The problem with having the low bits in the top of the path is that this will create a very deep directory tree, and there will only be a single entry in each of the leaf directories because the f_seq values never repeat. Also, the f_seq values will be constant for a client for 128K creates until a client gets a new one, so that isn't giving you any fan-out, so those levels of the tree should just be dropped.

This is a serious design mistake in the POSIX copytool that we shouldn't repeat here.

Instead, it probably makes sense to have a 2^16 fanout by f_oid at the top level, and then store the FID in the leaf. That will give 16 files/leaf per million files created by this client (== 16K files/leaf per billion creates), and avoids all contention (there are not going to be 64K threads active on this client). Currently, "f_oid >> 16" will mostly be 0 or sometimes 1, so it doesn't provide much fan-out.}}

A good layout would be:

(fid->f_oid & oxFFFF)/FID

Comments

[Andreas Dilger]

I agree that the current archive layout is sub-optimal and could be improved. If this is done, the archive root directory of the new-style tree should be named differently from the old archive root, to allow for the copytool to be upgraded on an existing archive. The copytool can check at startup time whether either/both the old and new archive root directory exist, and then do all creation and lookup in the new directory, but on ENOENT it would also do lookups in the old directory. On removal of archive entries from the old directory tree it should also try to remove the containing directories (ignoring errors), since most leaf directories will be empty. At some point the old root directory will be removed and the double lookup will no longer be needed.

That said, it would be even better to store the identifier from the archive as an xattr on the Lustre file. That would allow MDT inode migration for HSM files, which is currently not allowed because the Lustre FID will change when an inode is moved to a new MDT (LU-6866). It would also allow us to use a "normal" directory tree in the archive to shadow the directory tree in the source filesystem, rather than just seq/oid & 0xffff/FID.