-
Bug
-
Resolution: Fixed
-
Medium
-
None
-
None
-
3
-
9223372036854775807
llapi_semantic_traverse() hands a directory descriptor to its sem_init
callback by pointer, then calls fdopendir() on its own copy.
cb_get_dirstripe() closes that descriptor and stores an O_NOFOLLOW reopen
through the pointer when LL_IOC_LMV_GETSTRIPE answers ENOTTY, which it does
for the fake symlink a foreign file or directory presents and for any
directory not on Lustre at all.
Two callers pass the address of a local copy, so the traversal keeps the
number that was closed:
lustre/utils/liblustreapi_pfind.c:2423 int d = dp == NULL ? -1 : *dp; lustre/utils/liblustreapi_pfind.c:2510 ret = cb_get_dirstripe(path, &d, param); lustre/utils/liblustreapi.c:3551 int d = dp == NULL ? -1 : *dp, fd = -1; lustre/utils/liblustreapi.c:3569 ret = cb_get_dirstripe(path, &d, param);
It then descends on a closed descriptor, so every object below that
directory is skipped in silence; closes that number a second time on the way
out, by which point the walk may have reissued it to another thread; and
leaks the descriptor that was opened.
Reproduced on master at 5afbab284e with a tmpfs mounted inside a Lustre
mount point, holding a.txt and sub/b.txt:
# no predicate, so cb_get_dirstripe() is never called: correct $ lfs find /mnt/testfs/nl /mnt/testfs/nl /mnt/testfs/nl/a.txt /mnt/testfs/nl/sub /mnt/testfs/nl/sub/b.txt # any predicate needing the directory stripe: the subtree is gone $ lfs find /mnt/testfs/nl --printf '%p\n' lt-lfs: failed for '/mnt/testfs/nl': Operation not permitted $ echo $? 1
Each of these reproduces identically:
--printf --links --mdt-count --mdt-hash --foreign
The errno is wrong too: cb_get_dirstripe() returns the ioctl's -1 rather
than a negative errno and cb_find_init() passes it out unchanged, so ENOTTY
is reported as "Operation not permitted".
lfs getdirstripe and lfs getstripe -D reach the same call through
cb_getstripe(), but stop at the ENOTTY in any case, so there it is the
double close that matters rather than a lost walk. Plain lfs getstripe -r
sets neither fp_get_lmv nor fp_get_default_lmv and does not reach it.
That makes lfs getdirstripe -r the clearest view of the descriptor, the two
runs being identical until the final close (strace of the main process):
openat(AT_FDCWD, "/mnt/testfs/nl2", O_RDONLY|O_NONBLOCK|O_DIRECTORY) = 3 openat(AT_FDCWD, "/mnt/testfs/nl2", O_RDONLY|O_NONBLOCK|O_NOFOLLOW) = 4 close(3) = 0 openat(AT_FDCWD, "/mnt/testfs/nl2", O_RDONLY|O_NONBLOCK|O_NOFOLLOW) = 3 close(4) = 0 close(4) = -1 EBADF <- without the fix close(3) = 0 <- with it
3 is the traversal's descriptor and 4 the reopen. The third line is
cb_get_dirstripe() closing the original, and the openat after it takes 3
straight back while the caller still believes it owns that number.
The fix is to write the descriptor back in both callers:
ret = cb_get_dirstripe(path, &d, param);
if (dp != NULL)
*dp = d;
The third caller, in the dangling-symlink arm of cb_getstripe(), opens a
descriptor it closes itself and is correct as it stands.