-
Improvement
-
Resolution: Unresolved
-
Medium
-
None
-
None
-
None
-
3
-
9223372036854775807
Run xfstests' fsx (ltp/fsx.c) against Lustre as part of regular testing, alongside Lustre's own lustre/tests/fsx.c rather than in place of it. The starting point is one or more single-mount tests, each running the unmodified upstream tool with a different operation mix – for example buffered I/O, O_DIRECT, and AIO – with the op-mix flags, durations and any Lustre-specific exclusions kept in the test script, not in a patched fsx.
Why
Lustre's fsx.c and the xfstests fsx share an ancestor, but have diverged so far that they are effectively different tools. Ours carries the Lustre-only features that matter for a distributed filesystem (several mounts of one file, path rotation, and now burst mode in LU-20724), and is not going away. The xfstests copy is what Linux filesystem developers maintain and run every day; it keeps gaining operations and I/O paths that ours has never had, and nobody is porting them across.
Running both gives us that upstream coverage without a resync. Resyncing our fsx.c with xfstests would be a large, risky merge of two very different codebases; running upstream fsx as-is costs a build step and a test script, and it keeps picking up upstream additions for free.
Timothy Day raised this on the review of LU-20724: how far has our fsx diverged, and given how far, should we be testing with both? A trial run on a single mount says yes – it exercises paths our tests do not reach, and on its first run it found a data corruption bug (below).
Things to settle:
- How to get the tool into the test environment: build from an xfstests checkout, package it, or carry a pinned copy under lustre/tests.
- Which op mixes to run, and whether as one test or several.
- Whether it can later be pointed at two mounts of one file, as ours can, or stays single-mount.
Background
Lustre's fsx.c came into the tree in 2002 from the Apple/BSD lineage and was never resynced with the xfstests fork. Against current kdave/xfstests ltp/fsx.c, only about 390 of our ~2100 lines match; the xfstests copy is about 3700 lines. Some of what it adds does not apply to Lustre (reflink, dedupe, exchange range), but it tests for those at startup and disables them cleanly, so they do no harm.
Trial run on a single mount
xfstests' ltp/fsx (kdave/xfstests master), unmodified, against a single Lustre client mount, 1 MDT / 2 OST, rocky9 5.14 kernel. It builds with autoreconf plus stock libaio-devel, liburing-devel, xfsprogs-devel and libacl-devel.
- Buffered mode, 3000+ ops: clean. At startup it probes for, and disables, every fallocate mode and range op Lustre returns -EOPNOTSUPP for (collapse, insert, zero and unshare range; clone, dedupe and exchange range; dontcache I/O). That matches the allow-list in ll_fallocate() exactly.
- copy_file_range and AIO, which our fsx.c never issues: clean in buffered mode.
- io_uring: not tested. The image sets kernel.io_uring_disabled=2, and upstream fsx segfaults when the queue fails to initialise instead of skipping it; a test script would have to check for this first.
Data corruption with O_DIRECT and copy_file_range. With O_DIRECT (-Z -R -W, 4096-byte aligned), copy_file_range mixed with direct reads and writes corrupts the file: reads return stale or garbage bytes where zeroes or written data are expected. Reproduced twice, with the same kind of mismatch but different bytes each time. The same run with only copy_file_range disabled (-E) passes, so copy_file_range is what triggers it. Lustre has no copy_file_range of its own (LU-11621 is open), so this is the generic VFS fallback running over Lustre O_DIRECT. The root cause has not been investigated yet; it should probably have its own bug.
Details are on the LU-20724 review, https://review.whamcloud.com/68763, comments f8343391_0bd73f4f and 17ad468c_360ddffc (the proposal) and 09df6596_fe3cc215 (the trial run).