-
Bug
-
Resolution: Unresolved
-
Medium
-
None
-
None
-
None
-
3
-
9223372036854775807
A buffered write(2) on a Lustre client can return 0 without an error
while the data is dropped.
ll_file_io_generic() restarts an IO when the lower layer sets
ci_need_restart. The budget is RETRY_ATTEMPTS, 1000
(lustre/llite/file.c:2058, :2081, :2241). One producer of the restart
is a layout generation mismatch: can_populate_pages()
(lustre/llite/vvp_io.c:76-92) compares the inode's cached layout
generation against the generation the IO snapshotted, and
vvp_io_write_start() then does RETURN(0) (lustre/llite/vvp_io.c:
1461-1462 on 2.14.0-ddn259, :1262-1263 on master), a success that
moves no bytes.
If the generation keeps changing, all 1000 attempts are used. Control
then falls through to RETURN(result > 0 ? result : rc) with result 0
and rc 0. The io->ci_result fallback does not help, because
cl_io_start() sets ci_result to 0 when the start method returns 0. The
VFS gets 0 and write(2) returns 0.
POSIX lets a write of a non-zero count return 0 only in narrow cases,
and applications do not usually treat it as an error. glibc, MPI-IO
and many application IO loops either stop or spin. The bytes are not
written and nothing above D_VFSTRACE is logged, so the loss is silent.
Exhausting the restart budget must be reported as an error, for
example -EAGAIN or -EIO, with a rate-limited console message that
names the FID and the reason for the restart. This is separate from
the causes of the generation churn; on the reported system the churn
came from client-side LDLM LRU reclaim under memory pressure.
Seen at 2.14.0-ddn259. The llite files are byte-identical on
b_es6_0, and master carries the same code.
Related: LU-18435 removed one server-side cause of the churn (lod
only, no client change). LU-6389 (gerrit 14123) added the restart loop
and the 1000-attempt budget. Gerrit 67432 (LU-17936) adds a new
-ENOLCK producer that reuses the same budget.
- is related to
-
LU-20590 llite: IO restarts when a cancel does not change the layout
-
- In Progress
-
- is related to
-
LU-17936 racer test_1: hang in mdt_reint_rename vs. mdt_object_local_lock with ldlm_completion_ast
-
- Open
-
-
LU-6389 read()/write() returning less than available bytes intermittently
-
- Resolved
-
-
LU-18435 replay of pfl layout resets layout generation
-
- Resolved
-
- resolves
-
EX-15147 Loading...