[LU-16409] ior-hard-write test fails with EINVAL

Fri, 16 Dec 2022 17:27:57 +0000

During io500 run (sc22 version) we have encoutered write error during ior-hard-write phase when posix.odirect option is supplied.

Problem doesn not occur without O_DIRECT

We have modified ior to be more verbose about the errno of the code.
write call is interrupted with errno 22 (EINVAL)

ERROR: write(114, 0x4c33000, 47008) failed, rc = 22, (aiori-POSIX.c:703)
ERROR: write(114, 0x2e80000, 47008) failed, rc = 22, (aiori-POSIX.c:703)

clients: 10, 32 processes per client
servers: 6 servers, 2 osts per server

ior commandline:

./ior --dataPacketType=timestamp -C -Q 1 -g -G=1853316614 -k -e -o /net/tscratch/tests/dw//2022.12.16-15.30.44/ior-hard/file -O stoneWallingStatusFile=./results/2022.12.16-15.30.44/ior-hard.stonewall -t 47008 -b 47008 -s 200000 -w -D 300 -a POSIX --posix.odirect -O saveRankPerformanceDetailsCSV=./results/2022.12.16-15.30.44/ior-hard-write.csv -O stoneWallingWearOut=1

striping setup: -c -1 -S 32m

lfs getstripe /net/tscratch/tests/dw//2022.12.16-15.30.44/ior-hard/file
/net/tscratch/tests/dw//2022.12.16-15.30.44/ior-hard/file
lmm_stripe_count:  12
lmm_stripe_size:   33554432
lmm_pattern:       raid0
lmm_layout_gen:    0
lmm_stripe_offset: 9
    obdidx         objid         objid         group
         9           2216075         0x21d08b                 0
        11           2211594         0x21bf0a                 0
         0            848231          0xcf167                 0
         2           2231886         0x220e4e                 0
         4            800648          0xc3788                 0
         6           2251269         0x225a05                 0
         8           2235886         0x221dee                 0
        10           2199341         0x218f2d                 0
         1           2008582         0x1ea606                 0
         3           2203243         0x219e6b                 0
         5            827881          0xca1e9                 0
         7           2201995         0x21998b                 0

Additional information:

No timeouts, no evictions happened during the test phase

This issue is reproducible on every run

–

Lukasz Flis

Whamcloud Community JIRA

[LU-16409] ior-hard-write test fails with EINVAL