[LU-19678] interop sanity-flr test_70a: FAIL: fsx FLR file /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr failed

• Type: Bug
• Resolution: Unresolved
• Priority: Medium
• Affects Version/s: Lustre 2.17.0
• Labels: tstu_interop
• Severity: 3
• Assignee: WC Triage
• Reporter: Maloo

Description

This issue was created by maloo for jianyu <[e-mail]>

This issue relates to the following test suite run: https://testing.whamcloud.com/test_sets/d50bba9d-cf2a-412a-9e1e-c296e60f2e3c

test_70a failed with the following error:

READ BAD DATA: offset = 0xaf32, size = 0xd813
OFFSET	GOOD	BAD	RANGE
0x0b235	000000	0x0218	 0x1774
operation# (mod 256) for the bad data may be 24
LOG DUMP (85 total operations):
1: 1764923582.108175 FALLOC   0x1982b thru 0x22fd4	(0x097a9 bytes)EXTENDING
2: 1764923582.109859 MIRROR_EXTEND to 2 mirrors
3: 1764923582.297048 SKIPPED (no operation)
4: 1764923582.298702 MIRROR_RESYNC 2 mirrors
5: 1764923582.347134 PUNCH    0x06aa5 thru 0x0e6eb	(0x07c47 bytes)	******PPPP
6: 1764923582.352179 MAPWRITE 0x0d87b thru 0x14246 (0x069cc bytes)
7: 1764923582.376338 PUNCH    0x1bc8e thru 0x22fd3	(0x07346 bytes)
8: 1764923582.377168 MAPWRITE 0x19429 thru 0x1bfe9 (0x02bc1 bytes)
9: 1764923582.393318 SKIPPED (no operation)
10: 1764923582.393641 SKIPPED (no operation)
11: 1764923582.393960 READ     0x02aeb thru 0x0a0e6 (0x075fc bytes)
12: 1764923582.395064 MAPWRITE 0x2baad thru 0x33cc2 (0x08216 bytes)
13: 1764923582.418941 READ     0x040d3 thru 0x06bc2 (0x02af0 bytes)
14: 1764923582.419454 PUNCH    0x1e0a4 thru 0x27e37	(0x09d94 bytes)
15: 1764923582.420343 TRUNC_DN 0x33cc3 to 0x25b2c
16: 1764923582.422229 SKIPPED (no operation)
17: 1764923582.422553 READ     0x216dd thru 0x25b2b (0x0444f bytes)
18: 1764923582.423624 PUNCH    0x0ea55 thru 0x19170	(0x0a71c bytes)
19: 1764923582.424438 WRITE    0x3a2f1 thru 0x3dde0 (0x03af0 bytes) HOLE
20: 1764923582.425087 MAPREAD  0x0afda thru 0x0ed91 (0x03db8 bytes)	***RRRR***
21: 1764923582.426313 WRITE    0x1f8b3 thru 0x2f0b0 (0x0f7fe bytes)
22: 1764923582.427150 MAPWRITE 0x0b235 thru 0x0c9b6 (0x01782 bytes)	******WWWW
23: 1764923582.444013 SKIPPED (no operation)
24: 1764923582.444352 MAPWRITE 0x178ea thru 0x23371 (0x0ba88 bytes)
25: 1764923582.461004 SKIPPED (no operation)
26: 1764923582.461341 WRITE    0x09fe8 thru 0x0a5c5 (0x005de bytes)
27: 1764923582.462443 MIRROR_EXTEND to 3 mirrors
28: 1764923582.652191 SKIPPED (no operation)
29: 1764923582.653510 MAPREAD  0x2d6f4 thru 0x36783 (0x09090 bytes)
30: 1764923582.656398 FALLOC   0x1d49c thru 0x2b075	(0x0dbd9 bytes)INTERIOR
31: 1764923582.660705 READ     0x20759 thru 0x2cc5c (0x0c504 bytes)
32: 1764923582.661853 TRUNC_DN 0x3dde1 to 0x1616f
33: 1764923582.667731 FALLOC   0x22c67 thru 0x295aa	(0x06943 bytes)EXTENDING
34: 1764923582.668650 MAPREAD  0x22f70 thru 0x295a9 (0x0663a bytes)
35: 1764923582.670522 SKIPPED (no operation)
36: 1764923582.670844 WRITE    0x3ed94 thru 0x3ffff (0x0126c bytes) HOLE
37: 1764923582.671469 SKIPPED (no operation)
38: 1764923582.672571 MIRROR_SPLIT mirror 1 to 2 mirrors
39: 1764923582.702569 SKIPPED (no operation)
40: 1764923582.702899 READ     0x2b2bf thru 0x357c1 (0x0a503 bytes)
41: 1764923582.703906 PUNCH    0x22543 thru 0x2fcf7	(0x0d7b5 bytes)
42: 1764923582.704614 TRUNC_DN 0x40000 to 0x0f4d5
43: 1764923588.447604 MIRROR_RESYNC 2 mirrors
44: 1764923588.611739 READ     0x0cfbf thru 0x0f4d4 (0x02516 bytes)
45: 1764923588.611987 MAPWRITE 0x153b5 thru 0x15c71 (0x008bd bytes)
46: 1764923588.637218 MIRROR_RESYNC 2 mirrors
47: 1764923588.746654 PUNCH    0x0cd3b thru 0x15c71	(0x08f37 bytes)
48: 1764923588.750291 SKIPPED (no operation)
49: 1764923588.750617 MAPREAD  0x06c59 thru 0x102b0 (0x09658 bytes)	***RRRR***
50: 1764923588.752458 SKIPPED (no operation)
51: 1764923588.752780 READ     0x0d20d thru 0x15c71 (0x08a65 bytes)
52: 1764923588.754194 MIRROR_EXTEND to 3 mirrors
53: 1764923588.953387 SKIPPED (no operation)
54: 1764923588.954667 MAPREAD  0x05cf4 thru 0x11bd1 (0x0bede bytes)	***RRRR***
55: 1764923588.957480 READ     0x0e556 thru 0x15c71 (0x0771c bytes)
56: 1764923588.958861 PUNCH    0x11427 thru 0x15c71	(0x0484b bytes)
57: 1764923588.963534 MIRROR_RESYNC 3 mirrors
58: 1764923589.115295 WRITE    0x302c8 thru 0x3658f (0x062c8 bytes) HOLE
59: 1764923589.117999 MAPREAD  0x00e4a thru 0x065d1 (0x05788 bytes)
60: 1764923589.120240 TRUNC_DN 0x36590 to 0x1f57d
61: 1764923589.122743 MAPWRITE 0x0a5f0 thru 0x14cd4 (0x0a6e5 bytes)	******WWWW
62: 1764923589.145678 PUNCH    0x0d59e thru 0x1a2af	(0x0cd12 bytes)
63: 1764923589.146910 MIRROR_EXTEND to 4 mirrors
64: 1764923589.339939 TRUNC_DN 0x1f57d to 0x004ce	******TTTT
65: 1764923589.349250 SKIPPED (no operation)
66: 1764923589.349575 READ     0x00472 thru 0x004cd (0x0005c bytes)
67: 1764923589.350731 TRUNC_UP 0x004ce to 0x3dd77	******TTTT
68: 1764923589.352853 TRUNC_DN 0x3dd77 to 0x1abe8
69: 1764923589.354580 MAPREAD  0x18a2e thru 0x1abe7 (0x021ba bytes)
70: 1764923589.356774 READ     0x08051 thru 0x09da4 (0x01d54 bytes)
71: 1764923589.357794 SKIPPED (no operation)
72: 1764923589.358132 FALLOC   0x11681 thru 0x1b9e4	(0x0a363 bytes)EXTENDING
73: 1764923589.359489 MIRROR_EXTEND to 5 mirrors
74: 1764923589.543610 SKIPPED (no operation)
75: 1764923589.544943 TRUNC_UP 0x1b9e4 to 0x283fd
76: 1764923589.554116 PUNCH    0x0f207 thru 0x19125	(0x09f1f bytes)
77: 1764923589.554833 MAPWRITE 0x3cf24 thru 0x3d92c (0x00a09 bytes)
78: 1764923589.577917 SKIPPED (no operation)
79: 1764923589.578261 WRITE    0x0a318 thru 0x0a6f1 (0x003da bytes)
80: 1764923589.579375 READ     0x028e8 thru 0x10ec8 (0x0e5e1 bytes)	***RRRR***
81: 1764923589.580179 SKIPPED (no operation)
82: 1764923589.580499 MAPREAD  0x3381a thru 0x3d92c (0x0a113 bytes)
83: 1764923589.581694 PUNCH    0x1d33d thru 0x27da3	(0x0aa67 bytes)
84: 1764923589.582834 MIRROR_SPLIT mirror 4 to 4 mirrors
85: 1764923589.628964 READ     0x0af32 thru 0x18744 (0x0d813 bytes)	***RRRR***
Using seed 3582
Correct content saved for comparison
(maybe hexdump "/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr" vs "/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr.fsxgood")
 sanity-flr test_70a: @@@@@@ FAIL: fsx FLR file /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr failed

Test session details:
clients: https://build.whamcloud.com/job/lustre-master/4681 - 6.4.0-150700.53.22-default
servers: https://build.whamcloud.com/job/lustre-b2_15/117 - 4.18.0-553.82.1.el8_lustre.x86_64

<<Please provide additional information about the failure here>>

VVVVVVV DO NOT REMOVE LINES BELOW, Added by Maloo for auto-association VVVVVVV
sanity-flr test_70a - fsx FLR file /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr failed