[LU-15665] Interop sanity test_77b: BAD WRITE CHECKSUM: changed on the client after we checksummed i

Mon, 21 Mar 2022 22:30:24 +0000

This issue was created by maloo for sarah <sarah@whamcloud.com>

This issue relates to the following test suite run: https://testing.whamcloud.com/test_sets/4834f8fb-fd5c-4194-9ccf-e9d835f42a9b

test_77b failed with the following error:

file compare failed

test_77b and 77c failed with issue with checksum

[ 5023.052860] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 77b: checksum error on client write, read ============================================= 00:25:51 \(1644366351\)
[ 5023.509654] Lustre: DEBUG MARKER: == sanity test 77b: checksum error on client write, read ============================================= 00:25:51 (1644366351)
[ 5023.617470] Lustre: *** cfs_fail_loc=409, val=0***
[ 5023.640519] LustreError: 132-0: lustre-OST0004-osc-ffff8adfbf218000: BAD WRITE CHECKSUM: changed on the client after we checksummed it - likely false positive due to mmap IO (bug 11742): from 10.240.24.90@tcp inode [0x200001b75:0xb13a:0x0] object 0x0:23283 extent [0-4194303], original client csum eb84e6f3 (type 4), server csum eb84e6f2 (type 4), client csum now eb84e6f2
[ 5023.646579] LustreError: 7532:0:(osc_request.c:2214:osc_brw_redo_request()) @@@ redo for recoverable error -11  req@00000000080c9e50 x1724238243302336/t8589962009(8589962009) o4->lustre-OST0004-osc-ffff8adfbf218000@10.240.24.90@tcp:6/4 lens 488/448 e 0 to 0 dl 1644366396 ref 3 fl Interpret:RQU/4/0 rc 0/0 job:'dd.0'
[ 5025.182530] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to crc32, rc = 0
[ 5025.616448] Lustre: DEBUG MARKER: set checksum type to crc32, rc = 0
[ 5025.693759] Lustre: *** cfs_fail_loc=408, val=0***
[ 5025.696558] LustreError: 133-1: lustre-OST0004-osc-ffff8adfbf218000: BAD READ CHECKSUM: from 10.240.24.90@tcp inode [0x200001b75:0xb13a:0x0] object 0x0:23283 extent [0-4194303], client a41ffb01, server 6be2734b, cksum_type 1
[ 5025.700308] LustreError: 7533:0:(osc_request.c:2214:osc_brw_redo_request()) @@@ redo for recoverable error -11  req@000000006fb07563 x1724238243303808/t0(0) o3->lustre-OST0004-osc-ffff8adfbf218000@10.240.24.90@tcp:6/4 lens 488/440 e 0 to 0 dl 1644366398 ref 2 fl Interpret:RMQU/0/0 rc 4194304/4194304 job:'cmp.0'
[ 5027.214163] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to adler, rc = 0
[ 5027.666981] Lustre: DEBUG MARKER: set checksum type to adler, rc = 0
[ 5027.753280] Lustre: *** cfs_fail_loc=408, val=0***
[ 5027.756489] LustreError: 133-1: lustre-OST0004-osc-ffff8adfbf218000: BAD READ CHECKSUM: from 10.240.24.90@tcp inode [0x200001b75:0xb13a:0x0] object 0x0:23283 extent [0-4194303], client aae10318, server ab1f0318, cksum_type 2
[ 5027.760191] LustreError: 7535:0:(osc_request.c:2214:osc_brw_redo_request()) @@@ redo for recoverable error -11  req@000000001555dd03 x1724238243304576/t0(0) o3->lustre-OST0004-osc-ffff8adfbf218000@10.240.24.90@tcp:6/4 lens 488/440 e 0 to 0 dl 1644366400 ref 2 fl Interpret:RMQU/0/0 rc 4194304/4194304 job:'cmp.0'
[ 5029.260770] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to crc32c, rc = 0
[ 5029.730366] Lustre: DEBUG MARKER: set checksum type to crc32c, rc = 0
[ 5029.809203] Lustre: *** cfs_fail_loc=408, val=0***
[ 5029.810794] LustreError: 133-1: lustre-OST0004-osc-ffff8adfbf218000: BAD READ CHECKSUM: from 10.240.24.90@tcp inode [0x200001b75:0xb13a:0x0] object 0x0:23283 extent [0-4194303], client 9ed7add7, server eb84e6f2, cksum_type 4
[ 5029.815302] LustreError: 7534:0:(osc_request.c:2214:osc_brw_redo_request()) @@@ redo for recoverable error -11  req@000000008433361a x1724238243305344/t0(0) o3->lustre-OST0004-osc-ffff8adfbf218000@10.240.24.90@tcp:6/4 lens 488/440 e 0 to 0 dl 1644366402 ref 2 fl Interpret:RMQU/0/0 rc 4194304/4194304 job:'cmp.0'
[ 5031.154848] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to t10ip512, rc = 0
[ 5031.572089] Lustre: DEBUG MARKER: set checksum type to t10ip512, rc = 0
[ 5032.071143] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  sanity test_77b: @@@@@@ FAIL: file compare failed 
[ 5032.497176] Lustre: DEBUG MARKER: sanity test_77b: @@@@@@ FAIL: file compare failed

VVVVVVV DO NOT REMOVE LINES BELOW, Added by Maloo for auto-association VVVVVVV
sanity test_77b - file compare failed

Whamcloud Community JIRA

[LU-15665] Interop sanity test_77b: BAD WRITE CHECKSUM: changed on the client after we checksummed i