[LU-11697] BAD WRITE CHECKSUM with t10ip4K and t10ip512 checksums

• Type: Bug
• Resolution: Fixed
• Priority: Blocker
• Fix Version/s: Lustre 2.12.0
• Affects Version/s: Lustre 2.12.0
• Severity: 3
• Assignee: Li Xi
• Reporter: Alex Zhuravlev

Description

I'm running racer and get the following messages quite often. I think Oleg can confirm this.
if I set checksum_type to adler, then racer runs fine with no single bad checksum.

LustreError: 168-f: lustre-OST0000: BAD WRITE CHECKSUM: from 12345-0@lo inode [0x200000402:0x3b89:0x0] object 0x0:1921 extent [326-4194303]: client csum 53f1f04e, server csum 9bf3f057
LustreError: 132-0: lustre-OST0000-osc-ffff8801d25e8800: BAD WRITE CHECKSUM: changed in transit before arrival at OST: from 0@lo inode [0x200000402:0x3b89:0x0] object 0x0:1921 extent [326-4194303], original client csum 53f1f04e (type 20), server csum 9bf3f057 (type 20), client csum now 53f1f04e

Issue Links

• is related to: LU-11729 ARM: sanity test_810: BAD WRITE CHECKSUM with adler
• is related to: LU-10472 Data Integrity(T10PI) support for Lustre
• is related to: LU-11663 corrupt data after page-unaligned write with zfs backend lustre 2.10

Comments

[Li Xi]

Please check whether the patch 33752 on top of 33727 works well. I am setuping an environment to test it. And also, setuping a environment with T10PI hardware to check whether everything works.

[Gerrit Updater]

Li Xi ([e-mail]) uploaded a new patch: https://review.whamcloud.com/33752
Subject: LU-11697 obdclass: generate T10PI correctly for unaligned buffer
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 267bc620de222b23ad250f300b8ac0ad796e52f2

[Li Xi]

obd_page_dif_generate_buffer() doesn't handle the non-aligned buffer well. I am checking how T10PI handles the unaligned buffer.

[Alex Zhuravlev]

lixi_wc, can you please explain why it fails depending on page offset? if we put data (2K, iirc) at 2K offset, then the checksums match, but if we put data at 0 offset, then the checksums don't match. probably I missed something, of course..

[Li Xi]

The sanity:810 test that https://review.whamcloud.com/#/c/33726/ adds is still 512 bytes aligned. I guess that is why T10PI512 passes the test, but T10PI4096 doesn't. If the size is not 512 bytes aliged, I guess T10PI512 will fail too.

[Alex Zhuravlev]

adilger, yes it still fails with 4K version, but passes with 512.

[Andreas Dilger]

Alex, is that still failing with the 33727 patch applied on top of your 33726 patch?

[Alex Zhuravlev]

so that test from LU-11663 passes with t10ip512, but not with 4K. how 4K version is supposed to work with partial pages?

[Alex Zhuravlev]

lixi_wc then please try to run the test mentioned above. basically the client sends partial non-aligned page, but the server puts data with page offset=0.

[Li Xi]

Hi Alex, page offset is not used in T10 algorithm.

[Alex Zhuravlev]

lixi_wc, is page offset used in T10 algo?
it looks like the checksum is different if page offset on the server and the client don't match. other algorithms (e.g. adler) seem to be insensitive to this.