[#LU-1039] data corruption in check

[LU-1039] data corruption in check_set Created: 26/Jan/12 Updated: 22/Dec/12 Resolved: 02/Mar/12
Status:	Resolved
Project:	Lustre
Component/s:	None
Affects Version/s:	Lustre 2.2.0, Lustre 2.1.4
Fix Version/s:	Lustre 2.2.0

Type:

Bug

Priority:

Blocker

Reporter:

Alexey Lyashkov

Assignee:

Oleg Drokin

Resolution:

Fixed

Votes:

Labels:

None

Environment:

any lustre

Attachments:

0001-MRP-303-handle-bulk-IO-errors-correctly.patch

Issue Links:

Duplicate
is duplicated by	~~LU-2260~~	2.1.3 Client LBUG	Resolved

Sub-Tasks:

Key

Summary

Type

Status

Assignee

LU-1791

sanity.sh test_224b takes too long to...

Technical task

Resolved

WC Triage

Severity:

Rank (Obsolete):

4702

Description

ost_brw_read set a number bytes as rq_status, that confuses check_set() function.
that easy see if checksumming enabled.
i found it's with testing solution to avoid panic in check_set() with request reorder and failed bulk read.
attached patch solves both issues, but broke a request flags policy, and don't resend a bulk request so that can be short time solution, until check_set will cleared.

Comments

Comment by Alexey Lyashkov [ 26/Jan/12 ]

Jan 26 05:58:31 rhel6-64 kernel: Lustre: DEBUG MARKER: cancel_lru_locks osc stop
LustreError: 4233:0:(libcfs_fail.h:81:cfs_fail_check_set()) *** cfs_fail_loc=515 ***
LustreError: 4233:0:(events.c:201:client_bulk_callback()) event type 5, status -5, desc ffff880050adab90
Jan 26 05:58:31 LustreError: 133-1: lustre-OST0001-osc-ffff880053338348: BAD READ CHECKSUM: from 0@lo via 0@<0:0> inode [0x0:0x0:0x0] object 18499/0 extent [0-409
5]
rhel6-64 kernel: LustreError: 42LustreError: 4233:0:(osc_request.c:1660:osc_brw_fini_request()) client ffffffff, server 6706be76, cksum_type 4
33:0:(libcfs_fail.h:81:cfs_fail_LustreError: 4233:0:(osc_request.c:1749:osc_brw_redo_request()) @@@ redo for recoverable error req@ffff8800855368a0 x139203734877
0001/t0(0) o3->lustre-OST0001-osc-ffff880053338348@0@lo:6/4 lens 456/400 e 0 to 0 dl 1327550318 ref 2 fl Interpret:R/0/0 rc 4096/4096
check_set()) *** cfs_fail_loc=515 ***
Jan 26 05:58:31 rhel6-64 kernel: LustreError: 4233:0:(events.c:201:client_bulk_callback()) event type 5, status -5, desc ffff880050adab90
Jan 26 05:58:31 rhel6-64 kernel: LustreError: 133-1: lustre-OST0001-osc-ffff880053338348: BAD READ CHECKSUM: from 0@lo via 0@<0:0> inode [0x0:0x0:0x0] object 1849
9/0 extent [0-4095]
Jan 26 05:58:31 rhel6-64 kernel: LustreError: 4233:0:(osc_request.c:1660:osc_brw_fini_request()) client ffffffff, server 6706be76, cksum_type 4
Jan 26 05:58:31 rhel6-64 kernel: LustreError: 4233:0:(osc_request.c:1749:osc_brw_redo_request()) @@@ redo for recoverable error req@ffff8800855368a0 x13920373487
70001/t0(0) o3->lustre-OST0001-osc-ffff880053338348@0@lo:6/4 lens 456/400 e 0 to 0 dl 1327550318 ref 2 fl Interpret:R/0/0 rc 4096/4096
LustreError: 133-1: lustre-OST0001-osc-ffff880053338348: BAD READ CHECKSUM: from 0@lo via 0@<0:0> inode [0x0:0x0:0x0] object 18499/0 extent [0-4095]
LustreError: 4233:0:(osc_request.c:1660:osc_brw_fini_request()) client ffffffff, server 6706be76, cksum_type 4
LustreError: 4233:0:(osc_request.c:1749:osc_brw_redo_request()) @@@ redo for recoverable error req@ffff880083a3b5b8 x1392037348770002/t0(0) o3->lustre-OST0001-os
c-ffff880053338348@0@lo:6/4 lens 456/400 e 0 to 0 dl 1327550324 ref 2 fl Interpret:R/0/0 rc 4096/4096
Jan 26 05:58:37 rhel6-64 kernel: LustreError: 133-1: lustre-OST0001-osc-ffff880053338348: BAD READ CHECKSUM: from 0@lo via 0@<0:0> inode [0x0:0x0:0x0] object 1849
9/0 extent [0-4095]

Comment by Peter Jones [ 26/Jan/12 ]

Shadow

Thanks for the report. Could you please upload your patch into gerrit? That will help us with reviewing and landing this fix.

Thanks

Peter

Comment by Alexey Lyashkov [ 26/Jan/12 ]

remote: New Changes:
remote: http://review.whamcloud.com/2023

Comment by Peter Jones [ 26/Jan/12 ]

Thanks Shadow!

Comment by Peter Jones [ 26/Jan/12 ]

Ah, but could you please fix the formatting error in the summary line - thanks!