[LU-9551] I/O errors when lustre uses multipath devices

• Type: Bug
• Resolution: Fixed
• Priority: Critical
• Fix Version/s: Lustre 2.12.0, Lustre 2.10.4
• Affects Version/s: Lustre 2.7.0
• Environment: CentOS Linux release 7.3.1611 (Core),OFED.3.4.2.0.0.1,lustre-2.7.19.8,Mellanox Technologies MT27500 Family

Epic/Theme

centos7.3 lustre-2.7.19.8

• Severity: 3

Epic

mount server

• Assignee: Nathaniel Clark
• Reporter: xiangmin shen (Inactive)

Description

When the lustre servers have OST configured with multipath devices, there are I/O errors that can lead to a server crash.

The following error appears in the system log:
Mar 31 00:02:44 oss01 kernel: blk_cloned_rq_check_limits: over max size limit.
Mar 31 00:02:44 oss01 kernel: device-mapper: multipath: Failing path 8:160.

Followed by several I/O errors
Mar 31 00:17:30 oss01 kernel: blk_update_request: I/O error, dev dm-17, sector 1182279680
Mar 31 00:17:30 oss01 kernel: blk_update_request: I/O error, dev dm-17, sector 1182291968
Mar 31 00:17:30 oss01 kernel: blk_update_request: I/O error, dev dm-17, sector 1182267392
Mar 31 00:17:30 oss01 kernel: blk_update_request: I/O error, dev dm-17, sector 1182304256
Mar 30 21:04:22 oss01 kernel: LDISKFS-fs (dm-17): Remounting filesystem read-only

Issue Links

• duplicates: LU-9132 Tuning max_sectors_kb on mount
• is duplicated by: LU-9552 lustre uses multipath devices I/O errors
• is related to: LU-12029 do not try to muck with max_sectors_kb on multipath configurations
• is related to: LU-12387 l_tunedisk does not properly handle multipath devices
• is related to: LU-11563 99-lustre.rules on clients tries to execute non-existent /usr/sbin/l_tunedisk
• is related to: LU-12530 udev add/change rule loads zfs module on clients
• is related to: LU-11736 Do not apply bulk IO tuning on MDT or MGT devices
• is related to: LU-10898 conf-sanity test 32a and 32d fail with ‘rmmod: ERROR: Module zfs is in use’