[LU-441] ll_fill_super()) Unable to process log: -108

• Type: Bug
• Resolution: Cannot Reproduce
• Priority: Minor
• Affects Version/s: Lustre 1.8.8, Lustre 1.8.7, Lustre 1.8.6

Environment

Lustre Branch: v1_8_6_RC2
Lustre Build: http://newbuild.whamcloud.com/job/lustre-b1_8/80/
e2fsprogs Build: http://newbuild.whamcloud.com/job/e2fsprogs-master/40/
Distro/Arch: RHEL6/x86_64(patchless client, in-kernel OFED, kernel version: 2.6.32-131.2.1.el6)
                    RHEL5/x86_64(server, OFED 1.5.3.1, kernel version: 2.6.18-238.12.1.el5_lustre)
ENABLE_QUOTA=yes
FAILURE_MODE=HARD

MGS/MDS Nodes: client-10-ib(active), client-12-ib(passive)
                                                           \ /
                                            1 combined MGS/MDT

OSS Nodes: fat-amd-1-ib(active), fat-amd-2-ib(active)
                                                   \ /
                                               OST1 (active in fat-amd-1-ib)
                                               OST2 (active in fat-amd-2-ib)
                                               OST3 (active in fat-amd-1-ib)
                                               OST4 (active in fat-amd-2-ib)
                                               OST5 (active in fat-amd-1-ib)
                                               OST6 (active in fat-amd-2-ib)

Client Nodes: fat-amd-3-ib, client-6-ib

• Severity: 3
• Bugzilla ID: 20,997
• Assignee: WC Triage
• Reporter: Jian Yu

Description

replay-single test 0c failed as follows:

== test 0c: expired recovery with no clients == 22:09:59
Filesystem           1K-blocks      Used Available Use% Mounted on
client-10-ib@o2ib:client-12-ib@o2ib:/lustre
                      11811168    485956  10724828   5% /mnt/lustre
Failing mds on node client-12-ib
+ pm -h powerman --off client-12
Command completed successfully
affected facets: mds
+ pm -h powerman --on client-12
Command completed successfully
df pid is 14399
Failover mds to client-10-ib
22:10:44 (1308633044) waiting for client-10-ib network 900 secs ...
22:10:44 (1308633044) network interface is UP
Starting mds: -o user_xattr,acl  /dev/disk/by-id/scsi-1IET_00010001 /mnt/mds
client-10-ib: lnet.debug=0x33f1504
client-10-ib: lnet.subsystem_debug=0xffb7e3ff
client-10-ib: lnet.debug_mb=48
Started lustre-MDT0000
Starting client: fat-amd-3-ib: -o user_xattr,acl,flock client-10-ib@o2ib:client-12-ib@o2ib:/lustre /mnt/lustre
mount.lustre: mount client-10-ib@o2ib:client-12-ib@o2ib:/lustre at /mnt/lustre failed: Cannot send after transport endpoint shutdown
 replay-single test_0c: @@@@@@ FAIL: mount fails

Dmesg on the client node fat-amd-3:

Lustre: 9996:0:(client.c:1487:ptlrpc_expire_one_request()) @@@ Request x1372200800092968 sent from MGC192.168.4.10@o2ib to NID 192.168.4.10@o2ib 0s ago has failed due to network error (5s prior to deadline).
  req@ffff8800d409d800 x1372200800092968/t0 o250->MGS@MGC192.168.4.10@o2ib_0:26/25 lens 368/584 e 0 to 1 dl 1308633093 ref 1 fl Rpc:N/0/0 rc 0/0
Lustre: 9996:0:(client.c:1487:ptlrpc_expire_one_request()) Skipped 6 previous similar messages
LustreError: 14511:0:(client.c:858:ptlrpc_import_delay_req()) @@@ IMP_INVALID  req@ffff8800d409d400 x1372200800092971/t0 o501->MGS@MGC192.168.4.10@o2ib_1:26/25 lens 264/432 e 0 to 1 dl 0 ref 1 fl Rpc:/0/0 rc 0/0
LustreError: 15c-8: MGC192.168.4.10@o2ib: The configuration from log 'lustre-client' failed (-108). This may be the result of communication errors between this node and the MGS, a bad configuration, or other errors. See the syslog for more information.
LustreError: 14511:0:(llite_lib.c:1095:ll_fill_super()) Unable to process log: -108
Lustre: client ffff8801192e9800 umount complete
LustreError: 14511:0:(obd_mount.c:2065:lustre_fill_super()) Unable to mount  (-108)
Lustre: DEBUG MARKER: replay-single test_0c: @@@@@@ FAIL: mount fails

Maloo report: https://maloo.whamcloud.com/test_sets/ed5b5ff0-9bca-11e0-9a27-52540025f9af

This is an known issue on Lustre b1_8 branch: bug 20997

Issue Links

• is related to: LU-630 mount failure after MGS connection lost and file system is unmounted
• Trackbacks: Lustre 1.8.6-wc1 release testing tracker Lustre 1.8.6wc1 RC1 Tag: v186RC1 Created Date: 20110610 RC1 was DOA due to a build failure related to tag name LU408
• Trackbacks: Lustre 1.8.7-wc1 release testing tracker Lustre 1.8.7wc1 RC1 Tag: v187WC1RC1 Build:
• Trackbacks: Lustre 1.8.8-wc1 release testing tracker Lustre 1.8.8wc1 RC1 Tag: v188WC1RC1 Build:
• Trackbacks: Lustre 1.8.x known issues tracker While testing against Lustre b18 branch, we would hit known bugs which were already reported in Lustre Bugzilla https://bugzilla.lustre.org/. In order to move away from relying on Bugzilla, we would create a JIRA
• Trackbacks: Lustre 2.1.0 release testing tracker Lustre 2.1.0 RC0 Tag: v2100RC0 Created Date: 20110820 The difference between RC0 and RC1 is only a date change in lustre/ChangeLog. Lustre 2.1....
• Trackbacks: Lustre 2.1.1 release testing tracker Lustre 2.1.1 RC4 Tag: v2110RC4 Build: