Details
-
Bug
-
Resolution: Cannot Reproduce
-
Critical
-
None
-
None
-
None
-
3
-
9223372036854775807
Description
Soak error:
Clients and servers are running soak build 20160324 - 2.8 plus http://review.whamcloud.com/#/c/18813/
https://wiki.hpdd.intel.com/pages/viewpage.action?title=Soak+Testing+on+Lola&spaceKey=Releases#SoakTestingonLola-20160324
Stopped soak due to separate issue.
Remounted all servers.
Ran umount -f on all clients to clear state.
Lola-34 would not amount
dimes:
Lustre: 4595:0:(client.c:2063:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1459465123/real 1459465123] req@ffff8803d37e29c0 x1529700237235520/t0(0) o38->soaked-MDT0001-mdc-ffff881032296000@192.168.1.109@o2ib10:12/10 lens 520/544 e 0 to 1 dl 1459465178 ref 1 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 4595:0:(client.c:2063:ptlrpc_expire_one_request()) Skipped 24 previous similar messages Lustre: Evicted from MGS (at MGC192.168.1.108@o2ib10_0) after server handle changed from 0x3514dbeb8b19d96b to 0xc17b0433aaa7b860 Lustre: MGC192.168.1.108@o2ib10: Connection restored to MGC192.168.1.108@o2ib10_0 (at 192.168.1.108@o2ib10) Lustre: soaked-MDT0001-mdc-ffff881032296000: Connection restored to 192.168.1.109@o2ib10 (at 192.168.1.109@o2ib10) Lustre: 4595:0:(client.c:2063:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1459465774/real 1459465774] req@ffff88043f0af980 x1529700237238268/t0(0) o400->soaked-MDT0000-mdc-ffff881032296000@192.168.1.108@o2ib10:12/10 lens 224/224 e 1 to 1 dl 1459465776 ref 1 fl Rpc:X/c0/ffffffff rc 0/-1 Lustre: 4595:0:(client.c:2063:ptlrpc_expire_one_request()) Skipped 93 previous similar messages Lustre: setting import soaked-MDT0000_UUID INACTIVE by administrator request LustreError: 4595:0:(client.c:1143:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff8801044fa0c0 x1529700237240872/t0(0) o38->soaked-MDT0000-mdc-ffff881032296000@192.168.1.108@o2ib10:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:N/0/ffffffff rc 0/-1 LustreError: 220977:0:(file.c:3201:ll_inode_revalidate_fini()) soaked: revalidate FID [0x200000007:0x1:0x0] error: rc = -5 LustreError: 239794:0:(file.c:3201:ll_inode_revalidate_fini()) soaked: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 LustreError: 239794:0:(file.c:3201:ll_inode_revalidate_fini()) Skipped 1 previous similar message LustreError: 239791:0:(file.c:3201:ll_inode_revalidate_fini()) soaked: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 LustreError: 239791:0:(file.c:3201:ll_inode_revalidate_fini()) Skipped 15 previous similar messages
Syslog:
Mar 31 16:19:07 lola-34 mrshd[239765]: root@lola-16.lola.whamcloud.com as root: cmd='PATH=/scratch/apps/openmpi/1.8.1/bin/:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin;umount -f /mnt/soaked' Mar 31 16:19:08 lola-34 kernel: Lustre: setting import soaked-MDT0000_UUID INACTIVE by administrator request Mar 31 16:19:08 lola-34 kernel: LustreError: 4595:0:(client.c:1143:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff8801044fa0c0 x1529700237240872/t0(0) o38->soaked-MDT0000-mdc-ffff881032296000@192.168.1.108@o2ib10:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:N/0/ffffffff rc 0/-1 Mar 31 16:19:08 lola-34 kernel: LustreError: 220977:0:(file.c:3201:ll_inode_revalidate_fini()) soaked: revalidate FID [0x200000007:0x1:0x0] error: rc = -5 Mar 31 16:19:08 lola-34 mrshd[239764]: pam_unix(mrsh:session): session closed for user root Mar 31 16:19:08 lola-34 xinetd[4025]: EXIT: mshell status=0 pid=239764 duration=1(sec) Mar 31 16:19:08 lola-34 kernel: LustreError: 239794:0:(file.c:3201:ll_inode_revalidate_fini()) soaked: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 Mar 31 16:19:08 lola-34 kernel: LustreError: 239794:0:(file.c:3201:ll_inode_revalidate_fini()) Skipped 1 previous similar message Mar 31 16:19:09 lola-34 kernel: LustreError: 239791:0:(file.c:3201:ll_inode_revalidate_fini()) soaked: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 Mar 31 16:19:09 lola-34 kernel: LustreError: 239791:0:(file.c:3201:ll_inode_revalidate_fini()) Skipped 15 previous similar messages