https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-4485 Lustre <p>On a client we have an initial problem with quota "kernel: LustreError: 11-0: an error occurred while communicating with 10.121.13.59@tcp. The ost_write operation failed with -122" , after we have some error message of which we do not understand the meaning. Do you have any suggestions?</p> <p>Regards</p> <p>Augusto Casciola</p> operating system redhat 5.7<br/> lustre 1.8.7 LU-4485

Some error message on lustre client

Task Major Resolved Not a Bug Niu Yawei Supporto Lustre Jnet2000 Tue, 14 Jan 2014 14:35:09 +0000 Thu, 27 Feb 2014 18:47:33 +0000 Thu, 27 Feb 2014 18:47:33 +0000 Lustre 1.8.7 0 4 <p>Niu</p> <p>Could you please advise with this ticket?</p> <p>Thanks</p> <p>Peter</p> <p>The messages means "write failed with EDQUOT (run out of quota)", looks some user is over quota.</p> <p>You can use "lfs quota -u uid/gid -v fsname" to check the quota limit and usage for the user, if it's not over quota, could you upload the syslog on OSS (10.121.13.59@tcp) to see why it returned EDQUOT?</p> <p>In the attachment file we have these errors after the error of quota. Refer to the quota ?</p> <p>"2014-01-07T17:11:40.337576+01:00 osiride-lp-041 kernel: Lustre: 23101:0:(client.c:1487:ptlrpc_expire_one_request()) @@@ Request x1436582400258712 sent from home-OST0008-osc-ffff81063fc2f800 to NID 10.121.13.28@tcp 7s ago has timed out (7s prior to deadline).<br/> 2014-01-07T17:11:40.337583+01:00 osiride-lp-041 kernel: req@ffff810806324c00 x1436582400258712/t0 o103-&gt;home-OST0008_UUID@10.121.13.28@tcp:17/18 lens 312/384 e 0 to 1 dl 1389111100 ref 2 fl Rpc:N/0/0 rc 0/0<br/> 2014-01-07T17:11:40.337589+01:00 osiride-lp-041 kernel: Lustre: home-OST0008-osc-ffff81063fc2f800: Connection to service home-OST0008 via nid 10.121.13.28@tcp was lost; in progress operations using this service will wait for recovery to complete.<br/> 2014-01-07T17:11:40.337595+01:00 osiride-lp-041 kernel: LustreError: 23101:0:(ldlm_request.c:1039:ldlm_cli_cancel_req()) Got rc -11 from cancel RPC: canceling anyway<br/> 2014-01-07T17:11:40.337601+01:00 osiride-lp-041 kernel: LustreError: 23101:0:(ldlm_request.c:1597:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -11<br/> 2014-01-07T17:11:40.866529+01:00 osiride-lp-041 kernel: Lustre: 7801:0:(client.c:1487:ptlrpc_expire_one_request()) @@@ Request x1436582400258725 sent from home-OST0008-osc-ffff81063fc2f800 to NID 10.121.13.28@tcp 7s ago has timed out (7s prior to deadline).<br/> 2014-01-07T17:11:40.866542+01:00 osiride-lp-041 kernel: req@ffff8107ddac0800 x1436582400258725/t0 o13-&gt;home-OST0008_UUID@10.121.13.28@tcp:7/4 lens 192/528 e 0 to 1 dl 1389111100 ref 2 fl Rpc:/0/0 rc 0/0<br/> 2014-01-07T17:11:41.263604+01:00 osiride-lp-041 kernel: Lustre: home-OST000b-osc-ffff81063fc2f800: Connection to service home-OST000b via nid 10.121.13.28@tcp was lost; in progress operations using this service will wait for recovery to complete.<br/> 2014-01-07T17:11:41.263618+01:00 osiride-lp-041 kernel: failure to allocate a tage (491)<br/> 2014-01-07T17:11:41.263624+01:00 osiride-lp-041 kernel: LustreError: 7780:0:(ldlm_request.c:1039:ldlm_cli_cancel_req()) Got rc -11 from cancel RPC: canceling anyway<br/> 2014-01-07T17:11:41.263628+01:00 osiride-lp-041 kernel: LustreError: 7780:0:(ldlm_request.c:1597:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -11<br/> 2014-01-07T17:11:41.566994+01:00 osiride-lp-041 kernel: Lustre: 23093:0:(client.c:1487:ptlrpc_expire_one_request()) @@@ Request x1436582400258713 sent from home-OST0007-osc-ffff81063fc2f800 to NID 10.121.13.28@tcp 8s ago has timed out (8s prior to deadline).<br/> 2014-01-07T17:11:41.567020+01:00 osiride-lp-041 kernel: req@ffff8109d781c400 x1436582400258713/t0 o103-&gt;home-OST0007_UUID@10.121.13.28@tcp:17/18 lens 304/384 e 0 to 1 dl 1389111101 ref 2 fl Rpc:N/0/0 rc 0/0<br/> 2014-01-07T17:11:41.567025+01:00 osiride-lp-041 kernel: Lustre: 23093:0:(client.c:1487:ptlrpc_expire_one_request()) Skipped 3 previous similar messages<br/> 2014-01-07T17:11:41.785397+01:00 osiride-lp-041 kernel: failure to allocate a tage (9)<br/> 2014-01-07T17:11:41.785412+01:00 osiride-lp-041 kernel: Lustre: home-OST0006-osc-ffff81063fc2f800: Connection to service home-OST0006 via nid 10.121.13.28@tcp was lost; in progress operations using this service will wait for recovery to complete.<br/> 2014-01-07T17:11:41.785416+01:00 osiride-lp-041 kernel: Lustre: Skipped 3 previous similar messages<br/> 2014-01-07T17:11:41.785421+01:00 osiride-lp-041 kernel: LustreError: 17973:0:(ldlm_request.c:1039:ldlm_cli_cancel_req()) Got rc -11 from cancel RPC: canceling anyway<br/> 2014-01-07T17:11:41.785426+01:00 osiride-lp-041 kernel: LustreError: 17973:0:(ldlm_request.c:1039:ldlm_cli_cancel_req()) Skipped 7 previous similar messages<br/> 2014-01-07T17:11:41.785430+01:00 osiride-lp-041 kernel: LustreError: 17973:0:(ldlm_request.c:1597:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -11<br/> 2014-01-07T17:11:41.785434+01:00 osiride-lp-041 kernel: LustreError: 17973:0:(ldlm_request.c:1597:ldlm_cli_cancel_list()) Skipped 7 previous similar messages<br/> 2014-01-07T17:11:41.814197+01:00 osiride-lp-041 kernel: failure to allocate a tage (18)"</p> <p>Sorry for the misunderstanding. We need to know because the connection to home-OST0008, home-OST0008 and home-OST0006 was lost by client and the meaning of the "failure to allocate a tage" error.<br/> Thanks</p> <p>The "failure to allocate a tage" means Lustre logging system can't allocate buffer to store debug message, and the result is that some debug message will be lost. It won't break the connection between client and OSTs.</p> <p>So, client lost connection to OST0006, OST0007 and OST0008? and you want to know why the client lost connections?</p> <p>Hi, we want know because there has been the client lost of connection to OST0006, OST0007 and OST0008.</p> <p>Regards</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java">2014-01-07T17:11:57.274407+01:00 osiride-lp-041 kernel: Lustre: 7567:0:(<span class="code-keyword">import</span>.c:517:import_select_connection()) home-OST000a-osc-ffff81063fc2f800: tried all connections, increasing latency to 3s </pre> </div></div> <p>I suspect it's a network problem, not related to the write failures (-122 EDUOT error).</p> <p>I have talked with the customer and we agreed that this is a network problem. We can close this issue. In case of other similar errors, we can activate the debug daemon in order to have more informations.</p> <p>ok - thanks Gabriele</p> Development Rank 1|hzwcsn: Rank (Obsolete) 12276