https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-4717 Lustre <p>Filing a ticket as instructed. Log file for a client is filled with stack traces from the following error. All stack traces are the same.</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>LustreError: 11692:0:(rw.c:128:ll_cl_init()) husk1: [0x280000f70:0x11c59:0x0] no active IO, please file a ticket. Pid: 11692, comm: ksh_so_hack.bin Trace: [&lt;ffffffff81005eb9&gt;] try_stack_unwind+0x169/0x1b0 [&lt;ffffffff81004919&gt;] dump_trace+0x89/0x450 [&lt;ffffffffa02158d7&gt;] libcfs_debug_dumpstack+0x57/0x80 [libcfs] [&lt;ffffffffa07f33ae&gt;] ll_cl_init+0x21e/0x320 [lustre] [&lt;ffffffffa07f34f8&gt;] ll_readpage+0x48/0x1b0 [lustre] [&lt;ffffffff81106418&gt;] __do_page_cache_readahead+0x1e8/0x260 [&lt;ffffffff81106538&gt;] force_page_cache_readahead+0x78/0xa0 [&lt;ffffffff810ff30d&gt;] sys_fadvise64_64+0xdd/0x230 [&lt;ffffffff810ff46e&gt;] sys_fadvise64+0xe/0x10 [&lt;ffffffff8145376b&gt;] system_call_fastpath+0x16/0x1b [&lt;00002aaaaaac11bd&gt;] 0x2aaaaaac11bd </pre> </div></div> <p>Also see these messages an hour prior to those above (in case there's a relationship):</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>LustreError: 4943:0:(mdc_request.c:1580:mdc_read_page()) husk1-MDT0000-mdc-ffff88044bc53800: read cache page: [0x280000f14:0x4:0x0] at 4753935872275117037: rc -5 LustreError: 5003:0:(mdc_request.c:1580:mdc_read_page()) husk1-MDT0000-mdc-ffff88044bc53800: read cache page: [0x280000f14:0x7:0x0] at 4753935872275117037: rc -5 LustreError: 5984:0:(mdc_request.c:1580:mdc_read_page()) husk1-MDT0000-mdc-ffff88044bc53800: read cache page: [0x280000f14:0x17fe9:0x0] at 6497832999440693922: rc -5 </pre> </div></div> <p>Attaching log file. Dump is available if you want it.</p> Master on SLES11 SP3 LU-4717

(rw.c:128:ll_cl_init()) husk1: [0x280000f70:0x11c59:0x0] no active IO, please file a ticket.

Bug Critical Resolved Fixed Zhenyu Xu Ann Koehler Wed, 5 Mar 2014 20:26:27 +0000 Wed, 5 Apr 2017 17:18:40 +0000 Thu, 8 May 2014 13:19:10 +0000 Lustre 2.6.0 Lustre 2.6.0 0 9 <p>Bobijam</p> <p>Could you please look into this one?</p> <p>Thanks</p> <p>Peter</p> <p>I've uploaded the kdumps of 2 nodes that exhibited this bug to:</p> <p>ftp.whamcloud.com:/uploads/<a href="https://jira.whamcloud.com/browse/LU-4717" title="(rw.c:128:ll_cl_init()) husk1: [0x280000f70:0x11c59:0x0] no active IO, please file a ticket." class="issue-link" data-issue-key="LU-4717"><del>LU-4717</del></a>/LU4717_no_active_io.tgz</p> <p>I'm not sure how much they will help. The processes issuing the errors had terminated by the time the dumps were taken, but I'm passing them along in case there might be cached data structures with useful info.</p> <p>please try patch <a href="http://review.whamcloud.com/9658" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/9658</a></p> <p>The patch is scheduled for testing this week. Will let you know the results when available.</p> <p>Is the testing of Change, 9658 still in progress or have you gotten results yet?</p> <p>Testing is still in progress.</p> <p>Finished testing this afternoon. </p> <p>Looks as though this patch fixed the issue, after running IOSTRESS for 5 hours I haven't seen the issue. Previously it occurred with in an hour of running.</p> <p>Thanks Mark!</p> <p>Is it really necessary for Cray to have fadvise() support? I would like to return an error value in this case so that fadivse() will be actually disabled.</p> <p>One way or another this bug should get cleaned up. If fadvise won't be supported in CLIO then we should update the Ops Manual with a discussion about that in the API section. But wouldn't it be better in the long run to actually use input from fadvise() to make good decisions about what Linux should do with page cache, even if CLIO can't (currently) make better use of the advise?</p> <p>The major problem with fadvise() is that it doesn't have a callback for file system, therefore Lustre can only provide limited support.</p> <p>However, Lustre can easily support POSIX_FADV_WILLNEED and I believe this is the most frequent option for fadvise(). We can just check if a lock is already existing on the client side and if this is the case, we can read ahead pages as requested by fadvise(). How does this sound?</p> <p>That sounds ok to me. I guess we'll have to wait for more use and exposure to see what the application writers will want. We can track those needs in new tickets.</p> <p>We've hit this error again, repeatedly, running Sanity - test 54c this time against <a href="https://jira.whamcloud.com/browse/LU-3321" title="2.x single thread/process throughput degraded from 1.8" class="issue-link" data-issue-key="LU-3321"><del>LU-3321</del></a> built into our 2.6 branch. Patch <a href="http://review.whamcloud.com/9658" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/9658</a> was not yet in our 2.6 build. </p> <p>We had not seen this error until recently, are there changes that are bringing this to light more? <br/> In test 54c this fails when attempting to mount the loop device created, with the following in dmesgs:<br/> Buffer I/O error on device loop3, logical block 0<br/> lost page write due to I/O error on loop3</p> <p>in log file: <br/> mount: wrong fs type, bad option, bad superblock on /tmp/dal/loop54c,<br/> missing codepage or helper program, or other error<br/> In some cases useful info is found in syslog - try<br/> dmesg | tail or so</p> <p>Patch landed to Master. Please reopen ticket if more work is needed</p> Related Development Rank 1|hzwgun: Rank (Obsolete) 12967 Severity