https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-7313 Lustre <p>Server 2.7.61<br/> Client 2.7.61</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre> 404.console.fre0304.log LustreError: 11377:0:(fld_request.c:489:fld_client_lookup()) ASSERTION( env != ((void *)0) ) failed: LustreError: 11377:0:(fld_request.c:489:fld_client_lookup()) LBUG stdout.log sanity-hsm test_404: @@@@@@ FAIL: request on 0x200000405:0x1:0x0 is not SUCCEED on mds1 Trace dump: = /usr/lib64/lustre/tests/test-framework.sh:4784:error_noexit() = /usr/lib64/lustre/tests/test-framework.sh:4815:error() = /usr/lib64/lustre/tests/sanity-hsm.sh:719:wait_request_state() = /usr/lib64/lustre/tests/sanity-hsm.sh:4495:test_404() = /usr/lib64/lustre/tests/test-framework.sh:5062:run_one() = /usr/lib64/lustre/tests/test-framework.sh:5099:run_one_logged() = /usr/lib64/lustre/tests/test-framework.sh:4916:run_test() = /usr/lib64/lustre/tests/sanity-hsm.sh:4505:main() Dumping lctl log to /tmp/test_logs/1445245656/sanity-hsm.test_404.*.1445245912.log fre0304: open /proc/sys/lnet/dump_kernel failed: No such file or directory fre0304: open(dump_kernel) failed: No such file or directory fre0304: Warning: Permanently added 'fre0303,192.168.103.3' (RSA) to the list of known hosts. fre0301: Warning: Permanently added 'fre0303,192.168.103.3' (RSA) to the list of known hosts. fre0302: Warning: Permanently added 'fre0303,192.168.103.3' (RSA) to the list of known hosts. FAIL 404 (227s) sanity-hsm: FAIL: test_404 request on 0x200000405:0x1:0x0 is not SUCCEED on mds1 Stopping clients: fre0303,fre0304 /mnt/lustre2 (opts:) Stopping client fre0303 /mnt/lustre2 opts: stderr.log Using TIMEOUT=20 running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d0_runas_test/f22269] excepting tests: 34 35 36 pdsh@fre0303: fre0304: ssh exited with exit code 1 == sanity-hsm test complete, duration 257 sec == 09:11:53 (1445245913) </pre> </div></div> 4 node setup ( MDS / OSS / 2 Clients), DNE, Single MDS<br/> MDSCOUNT=2 OSTCOUNT=2<br/> LU-7313

sanity-hsm test_404 test failed: LustreError: 11377:0:(fld_request.c:489:fld_client_lookup()) ASSERTION( env != ((void *)0) ) failed

Bug Minor Resolved Cannot Reproduce WC Triage parinay v kondekar Mon, 19 Oct 2015 11:49:59 +0000 Mon, 7 Dec 2015 13:45:52 +0000 Mon, 7 Dec 2015 13:45:52 +0000 Lustre 2.8.0 0 2 <p>Could you please provide the stack trace for the failing thread. I don't think we can use the vmcore unless we have the exact kernel build and modules available, but it isn't mentioned if you are using our build, or for which kernel/distro/arch it is. </p> <p>Are you doing anything different in your testing or configuration to trigger this? We haven't hit anything similar in our testing.</p> <p>My bad, apologies for incomplete info</p> <p>Here are the details</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>491 &lt;4&gt;Lustre: DEBUG MARKER: == sanity-hsm test 404: Inactive MDT does not block requests for active MDTs == 09:08:05 (1445245685) 492 &lt;4&gt;Lustre: setting import lustre-MDT0001_UUID INACTIVE by administrator request 493 &lt;4&gt;Lustre: Skipped 1 previous similar message 494 &lt;0&gt;LustreError: 11377:0:(fld_request.c:489:fld_client_lookup()) ASSERTION( env != ((void *)0) ) failed: 495 &lt;0&gt;LustreError: 11377:0:(fld_request.c:489:fld_client_lookup()) LBUG 496 &lt;4&gt;Pid: 11377, comm: lhsmtool_posix 497 &lt;4&gt; 498 &lt;4&gt;Call Trace: 499 &lt;4&gt; [&lt;ffffffffa02f3875&gt;] libcfs_debug_dumpstack+0x55/0x80 [libcfs] 500 &lt;4&gt; [&lt;ffffffffa02f3e77&gt;] lbug_with_loc+0x47/0xb0 [libcfs] 501 &lt;4&gt; [&lt;ffffffffa08abd8b&gt;] fld_client_lookup+0x47b/0x4e0 [fld] 502 &lt;4&gt; [&lt;ffffffffa08df2e1&gt;] lmv_fld_lookup+0xf1/0x440 [lmv] 503 &lt;4&gt; [&lt;ffffffffa08d9eda&gt;] lmv_iocontrol+0x11fa/0x3230 [lmv] 504 &lt;4&gt; [&lt;ffffffffa02f327b&gt;] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 505 &lt;4&gt; [&lt;ffffffffa02ff523&gt;] ? libcfs_debug_vmsg2+0x5e3/0xbe0 [libcfs] 506 &lt;4&gt; [&lt;ffffffff8116fe9c&gt;] ? __kmalloc+0x20c/0x220 507 &lt;4&gt; [&lt;ffffffffa09ce9bb&gt;] ll_fid2path+0x3fb/0x870 [lustre] 508 &lt;4&gt; [&lt;ffffffffa09b40fc&gt;] ll_dir_ioctl+0x135c/0x7440 [lustre] 509 &lt;4&gt; [&lt;ffffffffa0a0666c&gt;] ? ll_authorize_statahead+0x2c/0xc0 [lustre] 510 &lt;4&gt; [&lt;ffffffffa09cb823&gt;] ? ll_file_open+0x5b3/0xca0 [lustre] 511 &lt;4&gt; [&lt;ffffffffa063d740&gt;] ? ptlrpc_req_finished+0x10/0x20 [ptlrpc] 512 &lt;4&gt; [&lt;ffffffffa09c48bd&gt;] ? __ll_inode_revalidate+0x1bd/0xc60 [lustre] 513 &lt;4&gt; [&lt;ffffffff81196643&gt;] ? generic_permission+0x23/0xb0 514 &lt;4&gt; [&lt;ffffffffa09aeb40&gt;] ? ll_dir_open+0x0/0xf0 [lustre] 515 &lt;4&gt; [&lt;ffffffffa09aeb40&gt;] ? ll_dir_open+0x0/0xf0 [lustre] 516 &lt;4&gt; [&lt;ffffffff8118639f&gt;] ? __dentry_open+0x23f/0x360 517 &lt;4&gt; [&lt;ffffffff812284cf&gt;] ? security_inode_permission+0x1f/0x30 518 &lt;4&gt; [&lt;ffffffff811865d4&gt;] ? nameidata_to_filp+0x54/0x70 519 &lt;4&gt; [&lt;ffffffff8119c31a&gt;] ? do_filp_open+0x6ea/0xd20 520 &lt;4&gt; [&lt;ffffffff8104fa68&gt;] ? flush_tlb_others_ipi+0x128/0x130 521 &lt;4&gt; [&lt;ffffffff8119e972&gt;] vfs_ioctl+0x22/0xa0 522 &lt;4&gt; [&lt;ffffffff8119eb14&gt;] do_vfs_ioctl+0x84/0x580 523 &lt;4&gt; [&lt;ffffffff81196dd6&gt;] ? final_putname+0x26/0x50 524 &lt;4&gt; [&lt;ffffffff8119f091&gt;] sys_ioctl+0x81/0xa0 525 &lt;4&gt; [&lt;ffffffff810e202e&gt;] ? __audit_syscall_exit+0x25e/0x290 526 &lt;4&gt; [&lt;ffffffff8100b072&gt;] system_call_fastpath+0x16/0x1b 527 &lt;4&gt; </pre> </div></div> <ul class="alternate" type="square"> <li></li> </ul> <p>Lustre: Build Version: 2.7.61-gef63c03-PRISTINE-2.6.32-431.29.2.el6</p> <ul class="alternate" type="square"> <li>&gt; Are you doing anything different in your testing or configuration to trigger this? We haven't hit anything similar in our testing.<br/> Its 4 node, DNE setup.</li> </ul> <ul class="alternate" type="square"> <li>cmd line</li> </ul> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java">SLOW=YES NAME=ncli NETTYPE=tcp mds1_HOST=fre0301 MDSDEV1=/dev/vdb mds_HOST=fre0301 MDSDEV=/dev/vdb mds2_HOST=fre0301 MDSDEV2=/dev/vdc MDSCOUNT=2 ost1_HOST=fre0302 OSTDEV1=/dev/vdb ost2_HOST=fre0302 OSTDEV2=/dev/vdc OSTCOUNT=2 CLIENTS=fre0303 RCLIENTS=<span class="code-quote">"fre0304"</span> DIR=/mnt/lustre PDSH=<span class="code-quote">"/usr/bin/pdsh -R ssh -S -w "</span> ONLY=404 MDS_MOUNT_OPTS=<span class="code-quote">"-o rw,user_xattr"</span> OST_MOUNT_OPTS=<span class="code-quote">"-o user_xattr"</span> MDSSIZE=0 OSTSIZE=0 ENABLE_QUOTA=<span class="code-quote">"yes"</span> MDSJOURNALSIZE=<span class="code-quote">"22"</span> MAXFREE=<span class="code-quote">"1400000"</span> mdtest_nFiles=<span class="code-quote">"50000"</span> mdtest_iteration=<span class="code-quote">"5"</span> SHARED_DIRECTORY=<span class="code-quote">"/shared/fremont/test-results/xperior-custom/914<span class="code-comment">//quad3-quartet-1/shared-dir//sanity-hsm"</span> /usr/lib64/lustre/tests/sanity-hsm.sh 2&gt; /<span class="code-keyword">var</span>/log/xperior/test_stderr.166789.log 1&gt; /<span class="code-keyword">var</span>/log/xperior/test_stdout.166789.log</span> </pre> </div></div> <p>Hope this helps. Let me know, if anything more is needed.</p> <p>Just realized that the client was not running "patchless" client RPMs. Re-test (MULTIRUN=10) with patchless clients RPMS on client, passed the test. The issue can be closed.</p> <p>Thanks</p> <p>sorry ran a wrong test. sanity-hsm/test_404 rerun in progress. Ignore earlier comment. Thanks</p> <p>Its observed during re-run that, as the clients running with patchless client RPMs, the ASSERT "ASSERTION( env != ((void *)0) ) failed" is not reproducible. The issue can b closed.</p> <p>Thanks</p> Development Rank 1|hzxqrb: Rank (Obsolete) 9223372036854775807 Severity