https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-1668 Lustre <p>This issue was created by maloo for sarah &lt;sarah@whamcloud.com&gt;</p> <p>This issue relates to the following test suite run: <a href="https://maloo.whamcloud.com/test_sets/5b019fd0-d5c0-11e1-b078-52540035b04c" class="external-link" target="_blank" rel="nofollow noopener">https://maloo.whamcloud.com/test_sets/5b019fd0-d5c0-11e1-b078-52540035b04c</a>.</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>== conf-sanity test 53a: check OSS thread count params == 13:54:59 (1343076899) Loading modules from /usr/lib64/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' WARNING: Deprecated config file /etc/modprobe.conf, all config files belong into /etc/modprobe.d/. debug=-1 subsystem_debug=0xffb7efff ../lnet/lnet/lnet options: 'accept=all networks="o2ib0(ib0),tcp0(eth0)" accept_port=7988' WARNING: Deprecated config file /etc/modprobe.conf, all config files belong into /etc/modprobe.d/. WARNING: Deprecated config file /etc/modprobe.conf, all config files belong into /etc/modprobe.d/. WARNING: Deprecated config file /etc/modprobe.conf, all config files belong into /etc/modprobe.d/. WARNING: Deprecated config file /etc/modprobe.conf, all config files belong into /etc/modprobe.d/. WARNING: Deprecated config file /etc/modprobe.conf, all config files belong into /etc/modprobe.d/. gss/krb5 is not supported WARNING: Deprecated config file /etc/modprobe.conf, all config files belong into /etc/modprobe.d/. WARNING: Deprecated config file /etc/modprobe.conf, all config files belong into /etc/modprobe.d/. WARNING: Deprecated config file /etc/modprobe.conf, all config files belong into /etc/modprobe.d/. WARNING: Deprecated config file /etc/modprobe.conf, all config files belong into /etc/modprobe.d/. WARNING: Deprecated config file /etc/modprobe.conf, all config files belong into /etc/modprobe.d/. WARNING: Deprecated config file /etc/modprobe.conf, all config files belong into /etc/modprobe.d/. WARNING: Deprecated config file /etc/modprobe.conf, all config files belong into /etc/modprobe.d/. WARNING: Deprecated config file /etc/modprobe.conf, all config files belong into /etc/modprobe.d/. WARNING: Deprecated config file /etc/modprobe.conf, all config files belong into /etc/modprobe.d/. start mds service on client-3-ib Starting mds1: -o user_xattr,acl /dev/sda3 /mnt/mds1 Started lustre-MDT0000 start ost1 service on fat-amd-4-ib Starting ost1: /dev/sdb1 /mnt/ost1 Started lustre-OST0000 mount lustre on /mnt/lustre..... Starting client: client-4.lab.whamcloud.com: -o flock,user_xattr,acl client-3-ib@o2ib:/lustre /mnt/lustre setup single mount lustre success checking (($tstarted &amp;&amp; $tmin &amp;&amp; $tmax)) (((14 &amp;&amp; 12 &amp;&amp; 384)))... checking (($tstarted &gt;= $tmin &amp;&amp; $tstarted &lt;= $tmax )) (((14 &gt;= 12 &amp;&amp; 14 &lt;= 384 )))... ost.OSS.ost.threads_min=14 ost.OSS.ost.threads_max=382 checking (($tmin2 == ($tmin + $ncpts) &amp;&amp; $tmax2 == ($tmax - $ncpts))) (((12 == (12 + 2) &amp;&amp; 380 == (384 - 2))))... conf-sanity test_53a: @@@@@@ FAIL: Assertion 25 failed: (($tmin2 == ($tmin + $ncpts) &amp;&amp; $tmax2 == ($tmax - $ncpts))) (expanded: ((12 == (12 + 2) &amp;&amp; 380 == (384 - 2)))) Insane OST thread counts </pre> </div></div> lustre-master-tag-2.2.91 OFED build LU-1668

Test failure on test suite conf-sanity 53a

Bug Minor Resolved Fixed Liang Zhen Maloo Tue, 24 Jul 2012 15:35:00 +0000 Sun, 16 Feb 2014 09:14:53 +0000 Sun, 16 Feb 2014 09:14:53 +0000 Lustre 2.3.0 0 6 <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>thread_sanity() { local modname=$1 local facet=$2 local parampat=$3 local opts=$4 local tmin local tmin2 local tmax local tmax2 local tstarted local paramp local msg="Insane $modname thread counts" local ncpts=$(check_cpt_number) </pre> </div></div> <p>check_cpt_number is checking ncpts on localhost, but it really should be "do_fact $facet check_cpt_number", I will post a patch soon</p> <p>Patch is here: <a href="http://review.whamcloud.com/3595" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/3595</a></p> <p>My testing also has run into this bug. I'm still failing with the patch.</p> <p>53b (Side note - I'm running with 1024 mds threads by default)</p> <p>setup single mount lustre success<br/> checking (($tstarted &amp;&amp; $tmin &amp;&amp; $tmax)) (((1024 &amp;&amp; 1024 &amp;&amp; 1024)))...<br/> checking (($tstarted &gt;= $tmin &amp;&amp; $tstarted &lt;= $tmax )) (((1024 &gt;= 1024 &amp;&amp; 1024 &lt;= 1024 )))...<br/> barry-mds1: error: set_param: writing to file /proc/fs/lustre/mdt/lustre-MDT0000/mdt//threads_min: Numerical result out of range<br/> pdsh@spoon01: barry-mds1: ssh exited with exit code 1<br/> mdt.lustre-MDT0000.mdt..threads_min=1026<br/> barry-mds1: error: set_param: writing to file /proc/fs/lustre/mdt/lustre-MDT0000/mdt//threads_max: Numerical result out of range<br/> pdsh@spoon01: barry-mds1: ssh exited with exit code 1<br/> mdt.lustre-MDT0000.mdt..threads_max=1022<br/> checking (($tmin2 == ($tmin + $ncpts) &amp;&amp; $tmax2 == ($tmax - $ncpts))) (((1024 == (1024 + 2) &amp;&amp; 1024 == (1024 - 2))))...<br/> conf-sanity test_53b: @@@@@@ FAIL: Assertion 25 failed: (($tmin2 == ($tmin + $ncpts) &amp;&amp; $tmax2 == ($tmax - $ncpts))) (expanded: ((1024 == (1024 + 2) &amp;&amp; 1024 == (1024 - 2))))<br/> Insane MDT thread counts</p> <p>For 53a I get</p> <p>mount lustre on /lustre/barry.....<br/> Starting client: spoon01: -o user_xattr,flock 10.37.248.67@o2ib1:/lustre /lustre/barry<br/> setup single mount lustre success<br/> checking (($tstarted == $tmin &amp;&amp; $tstarted == $tmax )) (((9 == 6 &amp;&amp; 9 == 192 )))...<br/> conf-sanity test_53a: @@@@@@ FAIL: Assertion 28 failed: (($tstarted == $tmin &amp;&amp; $tstarted == $tmax )) (expanded: ((9 == 6 &amp;&amp; 9 == 192 )))<br/> Insane OST thread counts<br/> Trace dump:<br/> = ./../tests/test-framework.sh:3683:error_noexit()<br/> = ./../tests/test-framework.sh:3705:error()<br/> = ./../tests/functions.sh:120:lassert()<br/> = ./conf-sanity.sh:2620:thread_sanity()<br/> = ./conf-sanity.sh:2632:test_53a()</p> <p>another failure, client is running kernel 2.6.38-fc15</p> <p><a href="https://maloo.whamcloud.com/test_sets/b0206b50-e5c3-11e1-ae4e-52540035b04c" class="external-link" target="_blank" rel="nofollow noopener">https://maloo.whamcloud.com/test_sets/b0206b50-e5c3-11e1-ae4e-52540035b04c</a></p> <p>Looking at the data for 53a I'm getting</p> <p>tmin = 6 <br/> tmax = 192 <br/> tstarted 8</p> <p>Which causes the ASSERT to fail</p> <p>I found the reason for the failure for my testing. In /etc/modprobe.d/lustre.conf I have </p> <p>options mdt mds_num_threads=1024</p> <p>When I pound it out both test pass.</p> <p>hi James, then could you change the review flag? or you want to me to make more change to the patch so it can pass even with setting this parameter? I think this parameter is conflicting with the conf-sanity because conf-sanity needs to set this parameter as well.</p> <p>Yes could you arrange for the test to pass even in the case with setting this parameter.</p> <p>Okay I can change my review since your current patch cover most cases whereas my problem is a specific corner case. If I can change my review can you keep this ticket open until my problem is resolved?</p> <p>Dropping priority as patch covering most cases has been landed to master for 2.3</p> <p>another failure seen in interop testing between 2.1.3 server and 2.4 client: <a href="https://maloo.whamcloud.com/test_sets/351a6028-4a86-11e2-8a7b-52540035b04c" class="external-link" target="_blank" rel="nofollow noopener">https://maloo.whamcloud.com/test_sets/351a6028-4a86-11e2-8a7b-52540035b04c</a></p> <p>Lustre client: <a href="http://build.whamcloud.com/job/lustre-b2_3/41/" class="external-link" target="_blank" rel="nofollow noopener">http://build.whamcloud.com/job/lustre-b2_3/41/</a> (2.3.0)<br/> Lustre server: <a href="http://build.whamcloud.com/job/lustre-b2_4/69/" class="external-link" target="_blank" rel="nofollow noopener">http://build.whamcloud.com/job/lustre-b2_4/69/</a> (2.4.2 RC1)</p> <p>conf-sanity test 53b failed:<br/> <a href="https://maloo.whamcloud.com/test_sets/dbed7af0-685f-11e3-a16f-52540035b04c" class="external-link" target="_blank" rel="nofollow noopener">https://maloo.whamcloud.com/test_sets/dbed7af0-685f-11e3-a16f-52540035b04c</a></p> <p>I think this has already been fixed</p> Development Rank 1|hzv5vj: Rank (Obsolete) 4486 Severity