https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-8340 Lustre <p>This issue was created by maloo for Saurabh Tandan &lt;saurabh.tandan@intel.com&gt;</p> <p>This issue relates to the following test suite run: <a href="https://testing.hpdd.intel.com/test_sets/12ba69e2-3a6a-11e6-a0ce-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/12ba69e2-3a6a-11e6-a0ce-5254006e85c2</a>.</p> <p>The sub-test test_25 failed with the following error:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>/usr/bin/lfs setquota -u quota_usr -b 13761540 -B 14449617 -i 916168 -I 961976 /mnt/lustre FAILED! </pre> </div></div> <p>test log:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>Started clients trevis-48vm1.trevis.hpdd.intel.com,trevis-48vm2: CMD: trevis-48vm1.trevis.hpdd.intel.com,trevis-48vm2 mount | grep /mnt/lustre' ' 10.9.6.52@tcp:/lustre on /mnt/lustre type lustre (rw,flock,user_xattr) 10.9.6.52@tcp:/lustre on /mnt/lustre type lustre (rw,flock,user_xattr) CMD: trevis-48vm2 PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/usr/lib64/lustre/tests//usr/lib64/lustre/tests:/usr/lib64/lustre/tests:/usr/lib64/lustre/tests//usr/lib64/lustre/tests/../utils:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/utils/gss:/usr/lib64/lustre/utils:/usr/lib64/qt-3.3/bin:/usr/lib64/compat-openmpi16/bin:/usr/bin:/bin:/usr/sbin:/sbin::/sbin:/sbin:/bin:/usr/sbin: NAME=autotest_config sh rpc.sh set_default_debug \"vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck\" \"all -lnet -lnd -pinger\" 4 CMD: trevis-48vm7 lctl get_param -n timeout Using TIMEOUT=20 CMD: trevis-48vm7 lctl dl | grep ' IN osc ' 2&gt;/dev/null | wc -l CMD: trevis-48vm1.trevis.hpdd.intel.com lctl dl | grep ' IN osc ' 2&gt;/dev/null | wc -l enable quota as required CMD: trevis-48vm7 /usr/sbin/lctl get_param -n osd-ldiskfs.lustre-MDT0000.quota_slave.enabled CMD: trevis-48vm8 /usr/sbin/lctl get_param -n osd-ldiskfs.lustre-OST0000.quota_slave.enabled [HOST:trevis-48vm1.trevis.hpdd.intel.com] [old_mdt_qtype:ug] [old_ost_qtype:ug] [new_qtype:ug3] CMD: trevis-48vm7 /usr/sbin/lctl conf_param lustre.quota.mdt=ug3 CMD: trevis-48vm7 /usr/sbin/lctl conf_param lustre.quota.ost=ug3 Total disk size: 13760516 block-softlimit: 13761540 block-hardlimit: 14449617 inode-softlimit: 916168 inode-hardlimit: 961976 Setting up quota on trevis-48vm1.trevis.hpdd.intel.com:/mnt/lustre for quota_usr... + /usr/bin/lfs setquota -u quota_usr -b 13761540 -B 14449617 -i 916168 -I 961976 /mnt/lustre setquota failed: Operation not permitted sanity-sec test_25: @@@@@@ FAIL: /usr/bin/lfs setquota -u quota_usr -b 13761540 -B 14449617 -i 916168 -I 961976 /mnt/lustre FAILED! </pre> </div></div> <p>This issue was first seen on 06/19/2016 and since then it has occurred around 22 times till now (i.e. in 10 days, 22 times).</p> Full - EL7.2 Server/EL7.2 Client<br/> master, build# 3399 LU-8340

sanity-sec test_25: /usr/bin/lfs setquota -u quota_usr -b 13761540 -B 14449617 -i 916168 -I 961976 /mnt/lustre FAILED!

Bug Critical Resolved Fixed Kit Westneat Maloo Tue, 28 Jun 2016 19:13:20 +0000 Wed, 5 Oct 2016 12:08:08 +0000 Wed, 5 Oct 2016 12:08:08 +0000 Lustre 2.9.0 Lustre 2.9.0 0 9 <p>The setquota failed for -EPERM, I think it's caused by nodemap permission check (nodemap_can_setquota()), but I don't see why the "nmf_allow_root_access" wasn't set properly in this case. Perhaps we need some nodemap expert to take a look.</p> <p>Hi Kit,</p> <p>Would you be able to look at this issue? Our engineer believes it is a nodemap permission check issue. Can you please advise?</p> <p>Thanks!<br/> Joe</p> <p>More failure instances on master branch:<br/> <a href="https://testing.hpdd.intel.com/test_sets/49a487de-541f-11e6-88a7-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/49a487de-541f-11e6-88a7-5254006e85c2</a><br/> <a href="https://testing.hpdd.intel.com/test_sets/808f2faa-524f-11e6-bf87-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/808f2faa-524f-11e6-bf87-5254006e85c2</a><br/> <a href="https://testing.hpdd.intel.com/test_sets/939e46d6-5037-11e6-bf87-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/939e46d6-5037-11e6-bf87-5254006e85c2</a></p> <p>Any update Kit? Do you think that this is something we ought to fix for 2.9 or could it wait for a future release?</p> <p>Hi Peter,</p> <p>I've taken a look at it, but I'm not sure why it fails sometimes and not others. My gut is that it's just a test issue, but I will spend some time trying to reproduce it on my system and get a clearer answer. I don't think it's a blocker, but it would be nice to have it sorted out before the next release.</p> <p>Kit</p> <p>Thanks for the update. Those kind of issues can be infuriating. Do you know how to leverage test parameters to easily run the test multiple times? Adding additional debug to the tests and doing that is usually how engineers manage to get to the bottom of such issues.</p> <p>Peter</p> <p>That's a good idea, I'll upload a debug patch as well with test params to see if I can reliably trigger it on maloo.</p> <p>Kit Westneat (kit.westneat@gmail.com) uploaded a new patch: <a href="http://review.whamcloud.com/21907" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/21907</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-8340" title="sanity-sec test_25: /usr/bin/lfs setquota -u quota_usr -b 13761540 -B 14449617 -i 916168 -I 961976 /mnt/lustre FAILED!" class="issue-link" data-issue-key="LU-8340"><del>LU-8340</del></a> nodemap: debug patch<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 07e836a33a3cd5613fcda80944cb41bad20cecec</p> <p>Another instance: Interop EL7.2 Server/2.8.0 EL7.2 Client<br/> Server: master, build# 3418<br/> Client: b2_8, build# 12<br/> <a href="https://testing.hpdd.intel.com/test_sets/77f2fecc-5f00-11e6-906c-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/77f2fecc-5f00-11e6-906c-5254006e85c2</a></p> <p>Kit</p> <p>Has the debug patch provided the information you hoped for?</p> <p>Peter</p> <p>Hi Peter,</p> <p>I haven't been able to reproduce the bug yet, I'll try some more this week.</p> <p>Thanks,<br/> Kit</p> <p>Andreas Dilger (andreas.dilger@intel.com) merged in patch <a href="http://review.whamcloud.com/21907/" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/21907/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-8340" title="sanity-sec test_25: /usr/bin/lfs setquota -u quota_usr -b 13761540 -B 14449617 -i 916168 -I 961976 /mnt/lustre FAILED!" class="issue-link" data-issue-key="LU-8340"><del>LU-8340</del></a> nodemap: debug patch<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 193987fbbb7595d29cec47f03f009f56caa90924</p> <p>Kit</p> <p>The debug patch has landed for over a week now. Have you been able to gather any useful info as a result?</p> <p>Peter</p> <p>Kit Westneat (kit.westneat@gmail.com) uploaded a new patch: <a href="http://review.whamcloud.com/22750" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/22750</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-8340" title="sanity-sec test_25: /usr/bin/lfs setquota -u quota_usr -b 13761540 -B 14449617 -i 916168 -I 961976 /mnt/lustre FAILED!" class="issue-link" data-issue-key="LU-8340"><del>LU-8340</del></a> tests: in sanity-sec 25 client nodemap must be admin<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 466a4add4b2bf92012920b0acff46745899547fd</p> <p>Hi Peter,</p> <p>I think it's an issue with the ENABLE_QUOTA flag being set. When it is set, restarting Lustre causes the client to issue quota commands, but quota permissions were disabled by nodemap at that spot. This patch reenables the admin permissions on the client, so that should fix it. </p> <p>Is it possible to confirm that ENABLE_QUOTA is disabled on the normal tests, but enabled on the full tests?</p> <p>Thanks,<br/> Kit</p> <blockquote><p>Is it possible to confirm that ENABLE_QUOTA is disabled on the normal tests, but enabled on the full tests?</p></blockquote> <p>Yes, Kit.</p> <p>For full test group, ENABLE_QUOTA=yes is specified. For review-* test groups, ENABLE_QUOTA is disabled.</p> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="http://review.whamcloud.com/22750/" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/22750/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-8340" title="sanity-sec test_25: /usr/bin/lfs setquota -u quota_usr -b 13761540 -B 14449617 -i 916168 -I 961976 /mnt/lustre FAILED!" class="issue-link" data-issue-key="LU-8340"><del>LU-8340</del></a> tests: in sanity-sec 25 client nodemap must be admin<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 1a92e40bd05c05820c902ce4ee738212a308d9e2</p> <p>Landed for 2.9</p> Related LU-3291 Development Rank 1|hzyg1z: Rank (Obsolete) 9223372036854775807 Severity