https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-9154 Lustre <p>I have one nodemapA and set fileset by "lctl set_param -P xxxx".<br/> The nodemapA fileset in MDS will be clear to empty when do <br/> "lctl nodemap_add_range --name nodemapA --range xxx" or "lctl nodemap_add nodemapB"<br/> But the nodemapA fileset still exist in MGS. Does anyone have the same problem? Thanks.</p> Centos 7.2.3.10.0-327.22.2.el7.x86_64<br/> Lustre 2.8.59_25_g40e3986<br/> ZFS 0.6.5.7 LU-9154

MDS Nodemap filset will be clear to empty after add new nodemap

Bug Minor Resolved Duplicate Emoly Liu sebg-crd-pm Fri, 24 Feb 2017 09:28:35 +0000 Thu, 1 Mar 2018 18:39:36 +0000 Thu, 1 Mar 2018 18:39:36 +0000 0 4 <p>Emoly</p> <p>Could you please assist with this issue?</p> <p>Thanks</p> <p>Peter</p> <p>Does anyone have the same issue?</p> <p>Could you show me your commands in details? Thanks.</p> <p><span class="error">&#91;test script &#93;</span><br/> echo "create nodemapA"<br/> pdsh -S -Rssh -w mgs lctl nodemap_add testnodemapA<br/> sleep 10<br/> echo "check mds nodemapA"<br/> pdsh -S -Rssh -w mds lctl get_param nodemap.testnodemapA <br/> echo "set nodemapA fileset"<br/> pdsh -S -Rssh -w mgs lctl set_param -P nodemap.testnodemapA.fileset=/testnodemapA<br/> sleep 10<br/> echo "check mgs nodemapA fileset"<br/> pdsh -S -Rssh -w mgs lctl get_param nodemap.testnodemapA.* | grep fileset<br/> echo "check mds nodemapA fileset"<br/> pdsh -S -Rssh -w mds lctl get_param nodemap.testnodemapA.* | grep fileset<br/> echo "create nodemapB"<br/> pdsh -S -Rssh -w mgs lctl nodemap_add testnodemapB<br/> sleep 10<br/> echo "check mgs nodemapA fileset"<br/> pdsh -S -Rssh -w mgs lctl get_param nodemap.testnodemapA.* | grep fileset<br/> echo "check mds nodemapA fileset"<br/> pdsh -S -Rssh -w mds lctl get_param nodemap.testnodemapA.* | grep fileset</p> <p><span class="error">&#91;result&#93;</span><br/> create nodemapA<br/> check mds nodemapA<br/> set nodemapA fileset<br/> check mgs nodemapA fileset<br/> mgs: nodemap.testnodemapA.fileset=/testnodemapA<br/> check mds nodemapA fileset<br/> mds: nodemap.testnodemapA.fileset=/testnodemapA<br/> create nodemapB<br/> check mgs nodemapA fileset<br/> mgs: nodemap.testnodemapA.fileset=/testnodemapA<br/> check mds nodemapA fileset<br/> mds: nodemap.testnodemapA.fileset=</p> <p>=&gt;the testnodemapA.fileset will be clear to empty </p> <p>Here is my local output:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>[root@centos6-2 tests]# sh lu-9154.sh create nodemapA check mds nodemapA set nodemapA fileset check mgs nodemapA fileset nodemap.testnodemapA.fileset=/testnodemapA check mds nodemapA fileset nodemap.testnodemapA.fileset=/testnodemapA create nodemapB check mgs nodemapA fileset nodemap.testnodemapA.fileset=/testnodemapA check mds nodemapA fileset nodemap.testnodemapA.fileset=/testnodemapA </pre> </div></div> <p>"testnodemapA" is still there. I will try with multiple nodes and the same lustre version to yours(Lustre 2.8.59_25_g40e3986) later.</p> <p>Just did a test on multiple nodes. The output of my first run is</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>[root@trevis-73vm6 tests]# sh lu-9154.patch create nodemapA check mds nodemapA set nodemapA fileset check mgs nodemapA fileset trevis-73vm3: nodemap.testnodemapA.fileset= check mds nodemapA fileset trevis-73vm4: nodemap.testnodemapA.fileset=/testnodemapA create nodemapB check mgs nodemapA fileset trevis-73vm3: nodemap.testnodemapA.fileset= check mds nodemapA fileset trevis-73vm4: nodemap.testnodemapA.fileset= </pre> </div></div> <p>Then, I found the patch <a href="http://review.whamcloud.com/18783" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/18783</a>, which introduced fileset to nodemap, the description says:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>commit 25420c75e45a91c6131d52a674869d6d30ab36bf Author: Sebastien Buisson &lt;sbuisson@ddn.com&gt; Date: Fri Feb 5 16:44:40 2016 +0100 LU-7846 nodemap: add fileset info to nodemap Give the ability to set a fileset information on a nodemap. When set, a client pertaining to this nodemap will be allowed to mount only the subdirectory represented by this fileset. Fileset info of nodemap is stored permanently by using 'lctl set_param -P' commands. It means MDS and OSS will be able to retrieve fileset info of nodemap after a restart (when they read params llog). MGS will not see fileset info after restart, but this does not prevent the feature from working. </pre> </div></div> <p>After I restarted mds, I can see fileset info there:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>[root@trevis-73vm4 ~]# umount /mnt/lustre-mds1 [root@trevis-73vm4 ~]# mount -t lustre -o loop /tmp/lustre-mdt1 /mnt/lustre-mds1 [root@trevis-73vm4 ~]# lctl get_param nodemap.testnodemapA.fileset nodemap.testnodemapA.fileset=/testnodemapA </pre> </div></div> <p>Can you have a try to restart your mds and see if fileset info is there?</p> <p>I have tried to restart mds then get nodemap.testnodemapA.fileset=/testnodemapA after mds started.</p> <p>Emoly, Could you tell me what's your lustre version (test nodemap script)? Thanks.</p> <p>I used 2.9.53 in my local test and 2.9.0 in multiple nodes test.</p> <p>I have test 2.9.0 and it also have the same problem.<br/> Did you set nodemap.active ? I test it with "lctl set_param nodemap.active=1"</p> <p>Emoly,</p> <p>I have set lctl set_param debug=-1 then test it again.<br/> Thereis somethiong wrong. (This log get form mds. ) Should I get something else?</p> <p>Jun 16 22:01:28 mdsb2 kernel: Lustre: 31848:0:(nodemap_handler.c:1020:nodemap_create()) adding nodemap 'nodemap1' to config without default nodemap<br/> Jun 16 22:01:28 mdsb2 kernel: Lustre: 31848:0:(nodemap_handler.c:1020:nodemap_create()) Skipped 1 previous similar message<br/> Jun 16 22:05:01 mdsb2 kernel: LustreError: 32302:0:(nodemap_storage.c:404:nodemap_idx_nodemap_update()) cannot add nodemap config to non-existing MGS.</p> <p>If possible, please post all the steps you did to reproduce this issue. Thanks.</p> <p> <span class="nobr"><a href="https://jira.whamcloud.com/secure/attachment/27061/27061_nodemap-LU9154.sh" title="nodemap-LU9154.sh attached to LU-9154">nodemap-LU9154.sh^{<img class="rendericon" src="https://jira.whamcloud.com/images/icons/link_attachment_7.gif" height="7" width="7" align="absmiddle" alt="" border="0"/>}</a></span> <span class="nobr"><a href="https://jira.whamcloud.com/secure/attachment/27062/27062_build-LU9154.sh" title="build-LU9154.sh attached to LU-9154">build-LU9154.sh^{<img class="rendericon" src="https://jira.whamcloud.com/images/icons/link_attachment_7.gif" height="7" width="7" align="absmiddle" alt="" border="0"/>}</a></span> <br/> Hi Emoly,<br/> I simplified the my build lustre steps and nodemap test scripts.<br/> The issue can be reprodced in my site only execute these 2 scripts. </p> <p>build-LU9154.sh <br/> create 3 nodes lustre system(based on zfs) , mgs:mdsb1, mds:mdsb2, oss:ossb1<br/> nodemap-LU915.sh <br/> create nodemapA( set fileset) then create nodemapB =&gt;mds nodemapA fileset missing </p> <p>Hi Emoly,<br/> Is it ok to reproduce this issue by build-LU9154.sh/nodemap-LU915.sh ?</p> <p><tt>I am seeing this issue in production with a Lustre 2.10.x system</tt></p> <p><tt>ON MDS0: lctl set_param -P nodemap.test.fileset='/filesystem/test/path'</tt></p> <p><tt>ON ADMIN NODE: cexec -p mds: 'lctl get_param nodemap.test.fileset'</tt><br/> <tt>mds0: }}{{nodemap.test.fileset=/filesystem/test/path</tt><br/> <tt>mds1: }}{{nodemap.test.fileset=/filesystem/test/path</tt><br/> <tt>mds2: }}{{nodemap.test.fileset=/filesystem/test/path</tt><br/> <tt>mds3: }}{{nodemap.test.fileset=/filesystem/test/path</tt></p> <p>ON ADMIN NODE: <tt>cexec -p oss: 'lctl get_param nodemap.test.fileset'</tt><br/> <tt>oss0: }}{{nodemap.test.fileset=/filesystem/test/path</tt><br/> <tt>oss1: }}{{nodemap.test.fileset=/filesystem/test/path</tt><br/> .............</p> <p>ON MDS0: lctl set_param nodemap.active=1</p> <p><tt>ON ADMIN NODE: cexec -p mds: 'lctl get_param nodemap.test.fileset'</tt><br/> <tt>mds0: }}{{nodemap.test.fileset=</tt><br/> <tt>mds1: }}{{nodemap.test.fileset=</tt><br/> <tt>mds2: }}{{nodemap.test.fileset=</tt><br/> <tt>mds3: }}{{nodemap.test.fileset=</tt></p> <p>ON ADMIN NODE: <tt>cexec -p oss: 'lctl get_param nodemap.test.fileset'</tt><br/> <tt>oss0: }}{{nodemap.test.fileset=</tt><br/> <tt>oss1: }}{{nodemap.test.fileset=</tt><br/> .............</p> <p>Hope this helps.</p> <p>It looks like its quite broken.</p> <p>Also I'm about to lodge another case because setting fileset twice on the same nodemap kills the MGS.</p> <p>We found that because we tried to enable nodemaps, it cleared the fileset value and we tried setting it again.</p> <p>Hi <a href="https://jira.whamcloud.com/secure/ViewProfile.jspa?name=mhaakddn" class="user-hover" rel="mhaakddn">mhaakddn</a> ,</p> <p>*Is there *any further update?</p> <p>This issue is a duplicate of <a href="https://jira.whamcloud.com/browse/LU-10703" title="All mds Nodemap filset will be cleared when do some nodemap operations (lustre 2.10.3)" class="issue-link" data-issue-key="LU-10703"><del>LU-10703</del></a>. I created a patch at <a href="https://review.whamcloud.com/31450" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/31450</a> .</p> <p>Close as a duplicate of <a href="https://jira.whamcloud.com/browse/LU-10703" title="All mds Nodemap filset will be cleared when do some nodemap operations (lustre 2.10.3)" class="issue-link" data-issue-key="LU-10703"><del>LU-10703</del></a>, since that has a patch.</p> Duplicate LU-10703 Development Rank 1|hzz4uf: Rank (Obsolete) 9223372036854775807 Severity