Whamcloud Community JIRA

Whamcloud Community JIRA https://jira.whamcloud.com This file is an XML representation of an issue en-us

9.4.14

940014

05-12-2023

[LU-4647] Add idmapping functions for nodemap https://jira.whamcloud.com/browse/LU-4647 Lustre LU-4647

Add idmapping functions for nodemap

Technical task LU-3291 Critical Resolved Fixed Nathaniel Clark Joshua Walgenbach HB patch Tue, 18 Feb 2014 18:14:18 +0000 Fri, 28 Aug 2015 17:49:00 +0000 Fri, 28 Aug 2015 17:49:00 +0000 Lustre 2.6.0 Lustre 2.8.0 0 10 <a href="http://review.whamcloud.com/9299" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/9299</a> <a href="http://review.whamcloud.com/#/c/9529/" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#/c/9529/</a> I noticed <a href="http://review.whamcloud.com/7936" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/7936</a> coming down the pipe as well, and while the client-side changes obviously don't affect this feature, it will need some idmap handling on the server side. Added tests here: <a href="http://review.whamcloud.com/10406" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/10406</a> I was doing some testing today and ran into a weird issue. Basically runas doesn't seem to work unless there is a supplemental group (it doesn't matter what it is): <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java">[root@c1 mnt]# /root/lustre-release/lustre/tests/runas -u 503 touch `pwd`/tmp_dir/file_test running as uid/gid/euid/egid 503/503/503/503, groups: [touch] [/mnt/tmp_dir/file_test] touch: cannot touch `/mnt/tmp_dir/file_test': Permission denied [root@c1 mnt]# /root/lustre-release/lustre/tests/runas -u 503 -G99999 touch `pwd`/tmp_dir/file_test adding supp group 99999 running as uid/gid/euid/egid 503/503/503/503, groups: 99999 [touch] [/mnt/tmp_dir/file_test] </pre> </div></div> I wanted to check in to see if anyone else can reproduce it, or if it's something about my build/system. Also if anyone has any theories as to why it's happening. Thanks, Kit This should probably be a separate bug. Kit, can you open your issue as a new bug, please? Kit, your patch <a href="http://review.whamcloud.com/#/c/10406/" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#/c/10406/</a> is failing testing, can you address the issues? Hi Cliff, I'm not sure what's going on. It seems like it's having a problem creating a file as root. Do these nodes run with root squash on or anything? Is there a way to get interactive access to the test or is it batch only? I can't reproduce the issue on my test machine. I think I figured it out - the earlier tests were enabling nodemap and the new tests expect it to be disabled at start. I uploaded a fix for it. There still seems to be a problem with the tests, but it's failing on other non-nodemap tests as well. They are all getting timeout errors. Is it possible that there is a problem with the test cluster? Looking at it again, it looks like the OSS kernel panic'd: 22:04:24:IP: [<ffffffffa08882e5>] tgt_request_preprocess+0x705/0x1470 [ptlrpc] I dug into it some, and it looks like the clients aren't being reassigned a nodemap after the nodemap_destroy, so they are just pointing to NULL nodemaps. Josh is working on an updated patch and will resubmit it. Any updates on this ? The last patch also failed testing. Found a new bug while working on 10406: <a href="http://review.whamcloud.com/#/c/12575/" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#/c/12575/</a> I hope Intel will block to close that ticket until oxygen comments will don't added to the code with good debug options. It's very strange to land code without ANY debug in code. Why Intel allow to do it? did Intel do any testing? recovery-small failed to stop any service with export reference leak with simple test SLOW=yes sh recovery-small.sh. Alex, this is in reference to <a href="http://review.whamcloud.com/#/c/12575/" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#/c/12575/</a> ? Cliff, I don't know a patch pointed you. my patch in <a href="https://jira.whamcloud.com/browse/LU-5963" title="wrong hashing in LU-4647" class="issue-link" data-issue-key="LU-5963"><del>LU-5963</del></a> very simple. <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java">@@ -50,7 +52,12 @@ void nm_member_putref(struct obd_export *exp) static __u32 nm_member_hashfn(cfs_hash_t *hash_body, const void *key, unsigned mask) { - return cfs_hash_djb2_hash(key, strlen(key), mask); + unsigned long hash; +// return cfs_hash_djb2_hash(key, strlen(key), mask); + hash = hash_long((unsigned long)key, hash_body->hs_bkt_bits) & mask; + + CDEBUG(D_INFO, "hashing %p -> %lu\n", key, hash); + return hash; } </pre> </div></div> but some debug need added anyway. Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="http://review.whamcloud.com/12575/" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/12575/</a> Subject: <a href="https://jira.whamcloud.com/browse/LU-4647" title="Add idmapping functions for nodemap" class="issue-link" data-issue-key="LU-4647"><del>LU-4647</del></a> nodemap: fix problem with node reclassification Project: fs/lustre-release Branch: master Current Patch Set: Commit: ec6aae2e3aa4174e3c8fdd8e75a9a4186e3abdc4 Patch landed to Master. I think the recently landed patch is not addressing the issue that Alexey pointed out. Sorry, I see that the hash function patch landed via <a href="http://review.whamcloud.com/12881" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/12881</a> from <a href="https://jira.whamcloud.com/browse/LU-5963" title="wrong hashing in LU-4647" class="issue-link" data-issue-key="LU-5963"><del>LU-5963</del></a>. Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="http://review.whamcloud.com/10406/" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/10406/</a> Subject: <a href="https://jira.whamcloud.com/browse/LU-4647" title="Add idmapping functions for nodemap" class="issue-link" data-issue-key="LU-4647"><del>LU-4647</del></a> nodemap: add tests to sanity-sec for nodemap mapping Project: fs/lustre-release Branch: master Current Patch Set: Commit: 50f95dd6e5159981f6b950d2fab674955442603b Andreas Dilger (andreas.dilger@intel.com) uploaded a new patch: <a href="http://review.whamcloud.com/13028" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/13028</a> Subject: <a href="https://jira.whamcloud.com/browse/LU-4647" title="Add idmapping functions for nodemap" class="issue-link" data-issue-key="LU-4647"><del>LU-4647</del></a> tests: enable sanity-sec for DNE mode Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 44d7be42028c31c49c9929053ea77adae3abde8e Kit Westneat (kit.westneat@gmail.com) uploaded a new patch: <a href="http://review.whamcloud.com/13343" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/13343</a> Subject: <a href="https://jira.whamcloud.com/browse/LU-4647" title="Add idmapping functions for nodemap" class="issue-link" data-issue-key="LU-4647"><del>LU-4647</del></a> tests: modify sanity-sec to test multiple MDSes Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: e2de239ed18bd9a346e5c4deb1e2762f6d6b07d0 Reopening based on comments and new patches Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="http://review.whamcloud.com/13478/" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/13478/</a> Subject: <a href="https://jira.whamcloud.com/browse/LU-4647" title="Add idmapping functions for nodemap" class="issue-link" data-issue-key="LU-4647"><del>LU-4647</del></a> lctl: add nodemap man pages to lctl Project: fs/lustre-release Branch: master Current Patch Set: Commit: e477f21ed67bfb76aba2b7cede8f62cc0cbd2ff7 Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="http://review.whamcloud.com/13343/" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/13343/</a> Subject: <a href="https://jira.whamcloud.com/browse/LU-4647" title="Add idmapping functions for nodemap" class="issue-link" data-issue-key="LU-4647"><del>LU-4647</del></a> tests: modify sanity-sec to test multiple MDSes Project: fs/lustre-release Branch: master Current Patch Set: Commit: e62ce6e1252b33b37130556f31ceb22e8410dfcb Andreas Dilger (andreas.dilger@intel.com) uploaded a new patch: <a href="http://review.whamcloud.com/14769" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/14769</a> Subject: <a href="https://jira.whamcloud.com/browse/LU-4647" title="Add idmapping functions for nodemap" class="issue-link" data-issue-key="LU-4647"><del>LU-4647</del></a> tests: clean up sanity-sec code style issues Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: d4240b487f7b16ac69f46452192bffdc475d6aae Dmitry Eremin (dmitry.eremin@intel.com) uploaded a new patch: <a href="http://review.whamcloud.com/14933" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/14933</a> Subject: <a href="https://jira.whamcloud.com/browse/LU-4647" title="Add idmapping functions for nodemap" class="issue-link" data-issue-key="LU-4647"><del>LU-4647</del></a> tests: properly cleanup after test failure Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: af43e0b4c60f7e7d1ab7202bfab66785fe6edd52 Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="http://review.whamcloud.com/14933/" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/14933/</a> Subject: <a href="https://jira.whamcloud.com/browse/LU-4647" title="Add idmapping functions for nodemap" class="issue-link" data-issue-key="LU-4647"><del>LU-4647</del></a> tests: properly cleanup after test failure Project: fs/lustre-release Branch: master Current Patch Set: Commit: c19992eb6809170acf885f1895fec628982b05ad Andreas Dilger (andreas.dilger@intel.com) merged in patch <a href="http://review.whamcloud.com/14769/" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/14769/</a> Subject: <a href="https://jira.whamcloud.com/browse/LU-4647" title="Add idmapping functions for nodemap" class="issue-link" data-issue-key="LU-4647"><del>LU-4647</del></a> tests: clean up sanity-sec code style issues Project: fs/lustre-release Branch: master Current Patch Set: Commit: 4fab3b2dfd4225f096e9a38ac14bcf61c41a81d4 Landed for 2.8 Related LU-5963 LU-1882 LUDOC-265 Development Rank 1|hzwfcn: Rank (Obsolete) 12712