<!-- 
RSS generated by JIRA (9.4.14#940014-sha1:734e6822bbf0d45eff9af51f82432957f73aa32c) at Sat Feb 10 03:00:37 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>Whamcloud Community JIRA</title>
    <link>https://jira.whamcloud.com</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.4.14</version>
        <build-number>940014</build-number>
        <build-date>05-12-2023</build-date>
    </build-info>


<item>
            <title>[LU-13360] getdents() against empty striped directory always returns 1365 dirents</title>
                <link>https://jira.whamcloud.com/browse/LU-13360</link>
                <project id="10000" key="LU">Lustre</project>
                    <description>&lt;p&gt;Created directory with &quot;lfs mkdir -c2 &amp;lt;thepath&amp;gt;&quot; on Lustre 2.10.8 based client.&lt;br/&gt;
May have attempted create subdirs or files (not sure, I&apos;d run a series of mdtest runs across multiple directories, and canceled via Ctrl-C partway through).&lt;br/&gt;
ls &amp;lt;thepath&amp;gt; appears to hang.&lt;br/&gt;
Running under strace shows that getdents() calls are always returning the same contents, never return EOF.&lt;/p&gt;

&lt;p&gt;No error messages are reported on the console of the servers or the client.&lt;/p&gt;

&lt;div class=&quot;preformatted panel&quot; style=&quot;border-width: 1px;&quot;&gt;&lt;div class=&quot;preformattedContent panelContent&quot;&gt;
&lt;pre&gt;[faaland1@opal174 branch:master src] $ps -f
UID        PID  PPID  C STIME TTY          TIME CMD
faaland1 52122 52121  0 09:58 pts/0    00:00:00 -bash
faaland1 59859 52122 99 10:16 pts/0    01:20:38 ls -l /p/lforge/faaland1/mdtest/mdt0 /p/lforge/faaland1/mdtest/mdt1 /p
faaland1 64327 52122  0 11:36 pts/0    00:00:00 ps -f
[faaland1@opal174 branch:master src] $strace -p 59859 2&amp;gt;&amp;amp;1 | head -n4
strace: Process 59859 attached
getdents(3, /* 1365 entries */, 32768)  = 32760
getdents(3, /* 1365 entries */, 32768)  = 32760
getdents(3, /* 1365 entries */, 32768)  = 32760
[faaland1@opal174 branch:master src] $ls -l /proc/59859/fd
total 0
lrwx------ 1 faaland1 faaland1 64 Mar 13 10:41 0 -&amp;gt; /dev/pts/0
lrwx------ 1 faaland1 faaland1 64 Mar 13 10:41 1 -&amp;gt; /dev/pts/0
lrwx------ 1 faaland1 faaland1 64 Mar 13 10:16 2 -&amp;gt; /dev/pts/0
lr-x------ 1 faaland1 faaland1 64 Mar 13 10:41 3 -&amp;gt; /p/lforge/faaland1/mdtest/mdtcount2
&lt;/pre&gt;
&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;The 1365 entries returned are always the same - 676 entries for &quot;..&quot; and 689 entries for &quot;.&quot;&lt;/p&gt;
&lt;div class=&quot;preformatted panel&quot; style=&quot;border-width: 1px;&quot;&gt;&lt;div class=&quot;preformattedContent panelContent&quot;&gt;
&lt;pre&gt;    676 {d_ino=144115272398143489, d_off=0, d_reclen=24, d_name=&quot;..&quot;, d_type=DT_DIR}
    689 {d_ino=144115339574197893, d_off=0, d_reclen=24, d_name=&quot;.&quot;, d_type=DT_DIR}
&lt;/pre&gt;
&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;I mounted the filesystem on another Lustre 2.10.8 node and ls of that directory produces the same symptoms.&lt;/p&gt;

&lt;p&gt;I mounted the file system on a Lustre 2.12.4 node and ls of that directory behaves as normal - getdents() is called twice, once it returns 2 entries, second time it returns 0 entries and 0 bytes (end of directory).&lt;/p&gt;</description>
                <environment>Client: opal174 with lustre-2.10.8_5.chaos-1.ch6.x86_64&lt;br/&gt;
Servers: zwicky with  lustre-2.12.4_2.chaos-1.ch6.x86_64&lt;br/&gt;
Both running TOSS 3.5-9rc1 based on RHEL 7.7&lt;br/&gt;
&lt;br/&gt;
Our Lustre tags are on github:&lt;br/&gt;
&lt;a href=&quot;https://github.com/LLNL/lustre/releases/tag/2.10.8_5.chaos&quot;&gt;https://github.com/LLNL/lustre/releases/tag/2.10.8_5.chaos&lt;/a&gt;&lt;br/&gt;
&lt;a href=&quot;https://github.com/LLNL/lustre/releases/tag/2.12.4_2.chaos&quot;&gt;https://github.com/LLNL/lustre/releases/tag/2.12.4_2.chaos&lt;/a&gt;&lt;br/&gt;
&lt;br/&gt;
zfs-0.7 based servers</environment>
        <key id="58364">LU-13360</key>
            <summary>getdents() against empty striped directory always returns 1365 dirents</summary>
                <type id="1" iconUrl="https://jira.whamcloud.com/secure/viewavatar?size=xsmall&amp;avatarId=11303&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.whamcloud.com/images/icons/priorities/major.svg">Major</priority>
                        <status id="5" iconUrl="https://jira.whamcloud.com/images/icons/statuses/resolved.png" description="A resolution has been taken, and it is awaiting verification by reporter. From here issues are either reopened, or are closed.">Resolved</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="2">Won&apos;t Fix</resolution>
                                        <assignee username="laisiyao">Lai Siyao</assignee>
                                    <reporter username="ofaaland">Olaf Faaland</reporter>
                        <labels>
                            <label>llnl</label>
                    </labels>
                <created>Fri, 13 Mar 2020 18:39:38 +0000</created>
                <updated>Tue, 16 Jun 2020 22:52:27 +0000</updated>
                            <resolved>Tue, 16 Jun 2020 22:52:19 +0000</resolved>
                                    <version>Lustre 2.10.8</version>
                    <version>Lustre 2.12.4</version>
                                                        <due></due>
                            <votes>0</votes>
                                    <watches>3</watches>
                                                                            <comments>
                            <comment id="265263" author="ofaaland" created="Fri, 13 Mar 2020 18:42:35 +0000"  >&lt;p&gt;For my tracking purposes, my internal ticket is TOSS4558&lt;/p&gt;</comment>
                            <comment id="265265" author="ofaaland" created="Fri, 13 Mar 2020 18:51:22 +0000"  >&lt;p&gt;Contents of the two FIDs reported by lfs getdirstripe, in case it helps.&lt;/p&gt;

&lt;p&gt;lfs getdirstripe:&lt;/p&gt;
&lt;div class=&quot;preformatted panel&quot; style=&quot;border-width: 1px;&quot;&gt;&lt;div class=&quot;preformattedContent panelContent&quot;&gt;
&lt;pre&gt;$lfs getdirstripe /p/lforge/faaland1/mdtest/mdtcount2
lmv_stripe_count: 2 lmv_stripe_offset: 0 lmv_hash_type: fnv_1a_64
mdtidx		 FID[seq:oid:ver]
     0		 [0x200002340:0x7:0x0]		
     1		 [0x280002340:0x7:0x0]
&lt;/pre&gt;
&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;MDT0:&lt;/p&gt;
&lt;div class=&quot;preformatted panel&quot; style=&quot;border-width: 1px;&quot;&gt;&lt;div class=&quot;preformattedContent panelContent&quot;&gt;
&lt;pre&gt;[root@zwicky1:toss4558]# find . -name &apos;*0x200002340:0x7:0x0*&apos; 2&amp;gt; /dev/null
./oi.23/0x200000417:0x1:0x0/mdtest/mdtcount2/[0x200002340:0x7:0x0]:0
./oi.64/0x200002340:0x7:0x0
[root@zwicky1:toss4558]# ls -l ./oi.64/0x200002340:0x7:0x0
ls: cannot access ./oi.64/0x200002340:0x7:0x0: Input/output error
[root@zwicky1:toss4558]# ls -al ./oi.23/0x200000417:0x1:0x0/mdtest/mdtcount2/[0x200002340:0x7:0x0]:0
total 22
drwx------ 2 faaland1 faaland1 2 Mar 13 10:10 .
drwx------ 4 faaland1 faaland1 2 Mar 13 10:10 ..
&lt;/pre&gt;
&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;MDT1:&lt;/p&gt;
&lt;div class=&quot;preformatted panel&quot; style=&quot;border-width: 1px;&quot;&gt;&lt;div class=&quot;preformattedContent panelContent&quot;&gt;
&lt;pre&gt;[root@zwicky2:toss4558]# find . -name &quot;*0x280002340:0x7:0x0*&quot; 2&amp;gt; /dev/null
./oi.64/0x280002340:0x7:0x0
./REMOTE_PARENT_DIR/0x280002340:0x7:0x0
[root@zwicky2:toss4558]# ls -al ./oi.64/0x280002340:0x7:0x0 ./REMOTE_PARENT_DIR/0x280002340:0x7:0x0
ls: cannot access ./REMOTE_PARENT_DIR/0x280002340:0x7:0x0: Input/output error
./oi.64/0x280002340:0x7:0x0:
total 22
drwx------ 2 faaland1 faaland1 2 Mar 13 10:10 .
drwxr-xr-x 0 root     root     0 Dec 31  1969 ..
&lt;/pre&gt;
&lt;/div&gt;&lt;/div&gt;</comment>
                            <comment id="265267" author="pjones" created="Fri, 13 Mar 2020 19:55:12 +0000"  >&lt;p&gt;Lai&lt;/p&gt;

&lt;p&gt;Could you please investigate&lt;/p&gt;

&lt;p&gt;Peter&lt;/p&gt;</comment>
                            <comment id="265273" author="ofaaland" created="Fri, 13 Mar 2020 21:23:26 +0000"  >&lt;p&gt;Hello Lai,&lt;/p&gt;

&lt;p&gt;I&apos;ve attached&#160;debug.toss-4558.a.tar.gz which contains debug logs from two clients.&#160; debug was +rpctrace and +vfstrace.&#160; An ls -al of the same striped directory was performed on each of the two nodes.&lt;/p&gt;

&lt;p&gt;opal64: Lustre 2.12, the ls is successful and shows &quot;.&quot; and &quot;..&quot; and exits.&lt;br/&gt;
opal174: Lustre 2.10, the ls is stuck in the loop calling getdents() as described above.&lt;/p&gt;</comment>
                            <comment id="265327" author="laisiyao" created="Mon, 16 Mar 2020 09:31:25 +0000"  >&lt;p&gt;2.12 contains several fixes for readdir of striped directory:&lt;br/&gt;
&lt;a href=&quot;https://review.whamcloud.com/#/c/27663&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/#/c/27663&lt;/a&gt;&lt;br/&gt;
&lt;a href=&quot;https://review.whamcloud.com/#/c/28548&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/#/c/28548&lt;/a&gt;&lt;br/&gt;
&lt;a href=&quot;https://review.whamcloud.com/#/c/32180&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/#/c/32180&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;You can apply them and try again. If you can enable &apos;trace&apos; in debuglog, it can help identify the exact cause, however IMO it should have been fixed by the above patches.&lt;/p&gt;</comment>
                            <comment id="271897" author="ofaaland" created="Wed, 3 Jun 2020 20:36:23 +0000"  >&lt;p&gt;Hi Lai,&lt;br/&gt;
I cherry-picked those patches and tried to push them to gerrit against b2_10 fortestonly, but I&apos;m being rejected in a way I haven&apos;t seen before.  I&apos;m not sure if there&apos;s something with my patch stack, or something else - maybe a permissions thing bubbling up with a non-obvious message.&lt;/p&gt;

&lt;p&gt;The patch stack is here:&lt;br/&gt;
&lt;a href=&quot;https://github.com/ofaaland/lustre/tree/b-toss-4558-stripedir&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/ofaaland/lustre/tree/b-toss-4558-stripedir&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;and the error I get is:&lt;/p&gt;
&lt;div class=&quot;preformatted panel&quot; style=&quot;border-width: 1px;&quot;&gt;&lt;div class=&quot;preformattedContent panelContent&quot;&gt;
&lt;pre&gt;[faaland1@oslic5 branch:b-toss-4558-stripedir lustre-210] $git push wcrev HEAD:refs/for/b2_10
Enter passphrase for key &apos;/g/g0/faaland1/.ssh/swdev&apos;: 
Counting objects: 126, done.
Delta compression using up to 36 threads.
Compressing objects: 100% (67/67), done.
Writing objects: 100% (95/95), 87.24 KiB | 0 bytes/s, done.
Total 95 (delta 69), reused 36 (delta 28)
remote: Resolving deltas: 100% (69/69)
remote: Processing changes: refs: 1, done    
To ssh://review.whamcloud.com/fs/lustre-release
 ! [remote rejected] HEAD -&amp;gt; refs/for/b2_10 (not Signed-off-by author/committer/uploader in commit message footer)
error: failed to push some refs to &apos;ssh://review.whamcloud.com/fs/lustre-release&apos;
&lt;/pre&gt;
&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;It looks straightforward, but I do not see patches without the Signed-off-by matching the author, and I also found that even just pushing a branch with just one commit produces the same result:&lt;br/&gt;
&lt;a href=&quot;https://github.com/ofaaland/lustre/tree/b-test-b210-push&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/ofaaland/lustre/tree/b-test-b210-push&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Can you either try pushing that branch for me, or help me troubleshoot this?  I don&apos;t know if the error message is coming from a script I can inspect to understand what&apos;s going on.  It doesn&apos;t seem to be from anything under contrib in lustre.&lt;/p&gt;

&lt;p&gt;thanks&lt;/p&gt;
</comment>
                            <comment id="271956" author="laisiyao" created="Thu, 4 Jun 2020 13:44:57 +0000"  >&lt;p&gt;I tried the first patch, and it looks working: &lt;a href=&quot;https://review.whamcloud.com/#/c/38826/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/#/c/38826/&lt;/a&gt;.&lt;/p&gt;</comment>
                            <comment id="272002" author="ofaaland" created="Thu, 4 Jun 2020 18:31:21 +0000"  >&lt;p&gt;Thanks Lai, that helped me figure it out.  Looks like it was the format of my commit messages.&lt;/p&gt;</comment>
                            <comment id="273043" author="ofaaland" created="Tue, 16 Jun 2020 22:50:01 +0000"  >&lt;p&gt;We&apos;re moving along on our updates to 2.12 that I&apos;m going to abandon this.  Thank you for your help, though.&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                            <attachment id="34454" name="debug.toss-4558.a.tar.gz" size="241431" author="ofaaland" created="Fri, 13 Mar 2020 21:20:20 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                                                                                                                                                            <customfield id="customfield_10890" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10390" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i00vfz:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10090" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                </customfields>
    </item>
</channel>
</rss>