<!-- 
RSS generated by JIRA (9.4.14#940014-sha1:734e6822bbf0d45eff9af51f82432957f73aa32c) at Sat Feb 10 01:26:55 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>Whamcloud Community JIRA</title>
    <link>https://jira.whamcloud.com</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.4.14</version>
        <build-number>940014</build-number>
        <build-date>05-12-2023</build-date>
    </build-info>


<item>
            <title>[LU-2638] corruption of MDT &quot;..&quot; entry in some ldiskfs directories</title>
                <link>https://jira.whamcloud.com/browse/LU-2638</link>
                <project id="10000" key="LU">Lustre</project>
                    <description>&lt;p&gt;It appears that there is a systematic corruption of the &quot;..&quot; entry in the directory, possibly only affecting htree directories when the &quot;dirdata&quot; feature is enabled.&lt;/p&gt;

&lt;div class=&quot;preformatted panel&quot; style=&quot;border-width: 1px;&quot;&gt;&lt;div class=&quot;preformattedContent panelContent&quot;&gt;
&lt;pre&gt;e2fsck 1.42.6.wc2 (10-Dec-2012)
nbp1-MDT0000 has been mounted 121 times without being checked, check forced.
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
[...snip...]
Entry &apos;..&apos; in /ROOT/arosen2/vmcluster/nodrp/T20K_alpha1_prodrun/pltfiles/plt00000 (40930307) is duplicate &apos;..&apos; entry.
Fix? no
Entry &apos;..&apos; in /ROOT/arosen2/vmcluster/nodrp/T20K_alpha1_prodrun/pltfiles/plt00000 (40930307) is duplicate &apos;..&apos; entry.
Fix? no
Entry &apos;..&apos; in /ROOT/arosen2/vmcluster/nodrp/T20K_alpha1_prodrun/pltfiles/plt00000 (40930307) is a link to directory /ROOT/arosen2/vmcluster/nodrp/T20K_alpha1_prodrun/pltfiles (40919273).
Second entry &apos;Header&apos; (inode=40967687) in directory inode 40967686 should be &apos;..&apos;
Fix? no
Entry &apos;..&apos; in /ROOT/arosen2/vmcluster/nodrp/T20K_alpha1_prodrun/pltfiles/plt00033 (40967686) is duplicate &apos;..&apos; entry.
Fix? no
Entry &apos;..&apos; in /ROOT/arosen2/vmcluster/nodrp/T20K_alpha1_prodrun/pltfiles/plt00033 (40967686) is duplicate &apos;..&apos; entry.
Fix? no
Entry &apos;..&apos; in /ROOT/arosen2/vmcluster/nodrp/T20K_alpha1_prodrun/pltfiles/plt00033 (40967686) is a link to directory /ROOT/arosen2/vmcluster/nodrp/T20K_alpha1_prodrun/pltfiles (40919273).
Second entry &apos;Header&apos; (inode=40971040) in directory inode 40971039 should be &apos;..&apos;
Fix? no
Entry &apos;..&apos; in /ROOT/arosen2/vmcluster/nodrp/T20K_alpha1_prodrun/pltfiles/plt00029 (40971039) is duplicate &apos;..&apos; entry.
Fix? no
Entry &apos;..&apos; in /ROOT/arosen2/vmcluster/nodrp/T20K_alpha1_prodrun/pltfiles/plt00029 (40971039) is duplicate &apos;..&apos; entry.
Fix? no
Entry &apos;..&apos; in /ROOT/arosen2/vmcluster/nodrp/T20K_alpha1_prodrun/pltfiles/plt00029 (40971039) is a link to directory /ROOT/arosen2/vmcluster/nodrp/T20K_alpha1_prodrun/pltfiles (40919273).
Second entry &apos;Header&apos; (inode=44588784) in directory inode 44588782 should be &apos;..&apos;
Fix? no
Entry &apos;..&apos; in /ROOT/arosen2/vmcluster/nodrp/T20K_alpha1_prodrun/pltfiles/plt00036 (44588782) is duplicate &apos;..&apos; entry.
Fix? no
Entry &apos;..&apos; in /ROOT/arosen2/vmcluster/nodrp/T20K_alpha1_prodrun/pltfiles/plt00036 (44588782) is duplicate &apos;..&apos; entry.
Fix? no
Entry &apos;..&apos; in /ROOT/arosen2/vmcluster/nodrp/T20K_alpha1_prodrun/pltfiles/plt00036 (44588782) is a link to directory /ROOT/arosen2/vmcluster/nodrp/T20K_alpha1_prodrun/pltfiles (40919273).
Second entry &apos;Header&apos; (inode=47195750) in directory inode 47195749 should be &apos;..&apos;
Fix? no
[...snip...]
Pass 3: Checking directory connectivity
&apos;..&apos; in /ROOT/arosen2/vmcluster/nodrp/T20K_alpha1_prodrun/pltfiles/plt00000 (40930307) is &amp;lt;The NULL inode&amp;gt; (0), should be /ROOT/arosen2/vmcluster/nodrp/T20K_alpha1_prodrun/pltfiles (40919273).
Fix? no
&apos;..&apos; in /ROOT/arosen2/vmcluster/nodrp/T20K_alpha1_prodrun/pltfiles/plt00033 (40967686) is &amp;lt;The NULL inode&amp;gt; (0), should be /ROOT/arosen2/vmcluster/nodrp/T20K_alpha1_prodrun/pltfiles (40919273).
Fix? no
&apos;..&apos; in /ROOT/arosen2/vmcluster/nodrp/T20K_alpha1_prodrun/pltfiles/plt00029 (40971039) is &amp;lt;The NULL inode&amp;gt; (0), should be /ROOT/arosen2/vmcluster/nodrp/T20K_alpha1_prodrun/pltfiles (40919273).
Fix? no
&apos;..&apos; in /ROOT/arosen2/vmcluster/nodrp/T20K_alpha1_prodrun/pltfiles/plt00036 (44588782) is &amp;lt;The NULL inode&amp;gt; (0), should be /ROOT/arosen2/vmcluster/nodrp/T20K_alpha1_prodrun/pltfiles (40919273).
Fix? no
[...snip...]
&lt;/pre&gt;
&lt;/div&gt;&lt;/div&gt;</description>
                <environment>The MDT is formatted/modified to have the &amp;quot;extents&amp;quot; feature enabled, but I&amp;#39;m not sure if that is relevant for this part of the problem.</environment>
        <key id="17215">LU-2638</key>
            <summary>corruption of MDT &quot;..&quot; entry in some ldiskfs directories</summary>
                <type id="7" iconUrl="https://jira.whamcloud.com/images/icons/issuetypes/task_agile.png">Technical task</type>
                            <parent id="17194">LU-2627</parent>
                                    <priority id="1" iconUrl="https://jira.whamcloud.com/images/icons/priorities/blocker.svg">Blocker</priority>
                        <status id="5" iconUrl="https://jira.whamcloud.com/images/icons/statuses/resolved.png" description="A resolution has been taken, and it is awaiting verification by reporter. From here issues are either reopened, or are closed.">Resolved</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="1">Fixed</resolution>
                                        <assignee username="yong.fan">nasf</assignee>
                                    <reporter username="adilger">Andreas Dilger</reporter>
                        <labels>
                            <label>LB</label>
                    </labels>
                <created>Thu, 17 Jan 2013 18:36:50 +0000</created>
                <updated>Mon, 4 May 2015 21:01:11 +0000</updated>
                            <resolved>Thu, 21 Feb 2013 10:19:03 +0000</resolved>
                                    <version>Lustre 2.4.0</version>
                    <version>Lustre 2.1.3</version>
                    <version>Lustre 2.1.5</version>
                                    <fixVersion>Lustre 2.4.0</fixVersion>
                    <fixVersion>Lustre 2.1.5</fixVersion>
                                        <due></due>
                            <votes>0</votes>
                                    <watches>13</watches>
                                                                            <comments>
                            <comment id="50758" author="adilger" created="Thu, 17 Jan 2013 21:00:36 +0000"  >&lt;p&gt;I&apos;m not sure of the root cause of this corruption yet, though it might relate to renaming a directory in an upgraded 1.8 filesystem.  Fan Yong has a patch for that in &lt;a href=&quot;http://review.whamcloud.com/4899&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://review.whamcloud.com/4899&lt;/a&gt;, but I&apos;m not yet sure if it is the same problem.&lt;/p&gt;</comment>
                            <comment id="50759" author="yong.fan" created="Thu, 17 Jan 2013 21:36:45 +0000"  >&lt;p&gt;There is a bug in old Lustre-2.x: when the &quot;..&quot; is re-insert for rename, it does not check whether there are enough space to hold FID-in-dirent, if no, then it will crash the whole directory. Such issue has been fixed in the patch &lt;a href=&quot;http://review.whamcloud.com/4899&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://review.whamcloud.com/4899&lt;/a&gt;.&lt;/p&gt;</comment>
                            <comment id="50945" author="adilger" created="Mon, 21 Jan 2013 18:38:33 +0000"  >&lt;p&gt;I&apos;m looking at this also in the context of the changes in &lt;a href=&quot;http://review.whamcloud.com/#patch,sidebyside,5134,2,lustre/osd-ldiskfs/osd_handler.c&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://review.whamcloud.com/#patch,sidebyside,5134,2,lustre/osd-ldiskfs/osd_handler.c&lt;/a&gt; and also &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-838&quot; title=&quot;&amp;quot;lfs path2fid /mnt/lustre&amp;quot; (ROOT) returns inode number&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-838&quot;&gt;&lt;del&gt;LU-838&lt;/del&gt;&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Looking at this code, it is actually quite twisty:&lt;/p&gt;
&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;osd_ea_add_rec() calls osd_add_dot_dotdot() for inserting &quot;.&quot; and &quot;..&quot;
	&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
		&lt;li&gt;osd_add_dot_dotdot()
		&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
			&lt;li&gt;adding &quot;.&quot; only sets a flag that &quot;.&quot; was &quot;inserted&quot;&lt;/li&gt;
			&lt;li&gt;adding &quot;..&quot; fails if &quot;.&quot; was not &quot;inserted&quot; first&lt;/li&gt;
			&lt;li&gt;adding &quot;..&quot; gets FID-in-dirent (ldp) if not IGIF FID, though I can&apos;t imagine when a newly created directory IS an IGIF FID?&lt;/li&gt;
			&lt;li&gt;adding &quot;..&quot; calls ldiskfs_add_dot_dotdot() (ldiskfs patched function) if directory is newly created (this will always work correctly)&lt;/li&gt;
			&lt;li&gt;adding &quot;..&quot; calls __osd_ea_add_rec() for existing directories on rename&lt;/li&gt;
		&lt;/ul&gt;
		&lt;/li&gt;
	&lt;/ul&gt;
	&lt;/li&gt;
	&lt;li&gt;osd_ea_add_rec() calls __osd_ea_add_rec() for inserting other entries
	&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
		&lt;li&gt;__osd_ea_add_rec() has a special case for adding the FID-in-dirent (ldp) for normal and IGIF FIDs&lt;/li&gt;
	&lt;/ul&gt;
	&lt;/li&gt;
&lt;/ul&gt;



&lt;p&gt;I&apos;d prefer to fix this at the osd-ldiskfs level instead of adding more changes to ldiskfs itself, if that is practical.  It may be possible to store the &quot;..&quot; rec_len into the directory object when calling osd_ea_index_delete() for &quot;..&quot;, and then use this to decide if the rec_len is large enough when storing the &quot;..&quot; entry back again in _&lt;em&gt;osd_ea_add_rec().  That way, the code wouldn&apos;t _incorrectly&lt;/em&gt; check IGIF FIDs to decide whether to store FID-in-dirent, since regular FID directories may not have space for FID-in-dirent if there was a backup/restore either.  Similarly, in the upstream ext4 there is a new feature for &quot;inline data&quot; that stores small files and directories inside the inode, and only has room for the &quot;..&quot; inode number and not a real dirent, which may break in a similar manner.&lt;/p&gt;

&lt;p&gt;Another option is simply just avoid storing dirdata for ALL &quot;.&quot; and/or &quot;..&quot; entries?  That would also avoid complexity related to the dx_root information, and all of the special case code for handling this.  The &quot;.&quot; FID can be found in the LMA.   However, the FID for the &quot;..&quot; entry would always need to be looked up in the OI, so I&apos;m not necessarily in favour of this if we can store this data easily for new directories (which will be the majority of cases).&lt;/p&gt;</comment>
                            <comment id="50949" author="yong.fan" created="Mon, 21 Jan 2013 21:08:59 +0000"  >&lt;p&gt;Firstly, I agreed with you that current handle &quot;.&quot; and &quot;..&quot; insert/delete/rename are quite confused. I have already fixed some of them during LFSCK 1.5 project:&lt;/p&gt;

&lt;p&gt;1) NOT add FID-in-dirent for &quot;.&quot; object, which is unnecessary. Because we can get &quot;.&quot;&apos;s FID directly from the object itself. Please check the patches:&lt;br/&gt;
&lt;a href=&quot;http://review.whamcloud.com/#patch,sidebyside,4902,11,lustre/osd-ldiskfs/osd_handler.c&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://review.whamcloud.com/#patch,sidebyside,4902,11,lustre/osd-ldiskfs/osd_handler.c&lt;/a&gt;&lt;br/&gt;
&lt;a href=&quot;http://review.whamcloud.com/#patch,sidebyside,4912,17,lustre/osd-ldiskfs/osd_handler.c&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://review.whamcloud.com/#patch,sidebyside,4912,17,lustre/osd-ldiskfs/osd_handler.c&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;2) Try to add FID-in-dirent for &quot;..&quot; object, which will not introduce new patches for ldiskfs, because we already have the ldiskfs patch for that, which was named ext4-hash-indexed-dir-dotdot-update-rhel5.patch. I just replaced it with the new one ext4-dir-dotdot-update.patch, which also cleanup others. Please check:&lt;br/&gt;
&lt;a href=&quot;http://review.whamcloud.com/#change,4899&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://review.whamcloud.com/#change,4899&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;3) NOT distinguish IGIF anymore for FID-in-dirent. Please check:&lt;br/&gt;
&lt;a href=&quot;http://review.whamcloud.com/#patch,sidebyside,4912,17,lustre/osd-ldiskfs/osd_handler.c&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://review.whamcloud.com/#patch,sidebyside,4912,17,lustre/osd-ldiskfs/osd_handler.c&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="50998" author="adilger" created="Tue, 22 Jan 2013 19:28:43 +0000"  >&lt;p&gt;Patch is something in the direction of what I&apos;d like to see for this fix.  It doesn&apos;t compile, and I&apos;m not sure that d_fsdata is actually set at this point, but it was at least checked earlier during osd_index_ea_lookup(), and that could save a flag to determine if the FID was in the dirent at lookup time or not.&lt;/p&gt;</comment>
                            <comment id="51000" author="yong.fan" created="Tue, 22 Jan 2013 20:15:58 +0000"  >&lt;p&gt;To be clear, in my patch (&lt;a href=&quot;http://review.whamcloud.com/#change,4899&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://review.whamcloud.com/#change,4899&lt;/a&gt;), it just rename the old ldiskfs patch &quot;ext4-hash-indexed-dir-dotdot-update-rhel5.patch&quot; to the new one &quot;ext4-dir-dotdot-update.patch&quot;, no additional ldiskfs patch.&lt;/p&gt;

&lt;p&gt;My patch tries to insert FID-in-dirent for &quot;..&quot;, which may cause data moving in the directory block. What you worry about is that such data moving may cause directory crash, right? If yes, I will consider your way and avoid such moving.&lt;/p&gt;</comment>
                            <comment id="51023" author="yong.fan" created="Wed, 23 Jan 2013 09:22:20 +0000"  >&lt;p&gt;When osd_indsx_ea_insert() is called, which will trigger dotdot reinsert (for rename case), at that time, from the OSD view, it does not know whether there is enough space to hold FID-in-dirent for the dotdot entry, unless it calls ldiskfs_bread() to read the directory block #0 and locate to the old dotdot entry. But such processing is just repeat the work for ldiskfs_update_dotdot() in ldiskfs.&lt;/p&gt;

&lt;p&gt;On the other hand, I have very simple solution to resolve your issues inside ldiskfs as following:&lt;/p&gt;

&lt;div class=&quot;code panel&quot; style=&quot;border-width: 1px;&quot;&gt;&lt;div class=&quot;codeContent panelContent&quot;&gt;
&lt;pre class=&quot;code-java&quot;&gt;--- a/ldiskfs/kernel_patches/patches/ext4_data_in_dirent-rhel6.patch
+++ b/ldiskfs/kernel_patches/patches/ext4_data_in_dirent-rhel6.patch
@@ -420,12 +420,13 @@ Index: linux-stage/fs/ext4/namei.c
        &lt;span class=&quot;code-keyword&quot;&gt;else&lt;/span&gt;
 -              &lt;span class=&quot;code-keyword&quot;&gt;assert&lt;/span&gt;(le16_to_cpu(de-&amp;gt;rec_len) &amp;gt;= EXT4_DIR_REC_LEN(2));
 +              &lt;span class=&quot;code-keyword&quot;&gt;assert&lt;/span&gt;(le16_to_cpu(de-&amp;gt;rec_len) &amp;gt;= __EXT4_DIR_REC_LEN(2));
-       de-&amp;gt;name_len = 2;
-       strcpy (de-&amp;gt;name, &lt;span class=&quot;code-quote&quot;&gt;&quot;..&quot;&lt;/span&gt;);
-       ext4_set_de_type(dir-&amp;gt;i_sb, de, S_IFDIR);
-+      &lt;span class=&quot;code-keyword&quot;&gt;if&lt;/span&gt; (data) {
+       de-&amp;gt;name_len = 2;
+       strcpy (de-&amp;gt;name, &lt;span class=&quot;code-quote&quot;&gt;&quot;..&quot;&lt;/span&gt;);
+-      ext4_set_de_type(dir-&amp;gt;i_sb, de, S_IFDIR);
++      &lt;span class=&quot;code-keyword&quot;&gt;if&lt;/span&gt; (data != NULL &amp;amp;&amp;amp; EXT4_DIR_REC_LEN(de) &amp;gt;= dlen) {
 +              de-&amp;gt;name[2] = 0;
 +              memcpy(&amp;amp;de-&amp;gt;name[2 + 1], data, dlen);
++              ext4_set_de_type(dir-&amp;gt;i_sb, de, S_IFDIR);
 +              de-&amp;gt;file_type |= EXT4_DIRENT_LUFID;
 +      }

&lt;/pre&gt;
&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;So I will drop the patch (&lt;a href=&quot;http://review.whamcloud.com/#change,4899&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://review.whamcloud.com/#change,4899&lt;/a&gt;) of updating dotdot in LFSCK 1.5, and just as you suggested, keep the dotdot entry there without FID-in-dirent if there is not enough space to hold the FID-in-dirent. The way is just like above. (contained in the patch &lt;a href=&quot;http://review.whamcloud.com/#change,5046&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://review.whamcloud.com/#change,5046&lt;/a&gt;)&lt;/p&gt;</comment>
                            <comment id="51111" author="adilger" created="Thu, 24 Jan 2013 12:12:18 +0000"  >&lt;p&gt;Nasf, can you please also submit a separate patch for b2_1.&lt;/p&gt;</comment>
                            <comment id="51267" author="yong.fan" created="Fri, 25 Jan 2013 19:36:34 +0000"  >&lt;p&gt;I have split the patch for dotdot updating in this one:&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;http://review.whamcloud.com/#change,5179&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://review.whamcloud.com/#change,5179&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;I will port it to b2_1 also.&lt;/p&gt;</comment>
                            <comment id="51450" author="adilger" created="Wed, 30 Jan 2013 06:29:43 +0000"  >&lt;p&gt;The patches that modify the handling of &quot;.&quot; and &quot;..&quot; are overly complex, and are distributed across several different patches:&lt;/p&gt;
&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;ext4-hash-indexed-dir-dotdot-update-rhel5.patch&lt;/li&gt;
	&lt;li&gt;ext4_data_in_dirent-rhel6.patch&lt;/li&gt;
	&lt;li&gt;ext4-osd-iop-common-rhel6.patch&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;This problem isn&apos;t caused by the fix in &lt;a href=&quot;http://review.whamcloud.com/#change,5179&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://review.whamcloud.com/#change,5179&lt;/a&gt;, but should be cleaned up as a separate bug fix patch once the other feature landings are done.&lt;/p&gt;

&lt;p&gt;It makes sense to clean this code up and put the changes in a single place.  My preference would also be to avoid patching ldiskfs if possible, and instead handle this more like Alex&apos;s ZFS &quot;.&quot;/&quot;..&quot; patch.  Essentially, ldiskfs will create &quot;.&quot; and &quot;..&quot; by itself when the directory is first created, as ext4_mkdir() normally would.  Inserting &quot;.&quot; is a no-op, removing &quot;..&quot; is also a no-op, and inserting &quot;..&quot; will rewrite the inode and lu_fid information in the existing &quot;..&quot; record in-place.&lt;/p&gt;

&lt;p&gt;I suspect this will reduce the complexity of changes to ext4, and result in fewer patches that need to be maintained for every kernel going forward.&lt;/p&gt;</comment>
                            <comment id="51671" author="adilger" created="Sat, 2 Feb 2013 06:30:18 +0000"  >&lt;p&gt;I see a similar intermittent failure in &lt;a href=&quot;https://maloo.whamcloud.com/test_sets/229b2948-59f2-11e2-8604-52540035b04c&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://maloo.whamcloud.com/test_sets/229b2948-59f2-11e2-8604-52540035b04c&lt;/a&gt; for sanity.sh test_214():&lt;/p&gt;

&lt;div class=&quot;preformatted panel&quot; style=&quot;border-width: 1px;&quot;&gt;&lt;div class=&quot;preformattedContent panelContent&quot;&gt;
&lt;pre&gt;[client]
10:00:05:LustreError: 2398:0:(dir.c:421:ll_get_dir_page()) read cache page: [0x3c0001b71:0x1fe7:0x0] at 0: rc -5
10:00:05:LustreError: 2398:0:(dir.c:583:ll_dir_read()) error reading dir [0x3c0001b71:0x1fe7:0x0] at 0: rc -5

drwxr-xr-x 2 root root 20480 Jan  8 10:00 d214c
ls: reading directory /mnt/lustre/d214c: Input/output error

[MDS console]
10:00:07:Lustre: DEBUG MARKER: == sanity test 214: hash-indexed directory test - bug 20133 == 09:59:54 (1357667994)
10:00:07:LDISKFS-fs warning (device dm-0): dx_probe: Unrecognised inode hash code 192 for directory #524991
10:00:07:LDISKFS-fs warning (device dm-0): dx_probe: Corrupt dir inode 524991, running e2fsck is recommended.
10:00:07:Lustre: 8324:0:(mdd_object.c:2046:mdd_dir_page_build()) build page failed: -5!
&lt;/pre&gt;
&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;It looks like the dx_root_info got overwritten.  It looks like it was failing for a while, then stopped on 01/23 (which was a full week before the 5179 patch landed).  Not sure what caused that.&lt;/p&gt;</comment>
                            <comment id="52071" author="adilger" created="Fri, 8 Feb 2013 17:08:42 +0000"  >&lt;p&gt;For testing this patch resolves &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-2627&quot; title=&quot;/bin/ls gets Input/output error&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-2627&quot;&gt;&lt;del&gt;LU-2627&lt;/del&gt;&lt;/a&gt;, the following things must be done:&lt;/p&gt;
&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;format filesystem with b1_8, and mount it&lt;/li&gt;
	&lt;li&gt;create several large subdirectories (10k entries is probably enough)&lt;/li&gt;
	&lt;li&gt;unmount filesystem&lt;/li&gt;
	&lt;li&gt;&quot;&lt;tt&gt;tune2fs -O dirdata /dev/mdtdev&lt;/tt&gt;&quot;&lt;/li&gt;
	&lt;li&gt;mount filesystem with 2.1&lt;/li&gt;
	&lt;li&gt;rename the subdirectories&lt;/li&gt;
	&lt;li&gt;&quot;&lt;tt&gt;ls -l&lt;/tt&gt;&quot; the subdirectories (should fail)&lt;/li&gt;
	&lt;li&gt;run &quot;&lt;tt&gt;e2fsck -f&lt;/tt&gt;&quot; to verify corruption&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;If this test is run with master instead of 2.1, or the &lt;a href=&quot;http://review.whamcloud.com/5179&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://review.whamcloud.com/5179&lt;/a&gt; patch applied to b2_1 (with modifications as needed) no corruption should be seen.&lt;/p&gt;</comment>
                            <comment id="52554" author="green" created="Sun, 17 Feb 2013 01:04:45 +0000"  >&lt;p&gt;patch 5179 cheanly cherrypicks into b2_1, so I am going to test it with a bunch of other stuff.&lt;/p&gt;</comment>
                            <comment id="52797" author="yong.fan" created="Thu, 21 Feb 2013 07:22:43 +0000"  >&lt;p&gt;the patch for b2_1:&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;http://review.whamcloud.com/#change,5498&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://review.whamcloud.com/#change,5498&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="52802" author="jlevi" created="Thu, 21 Feb 2013 10:19:03 +0000"  >&lt;p&gt;Landed to master and b2_1&lt;/p&gt;</comment>
                            <comment id="93876" author="paf" created="Fri, 12 Sep 2014 18:25:58 +0000"  >&lt;p&gt;When upgrading file systems from Cray&apos;s 1.8.6 to 2.4.1, we&apos;ve seen exactly this bug.&lt;/p&gt;

&lt;p&gt;Today, I tested upgrading a file system from WC 1.8.9 (latest as of a months ago) to WC 2.4.0 as released, and I&apos;m seeing this bug.&lt;/p&gt;

&lt;p&gt;The patch (&lt;a href=&quot;http://review.whamcloud.com/#/c/5179/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://review.whamcloud.com/#/c/5179/&lt;/a&gt;) is definitely present in both cases, but the &quot;..&quot; directory entry is not being placed correctly.&lt;/p&gt;

&lt;p&gt;We cause this problem simply by formatting a file system under 1.8, creating a directory with at least one file in it, stopping the file system &amp;amp; adding the dirdata attribute to the MDT, then starting the same file system with 2.4/2.4.1 servers (we have not tried 2.5 or 2.6 yet; planning to test 2.5 just in case) and moving that directory to a new location.&lt;/p&gt;

&lt;p&gt;We see exactly the errors described above when we run e2fsck.  Looking at the directory block before and after the rename (move of the directory to a new location), we see the old &quot;..&quot; entry is still present but the record length of the &quot;.&quot; has been extended, making that effectively unused space.  The new &quot;..&quot; entry is placed in the first location where enough space has been found, with the FID included in it.  (In our simple test case, it&apos;s the last entry in the directory.  On real file systems, it ends up at some intermediate point that was previously unused space.)&lt;/p&gt;

&lt;p&gt;When we take the same test but instead make several files in the directory and delete the first one, that leaves a larger space after the &quot;.&quot; entry available for the new &quot;..&quot; entry.  In that case, the new &quot;..&quot; entry is placed in that location - And is the second entry, so everything works fine.&lt;/p&gt;

&lt;p&gt;It is very much as though the 5179 patch isn&apos;t there at all.  (Remember that this problem is seen going between WC 1.8.9 and WC 2.4.0; this is not a problem just in the Cray source.)&lt;/p&gt;

&lt;p&gt;My examination of the code hasn&apos;t helped me figure out what&apos;s wrong there, other than that the ldiskfs_update_dotdot code is extremely complex.  I may post some questions about that as well.&lt;/p&gt;

&lt;p&gt;When we use e2fsck to &apos;fix&apos; this, it simply forces the new &quot;..&quot; entry in to the second place, and overwrites the first actual &apos;file&apos; entry in the directory.  The result is a consistent directory that will survive further usage - And a file dumped to lost+found.&lt;/p&gt;

&lt;p&gt;Additionally, this bug can cause the MDT to go read-only.  It&apos;s not quite clear to us what&apos;s causing that, but the reported errors from e2fsck all look related, and after running e2fsck, the MDT can be mounted again.&lt;/p&gt;

&lt;p&gt;To make clear the importance of this:&lt;br/&gt;
It looks like all 1.8 file systems upgraded to 2.4+ (haven&apos;t tested 2.5 or 2.6 yet) will experience directory corruption upon enabling the dirdata feature and moving old directories.  This will not affect all old directories, but it will affect many of them.&lt;/p&gt;</comment>
                            <comment id="93877" author="paf" created="Fri, 12 Sep 2014 18:30:12 +0000"  >&lt;p&gt;This is an example of a directory block AFTER the problem has occurred.  Note the presence of the first entry, &quot;.&quot;.  Its length (24 decimal) includes the old &quot;..&quot; entry, which is seen in the second 12 bytes, but is not read because it&apos;s inside the rec_len of the first entry.  Looking further down the directory block, we see a normal file dentry, then after that, we see the new &quot;..&quot; entry (look for &apos;2e2e&apos;), which includes the FID of the new parent.  (It is followed by other file dentries.)&lt;/p&gt;

&lt;p&gt;debugfs:  bd 237582&lt;br/&gt;
0000  b782 0300 1800 0102 2e00 0000 8f8b cb02  ................&lt;br/&gt;
      ^^^^^^^^^ ^^   ^^   ^^&lt;br/&gt;
      inode     |    |    &quot;.&quot;&lt;br/&gt;
             reclen  |&lt;br/&gt;
                   namelen&lt;br/&gt;
0020  0c00 0202 2e2e 0000 b982 0300 2400 0812  ............$...&lt;br/&gt;
                          ^^^^^^^^^ ^^   ^^&lt;br/&gt;
                          inode     |    |&lt;br/&gt;
                                 reclen  |&lt;br/&gt;
                                       namelen&lt;br/&gt;
0040  4361 7461 6c79 7374 0011 0000 0000 0003  Catalyst........&lt;br/&gt;
      ^^^^^^^^^^^^^^^^^^^&lt;br/&gt;
      &quot;Catalyst&quot;&lt;br/&gt;
0060  82b9 2475 25ab 0000 0000 7374 1f80 0702  ..$u%.....st....&lt;br/&gt;
                                    ^^^^^^^^^&lt;br/&gt;
                                    inode&lt;br/&gt;
0100  2000 0212 2e2e 0011 0000 0002 0000 6dd8   .............m.&lt;br/&gt;
      ^^   ^^   ^^     ^^ ^^^^^^^^^^^^^^^^^^^&lt;br/&gt;
   reclen  |    &quot;..&quot;  LEN ^^^^^^^^^^^^^^^^^^^&lt;br/&gt;
         namelen          new parent fid ****&lt;br/&gt;
0120  0001 9551 0000 0000 6500 0000 bc82 0300  ...Q....e.......&lt;br/&gt;
      ^^^^^^^^^^^^^^^^^^^^^^&lt;br/&gt;
      **********************&lt;br/&gt;
0140  2800 0e11 434d 616b 654c 6973 7473 2e74  (...CMakeLists.t&lt;br/&gt;
...&lt;/p&gt;</comment>
                            <comment id="93879" author="paf" created="Fri, 12 Sep 2014 18:38:29 +0000"  >&lt;p&gt;I should clarify a bit further:  The corruption is not immediately harmful.  It is silent, and until e2fsck is run to repair the issue, no user-visible problems occur.  But e2fsck complains that &quot;..&quot; is not the second entry, and in repairing the directory, overwrites old information.&lt;/p&gt;</comment>
                            <comment id="93897" author="paf" created="Fri, 12 Sep 2014 22:11:32 +0000"  >&lt;p&gt;Further investigation this afternoon shows why we&apos;re still seeing a problem despite having the 5179 patch.  We&apos;re down a different code path:&lt;/p&gt;

&lt;p&gt;These are not htree directories, so the &quot;ldiskfs_update_dotdot&quot; code is not called (the &quot;is_dx&quot; test fails - I noticed this by kernel function tracing).  Rather, the standard ldiskfs_add_entry path is used, which then calls in to &quot;add_dirent_to_buf&quot;.  So this is a similar bug, but in that code.  As far as I can tell, that code has no special handling of &quot;..&quot;.  Since &quot;..&quot; cannot be allowed to move from the second position in the directory, this is a significant problem.&lt;/p&gt;

&lt;p&gt;It looks to me like it&apos;s not one ext4 needs to address, since ext4 &quot;..&quot; dentries should never change length, and the other dentries - which can change length when their name changes - can safely be relocated.&lt;/p&gt;

&lt;p&gt;It seems like maybe we can just add this check from ldiskfs_update_dotdot to add_dirent_to_buf:&lt;br/&gt;
&amp;amp;&amp;amp; ldiskfs_get_dirent_data_len(de) &amp;gt;= dlen&lt;/p&gt;

&lt;p&gt;But we have to special case it to &quot;..&quot;.  So I suppose a second if statement, checking that and the name against &quot;..&quot;...  That should be easy enough to do, though I don&apos;t have time at the moment.&lt;/p&gt;</comment>
                            <comment id="114172" author="rmohr" created="Mon, 4 May 2015 20:42:55 +0000"  >&lt;p&gt;I have just seen this exact same problem today, but am not sure how to go about addressing it.  Should I just avoid having e2fsck fix the problem (until a software fix has been created), or do I let e2fsck &quot;fix&quot; things and then try to recover items from lost+found?&lt;/p&gt;</comment>
                            <comment id="114173" author="paf" created="Mon, 4 May 2015 21:01:11 +0000"  >&lt;p&gt;Rick - You can avoid this problem happening to any more directories by unmounting your MDT, turning off dirdata, and remounting.  However, those which are damaged are damaged and e2fsck and recovery from lost+found is your only option.  The software fix* will prevent further damage, but it won&apos;t help with existing damaged directories.&lt;/p&gt;

&lt;p&gt;*About that fix: You&apos;re almost certainly seeing the closely related &lt;a href=&quot;https://jira.hpdd.intel.com/browse/LU-5626&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://jira.hpdd.intel.com/browse/LU-5626&lt;/a&gt;, rather than &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-2638&quot; title=&quot;corruption of MDT &amp;quot;..&amp;quot; entry in some ldiskfs directories&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-2638&quot;&gt;&lt;del&gt;LU-2638&lt;/del&gt;&lt;/a&gt;.  &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-2638&quot; title=&quot;corruption of MDT &amp;quot;..&amp;quot; entry in some ldiskfs directories&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-2638&quot;&gt;&lt;del&gt;LU-2638&lt;/del&gt;&lt;/a&gt; was fixed before release of 2.4, so unless you updated to 2.1 from 1.8 and are still running 2.1 (or 2.2 or 2.3, I guess), it&apos;s &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-5626&quot; title=&quot;Corruption of MDT &#8220;..&#8221; entry in non-HTree ldiskfs directories&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-5626&quot;&gt;&lt;del&gt;LU-5626&lt;/del&gt;&lt;/a&gt;.  &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-5626&quot; title=&quot;Corruption of MDT &#8220;..&#8221; entry in non-HTree ldiskfs directories&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-5626&quot;&gt;&lt;del&gt;LU-5626&lt;/del&gt;&lt;/a&gt; is similar but subtly differently and did get in to 2.4 and 2.5 as released.  Note also that if you&apos;re hitting this situation and running without the fix, there&apos;s also the possibility of kernel panic&apos;ing your MDS, so the workaround of turning off dirdata temporarily is a very good idea (if you can&apos;t get a software update quickly).&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                            <attachment id="12181" name="0001-LU-2638-osd-ldiskfs-clean-up-insertion-of-.-FID.patch" size="17246" author="adilger" created="Tue, 22 Jan 2013 19:28:43 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                                                                                                                                <customfield id="customfield_10890" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10390" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hzvfl3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10090" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>6170</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                </customfields>
    </item>
</channel>
</rss>