https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-254 Lustre <p>The RHEL6 way to build initrd is with dracut. If the Lustre e2fsprogs is installed, dracut builds an initrd that won't boot.</p> <p>This issue was discovered by John Hammond at TACC. Lustre e2fsprogs overwrites blkid, and this maybe the culprit.</p> <p>To reproduce:</p> <p>1. vanilla RHEL6.<br/> 2. install Lustre kernel.<br/> 3. install e2fsprogs (<a href="http://newbuild.whamcloud.com/job/e2fsprogs-reviews/arch=x86_64,distro=el6/12/" class="external-link" target="_blank" rel="nofollow noopener">http://newbuild.whamcloud.com/job/e2fsprogs-reviews/arch=x86_64,distro=el6/12/</a>)<br/> 4. run 'dracut /boot/e2fsprogs.img' <br/> 5. reboot with e2fsprogs.img<br/> 6. boot fails.</p> LU-254

e2fsprogs breaks building of initrd on RHEL6

Bug Major Closed Fixed Andreas Dilger Richard Henwood Fri, 29 Apr 2011 12:21:03 +0000 Wed, 26 Oct 2011 19:56:07 +0000 Thu, 19 May 2011 09:56:55 +0000 Lustre 2.1.0 0 3 <p>Note that force reinstalling the RHEL 6 e2fsprogs (after following 1 through 6) and rerunning dracut will result in a bootable image.</p> <p>It is odd that the blkid program should be replaced, since this RPM is using the same .spec file that RHEL6 uses. The .spec runs configure with --disable-blkid, and I don't see it building or installing anything related to blkid. On the other hand, I see it is running configure with --disable-e2initrd-helper, which seems like it might at least be related, or could just be a red herring. </p> <p>One way to investigate this is to unpack the initrd image files and compare the working one to the non-working one. The image files are gzipped cpio archives and can be extracted for examination. Presumably this has nothing to do with the Lustre kernel, and can ne reproduced for a stock RHEL kernel with just the updated e2fsprogs?</p> <p>Can you please report any files that differ between the images? Also, what files from e2fsprogs are installed into the initrd? I could only possibly imagine e2fsck to check the root fs before mounting it, but I'm not even sure about that. </p> <p>I've extracted the img's with:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java">gunzip &lt; /boot/initrd.img | cpio -i --make-directories </pre> </div></div> <p>and then performed a quick diff:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java">[root@localhost img]# diff -ri working/ fail/ Only in working/etc/ld.so.conf.d: kernel-2.6.32-71.el6.x86_64.conf </pre> </div></div> <p>I copied over kernel-2.6.32-71.el6.x86_64.conf, rebuilt the initrd and it booted.</p> <p>I noticed a problem with the updated e2fsprogs today on my FC13 test system. It was checking a snapshot of the /usr/src filesystem in the background and it marked all of the long symlinks as bad and removed them. It turns out that the ext4 filesystem creates all symlinks with the EXT4_EXTENTS_FL flag enabled, which we've always considered as corrupt for Lustre because extents are only enabled on OSTs, and symlinks are only ever created on the MDT.</p> <p>I'm not sure if this is the root cause of your problem (perhaps there was a long symlink referencing the kernel file that was deleted)? In any case, it should be fixed before using the lustre e2fsprogs on RHEL6 systems.</p> <p>The patch has been pushed to <a href="http://review.whamcloud.com/503" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/503</a> for testing.</p> <p>Richard, John, I've committed a fix via <a href="http://review.whamcloud.com/503" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/503</a> which may fix this problem. It fixes a problem with long symlinks in our e2fsprogs that would only appear for RHEL6-based installations using ext4 for the base filesystems. I'm not sure if that is the root of the problem, but I couldn't find any other problems related to e2fsprogs on RHEL6 that might cause problems with initrd image generation.</p> <p>Could you please give the new packages a try?</p> <p>Integrated in <span class="image-wrap" style=""><img src="http://newbuild.whamcloud.com/images/16x16/blue.png" style="border: 0px solid black" /></span> <a href="http://newbuild.whamcloud.com/job/e2fsprogs-master/./arch=x86_64,distro=el5/23/" class="external-link" target="_blank" rel="nofollow noopener">e2fsprogs-master » x86_64,el5 #23</a><br/> <a href="https://jira.whamcloud.com/browse/LU-254" title="e2fsprogs breaks building of initrd on RHEL6" class="issue-link" data-issue-key="LU-254"><del>LU-254</del></a>: don't mark long extent symlinks bad</p> <p>Andreas Dilger : <a href="http://git.whamcloud.com/gitweb/?p=tools/e2fsprogs.git;a=commit;h=refs/heads/master-lustre&amp;a=commit&amp;h=30b1d6a5221f2cfbafe1701cb91821349b0bffd0" class="external-link" target="_blank" rel="nofollow noopener">30b1d6a5221f2cfbafe1701cb91821349b0bffd0</a><br/> Files : </p> <ul> <li>patches/series</li> <li>patches/e2fsprogs-extents.patch</li> <li>patches/e2fsprogs-tests-f_extents_symlink.patch</li> </ul> <p>Integrated in <span class="image-wrap" style=""><img src="http://newbuild.whamcloud.com/images/16x16/blue.png" style="border: 0px solid black" /></span> <a href="http://newbuild.whamcloud.com/job/e2fsprogs-master/./arch=x86_64,distro=el6/23/" class="external-link" target="_blank" rel="nofollow noopener">e2fsprogs-master » x86_64,el6 #23</a><br/> <a href="https://jira.whamcloud.com/browse/LU-254" title="e2fsprogs breaks building of initrd on RHEL6" class="issue-link" data-issue-key="LU-254"><del>LU-254</del></a>: don't mark long extent symlinks bad</p> <p>Andreas Dilger : <a href="http://git.whamcloud.com/gitweb/?p=tools/e2fsprogs.git;a=commit;h=refs/heads/master-lustre&amp;a=commit&amp;h=30b1d6a5221f2cfbafe1701cb91821349b0bffd0" class="external-link" target="_blank" rel="nofollow noopener">30b1d6a5221f2cfbafe1701cb91821349b0bffd0</a><br/> Files : </p> <ul> <li>patches/e2fsprogs-tests-f_extents_symlink.patch</li> <li>patches/series</li> <li>patches/e2fsprogs-extents.patch</li> </ul> <p>dracut now works for me - I'll ask John tomorrow if he also has success with this release.</p> <p>John, have you had any chance to test if this change fixes the problem that you were seeing? You should be able to use the e2fsprogs packages built to fix this problem without any need to force installation or have any packaging conflicts. They can be downloaded for x86_64 EL6 from:</p> <p><a href="http://newbuild.whamcloud.com/job/e2fsprogs-master/arch=x86_64,distro=el6/23/" class="external-link" target="_blank" rel="nofollow noopener">http://newbuild.whamcloud.com/job/e2fsprogs-master/arch=x86_64,distro=el6/23/</a></p> <p>Success! After the following steps, I no longer experienced the "No root device found" issue in the ram disk's init.</p> <p>1) Reinstall util-linux-ng from the rhel6 ISO.<br/> 2) Upgraded to the e2fsprogs, libcom, and libss from build #23.<br/> 3) Reran dracut to create a new ram disk.</p> <p>Patch has landed to e2fsprogs-1.41.90.wc1</p> Development Rank 1|hzvslj: Rank (Obsolete) 8537 Severity Story Points 2.0