https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-4677 Lustre <p>While compiling e2fsprogs against Lustre master branch on RHEL6.5/ppc64, I hit the following errors:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>In file included from /root/lustre-release/build/lustre-2.5.56//lustre/include/lustre/lustre_user.h:51, from /root/lustre-release/build/lustre-2.5.56//lustre/include/lustre/lustreapi.h:46, from ../lib/ext2fs/lfsck.h:10, from lfsck_common.c:29: /root/lustre-release/build/lustre-2.5.56//libcfs/include/libcfs/posix/posix-types.h:48: error: conflicting types for 'umode_t' /usr/include/asm/types.h:31: note: previous declaration of 'umode_t' was here CC util.c In file included from /root/lustre-release/build/lustre-2.5.56//lustre/include/lustre/lustre_user.h:51, from /root/lustre-release/build/lustre-2.5.56//lustre/include/lustre/lustreapi.h:46, from ../lib/ext2fs/lfsck.h:10, from lfsck.c:80: /root/lustre-release/build/lustre-2.5.56//libcfs/include/libcfs/posix/posix-types.h:48: error: conflicting types for 'umode_t' /usr/include/asm/types.h:31: note: previous declaration of 'umode_t' was here make[3]: *** [lfsck_common.o] Error 1 </pre> </div></div> <p>The build log "build_e2fsprogs.ppc64.log" is attached.</p> <br/> Lustre Build: <a href="http://build.whamcloud.com/job/lustre-master/1909/">http://build.whamcloud.com/job/lustre-master/1909/</a><br/> Distro/Arch: RHEL6.5/ppc64<br/> LU-4677

e2fsprogs compile error on ppc64: posix-types.h:48: error: conflicting types for 'umode_t'

Bug Critical Resolved Fixed Jian Yu Jian Yu ppc Thu, 27 Feb 2014 10:46:34 +0000 Thu, 22 Nov 2018 23:54:28 +0000 Thu, 22 Nov 2018 23:54:28 +0000 0 5 <p>Here is the patch for e2fsprogs master-lustre branch to check if type “umode_t” is declared in asm/types.h: <a href="http://review.whamcloud.com/9437" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/9437</a></p> <p>With patch #9437, e2fsprogs passed building on ppc64 node.</p> <p>However, it failed at "Running e2fsprogs test suite..." stage:</p> <div class="panel" style="border-width: 1px;"><div class="panelContent"> <p>Running e2fsprogs test suite...</p> <p>/bin/cp ./mke2fs.conf.in mke2fs.conf<br/> Creating test_one script...<br/> Creating test_script...<br/> cmp: d_loaddump.ver.tmp: No such file or directory<br/> d_loaddump: debugfs load/dump test: failed<br/> &#8212; d_loaddump/expect 2014-02-27 09:06:42.000000000 +0000<br/> +++ d_loaddump.log 2014-02-28 14:45:47.251311888 +0000<br/> @@ -7,12 +7,26 @@<br/> e2fsck -yf -N test_filesys<br/> Pass 1: Checking inodes, blocks, and sizes<br/> Pass 2: Checking directory structure<br/> <font color="red">+Directory inode 2, block #0, offset 24: directory corrupted</font><br/> +Salvage? yes<br/> +<br/> Pass 3: Checking directory connectivity<br/> +Unconnected directory inode 11 (/???)<br/> +Connect to /lost+found? yes<br/> +<br/> +/lost+found not found. Create? yes<br/> +<br/> +Pass 3A: Optimizing directories<br/> Pass 4: Checking reference counts<br/> +Inode 11 ref count is 3, should be 2. Fix? yes<br/> +<br/> Pass 5: Checking group summary information<br/> -test_filesys: 12/64 files (0.0% non-contiguous), 158/512 blocks<br/> -Exit status is 0<br/> +<br/> +test_filesys: ***** FILE SYSTEM WAS MODIFIED *****<br/> +test_filesys: 13/64 files (0.0% non-contiguous), 148/512 blocks<br/> +Exit status is 1<br/> debugfs -R ''dump test_data d_loaddump.ver.tmp'' test.img<br/> <font color="red">+test_data: EXT2 directory corrupted</font> <br/> Exit status is 0<br/> cmp d_loaddump.tmp d_loaddump.ver.tmp<br/> -Exit status is 0<br/> +Exit status is 2<br/> -----------<del>8&lt;</del>-----------</p> <p><font color="red">118 tests succeeded 37 tests failed</font><br/> Tests failed: d_loaddump d_special_files f_badroot f_badsymlinks f_badtable f_dir_optimize f_dirdata f_dirdata_optimize f_dup4 f_dup_de f_dup_de2 f_dup_resize f_dupdot f_eofblocks f_expand f_ext_journal f_h_badnode f_h_badroot f_h_reindex f_illitable f_jchksum_remount f_lpf f_lpffile f_noroot f_pgsize_gt_blksize f_random_corruption f_recnect_bad f_reconnect f_rehash_dir f_uninit_bad_free_inodes f_uninit_blk_used_not_set f_uninit_set_inode_not_set f_zero_group f_zero_super m_no_opt s_basic_scan t_quota_1on <br/> make<span class="error">&#91;2&#93;</span>: *** <span class="error">&#91;test_post&#93;</span> Error 1</p> </div></div> <p>The log of "build_e2fsprogs_rpm.ppc64.log" is attached.</p> <p>Hi Andreas &amp; Alex,</p> <p>Do you have any idea for the error mentioned by Yujian?</p> <p>Jinshan</p> <p>It is possible that this error is caused by the dirdata patch. I would suggest to use "git bisect" to try and isolate it quickly. Please ping me in Skype if you haven't used this before and want some help.</p> <p>I can work with Yujian on this - do you have a rough idea about what was the latest working version?</p> <p>Note that we have probably never compiled the Lustre-patches e2fsprogs code on PPC. When I was suggesting to bisect the code, I was thinking about the patches that we apply on top of the base e2fsprogs. It is my guess that the base e2fsprogs (before the "Lustre version" patch will not have this problem. </p> <blockquote><p>It is my guess that the base e2fsprogs (before the "Lustre version" patch will not have this problem.</p></blockquote> <p>Yes, Andreas, I verified this.</p> <p>Here are the commits on e2fsprogs master-lustre branch: <a href="http://git.whamcloud.com/?p=tools/e2fsprogs.git;a=shortlog;h=refs/heads/master-lustre" class="external-link" target="_blank" rel="nofollow noopener">http://git.whamcloud.com/?p=tools/e2fsprogs.git;a=shortlog;h=refs/heads/master-lustre</a><br/> Since commit f6632828e4845041bce12f79d8f3aae095534bc6 (e2fsck: handle preallocation for large PAGE_SIZE), "make check" started failing on ppc64 node. And before that commit, "make check" passed.</p> <p>Here are the details about which test failures on ppc64 node are introduced by which commits:</p> <div class='table-wrap'> <table class='confluenceTable'><tbody> <tr> <th class='confluenceTh'>Failed tests</th> <th class='confluenceTh'>Introduced by</th> </tr> <tr> <td class='confluenceTd'>f_eofblocks</td> <td class='confluenceTd'>e2fsck: handle preallocation for large PAGE_SIZE</td> </tr> <tr> <td class='confluenceTd'>f_pgsize_gt_blksize</td> <td class='confluenceTd'>tests: PAGE_SIZE larger than blocksize with hole</td> </tr> <tr> <td class='confluenceTd'>f_badsymlinks f_dup_de2 f_dup_de f_ext_journal f_h_badnode f_h_badroot f_h_reindex f_jchksum_remount f_rehash_dir f_uninit_bad_free_inodes f_uninit_blk_used_not_set f_uninit_set_inode_not_set f_zero_group f_zero_super</td> <td class='confluenceTd'>e2fsck: add support for dir_data feature</td> </tr> <tr> <td class='confluenceTd'>f_dirdata</td> <td class='confluenceTd'>tests: add basic tests for dir_data feature</td> </tr> <tr> <td class='confluenceTd'>f_dirdata_optimize f_dir_optimize</td> <td class='confluenceTd'><a href="https://jira.whamcloud.com/browse/LU-1774" title="fsck -fD corrupts filesystem" class="issue-link" data-issue-key="LU-1774"><del>LU-1774</del></a> tests: e2fsck -D does not change dirdata</td> </tr> <tr> <td class='confluenceTd'>f_random_corruption</td> <td class='confluenceTd'>debugfs: dump "fid" and "lma" xattrs on inode stat</td> </tr> <tr> <td class='confluenceTd'>s_basic_scan</td> <td class='confluenceTd'>tests: add basic test case for e2scan</td> </tr> <tr> <td class='confluenceTd'>d_loaddump d_special_files f_badroot f_badtable f_dup4 f_dupdot f_dup_resize f_expand f_illitable f_lpf f_lpffile f_noroot f_recnect_bad f_reconnect m_no_opt t_quota_1on</td> <td class='confluenceTd'><a href="https://jira.whamcloud.com/browse/LU-2462" title="debugfs doesn&#39;t care about DIRENT_LUFID flag while inserting new entries" class="issue-link" data-issue-key="LU-2462"><del>LU-2462</del></a> e2fsprogs Consider DIRENT_LUFID flag in link_proc(). </td> </tr> </tbody></table> </div> <p>Could you and Jinshan please give me some instructions about how to make Lustre e2fsprogs codes pass building on ppc64 node without affecting the testing for <a href="https://jira.whamcloud.com/browse/LU-4665" title="utils: lfs setstripe to specify OSTs" class="issue-link" data-issue-key="LU-4665"><del>LU-4665</del></a>? Should I look into the above test failures and resolve them? Thanks a lot in advance.</p> <p>I;ve added Fanyong to the list because he can provide expertise in this topic.</p> <p>Fanyong will look into this.</p> <p>It should be related with bytes order in the structure of "ext2_dirent_entry" and "ext2_dirent_entry_2" on the PPC64 machine.</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java">struct ext4_dir_entry { __le32 inode; <span class="code-comment">/* Inode number */</span> __le16 rec_len; <span class="code-comment">/* Directory entry length */</span> __le16 name_len; <span class="code-comment">/* Name length */</span> <span class="code-object">char</span> name[EXT4_NAME_LEN]; <span class="code-comment">/* File name */</span> }; /* * The <span class="code-keyword">new</span> version of the directory entry. Since EXT4 structures are * stored in intel <span class="code-object">byte</span> order, and the name_len field could never be * bigger than 255 chars, it's safe to reclaim the extra <span class="code-object">byte</span> <span class="code-keyword">for</span> the * file_type field. */ struct ext4_dir_entry_2 { __le32 inode; <span class="code-comment">/* Inode number */</span> __le16 rec_len; <span class="code-comment">/* Directory entry length */</span> __u8 name_len; <span class="code-comment">/* Name length */</span> __u8 file_type; <span class="code-object">char</span> name[EXT4_NAME_LEN]; <span class="code-comment">/* File name */</span> }; </pre> </div></div> <p>In fact, the ext4_dir_entry_2::name_len and ext4_dir_entry_2::file_type should be exchanged on PPC64 as following:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java">struct ext2_dir_entry_2 { __le32 inode; <span class="code-comment">/* Inode number */</span> __le16 rec_len; <span class="code-comment">/* Directory entry length */</span> #ifndef WORDS_BIGENDIAN __u8 name_len; <span class="code-comment">/* Name length */</span> __u8 file_type; #<span class="code-keyword">else</span> __u8 file_type; __u8 name_len; <span class="code-comment">/* Name length */</span> #endif <span class="code-object">char</span> name[EXT2_NAME_LEN]; <span class="code-comment">/* File name */</span> }; </pre> </div></div> <p>Then we can access the ext2_dir_entry::name_len via ext2_dir_entry_2::name_len.</p> <p>In fact, because the CPU bytes-order for PPC64 and the on-disk bytes-order are different, any value (not only ext2_dir_dirent, but also others, such as inode, super_block, and so on) which is larger than 1-byte and loaded from disk (write or disk) should be converted via lexx_to_cpu (or cpu_to_lexx). Unfortunately, we missed to do that at a lot of points in current implantation.</p> <p>On the other hand, in current implementation, we assume the ext2_dir_entry_2::name_len is lower than ext2_dir_entry_2::file_type, so there are some logic like the following:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"><span class="code-object">int</span> filetype = dirent-&gt;name_len &gt;&gt; 8; ... dirent-&gt;name_len = ((dirent-&gt;name_len &amp; 0xFF) | (dirdata | should_be) &lt;&lt; 8); ... </pre> </div></div> <p>To make the e2fsck workable on PPC64 correctly, all related codes should be well checked and re-considere. In theory, it is not complex, but it involves a lot of trivial things and I am not sure whether there will be other data structures to be handled. We must check carefully one by one. So it is not a simple work.</p> <p>By using the following temporary fix suggested by nasf, the number of failed tests was reduced from 37 to 3 (f_eofblocks, f_pgsize_gt_blksize, f_dirdata):</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java">diff --git a/lib/ext2fs/ext2_fs.h b/lib/ext2fs/ext2_fs.h index a99ab37..95233ba 100644 --- a/lib/ext2fs/ext2_fs.h +++ b/lib/ext2fs/ext2_fs.h @@ -785,11 +785,16 @@ struct ext2_dir_entry { * file_type field. */ struct ext2_dir_entry_2 { - __u32 inode; <span class="code-comment">/* Inode number */</span> - __u16 rec_len; <span class="code-comment">/* Directory entry length */</span> - __u8 name_len; <span class="code-comment">/* Name length */</span> - __u8 file_type; - <span class="code-object">char</span> name[EXT2_NAME_LEN]; <span class="code-comment">/* File name */</span> + __u32 inode; <span class="code-comment">/* Inode number */</span> + __u16 rec_len; <span class="code-comment">/* Directory entry length */</span> +#ifndef WORDS_BIGENDIAN + __u8 name_len; <span class="code-comment">/* Name length */</span> + __u8 file_type; +#<span class="code-keyword">else</span> + __u8 file_type; + __u8 name_len; <span class="code-comment">/* Name length */</span> +#endif + <span class="code-object">char</span> name[EXT2_NAME_LEN]; <span class="code-comment">/* File name */</span> }; </pre> </div></div> <p>After temporarily disabling the 3 failed tests, I successfully built e2fsprogs on the ppc64 node and got the rpm packages.<br/> I'll see what errors I'll hit while testing Lustre server codes on the ppc64 node. In the meantime, nasf will continue digging into the failures and find out more proper way to resolve those failures. Thanks, nasf! </p> <p>Looking at the e2fsprogs code, it seems that there is almost no code using ext2_dir_entry_2, it all uses ext2_dir_entry and then masks name_len and shifts down file_type. Unfortunately, in the e2fsprogs code, the in-memory representation of the directory entry is often already swabbed, so the change to ext2_dir_entry_2 would break for entries that are already swabbed.</p> <p>I think it would be better to update the Lustre patches to use ext2_dir_entry and use the mask-and-shift method used by the rest of the code. Since I'm already updating the e2fsprogs patches to the latest maint release, I will do this part. I'm not sure what is wrong with the PAGE_SIZE patches or others since I don't know why they are failing.</p> <p>About the patch for PAGE_SIZE, I think the tests should be improved. Here are the files in the test image for f_eofblocks:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"># ls -ail /mnt/mds1 total 75K 2 drwxr-xr-x 3 root root 1.0K Apr 11 2012 ./ 8199 drwxr-xr-x. 115 root root 4.0K Dec 31 01:17 ../ 12 -rw-r--r-- 1 root root 1.0K Apr 11 2012 a1 13 -rw-r--r-- 1 root root 1.0K Apr 11 2012 a2 14 -rw-r--r-- 1 root root 1.0K Apr 11 2012 b1 15 -rw-r--r-- 1 root root 1.0K Apr 11 2012 b2 16 -rw-r--r-- 1 root root 2.0K Apr 11 2012 c1 17 -rw-r--r-- 1 root root 2.0K Apr 11 2012 c2 18 -rw-r--r-- 1 root root 2.0K Apr 11 2012 d1 19 -rw-r--r-- 1 root root 2.0K Apr 11 2012 d2 20 -rw-r--r-- 1 root root 2.0K Apr 11 2012 e1 21 -rw-r--r-- 1 root root 2.0K Apr 11 2012 e2 22 -rw-r--r-- 1 root root 1.0K Apr 11 2012 f1 23 -rw-r--r-- 1 root root 1.0K Apr 11 2012 f2 24 -rw-r--r-- 1 root root 4.0K Apr 11 2012 g1 25 -rw-r--r-- 1 root root 4.0K Apr 11 2012 g2 26 -rw-r--r-- 1 root root 4.0K Apr 11 2012 h1 27 -rw-r--r-- 1 root root 4.0K Apr 11 2012 h2 28 -rw-r--r-- 1 root root 0 Apr 11 2012 i1 29 -rw-r--r-- 1 root root 0 Apr 11 2012 i2 30 -rw-r--r-- 1 root root 2.0K Apr 11 2012 j1 31 -rw-r--r-- 1 root root 2.0K Apr 11 2012 j2 11 drwx------ 2 root root 12K Apr 11 2012 lost+found/ root@RHEL6:/tmp/ # stat /mnt/mds1/j1 File: `/mnt/mds1/j1' Size: 2048 Blocks: 8 IO Block: 1024 regular file Device: 700h/1792d Inode: 30 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2012-04-11 11:26:25.000000000 +0800 Modify: 2012-04-11 11:29:47.000000000 +0800 Change: 2012-04-11 11:29:47.000000000 +0800 root@RHEL6:/tmp/ # stat /mnt/mds1/j2 File: `/mnt/mds1/j2' Size: 2048 Blocks: 12 IO Block: 1024 regular file Device: 700h/1792d Inode: 31 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2012-04-11 11:26:30.000000000 +0800 Modify: 2012-04-11 11:29:50.000000000 +0800 Change: 2012-04-11 11:29:50.000000000 +0800 </pre> </div></div> <p>For the server with 4KB PAGE_SIZE, such as x86_64, the "j1" case is acceptable, but for other large PAGE_SIZE cases, such as 64KB on PPC64, "Blocks: 8" should be for "truncate" case, so the size for "j1" should be 512 * 8 = 4096, but not the showed 2048.</p> <p>To make the tests to be workable for difference arch, the file "j1" should be removed from the test image.</p> <p>This is fixed in the latest e2fsprogs release.</p> Related LU-5326 Development Rank 1|hzwg2v: Rank (Obsolete) 12838 Severity