https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-14025 Lustre <p>We are getting new systems that requires SLES15 SP2 and Mellanox OFED.</p> <p>Uploaded is a log-rpms file that showed compilation failure of lustre client 2.12.5 with SLES15 SP2 kernel and Mellanox OFED 5.1. The only mofed versions that support SLES15 SP2 are version 5.x.</p> <p>The failure looked not trivia. Probably due to some rules changes in newer version of gcc, make, rpmbuilds in SLES15 SP2.<br/> ...<br/> /usr/src/linux-5.3.18-24.15/scripts/Makefile.build:57: '/tmp/rpmbuild-lustre-jlan-c7QMO3MB/BUILD/lustre-2.12.5/libcfs/libcfs/libcfs.ko' will not be built even though obj-m is specified.<br/> /usr/src/linux-5.3.18-24.15/scripts/Makefile.build:58: You cannot use subdir-y/m to visit a module Makefile. Use obj-y/m instead.<br/> /usr/src/linux-5.3.18-24.15/scripts/Makefile.build:57: '/tmp/rpmbuild-lustre-jlan-c7QMO3MB/BUILD/lustre-2.12.5/lnet/selftest/lnet_selftest.ko' will not be built even though obj-m is specified.<br/> /usr/src/linux-5.3.18-24.15/scripts/Makefile.build:58: You cannot use subdir-y/m to visit a module Makefile. Use obj-y/m instead.<br/> ...</p> <p>I did not see this type of failures in compiling mofed, and other external kernel modules that I built. This only happened to Lustre build. </p> Lustre client will run in SLES15 SP2 system with Mellanox OFED 5.1 LU-14025

Need lustre client for SLES15 SP2 and Mellanox OFED 5.1

Bug Critical Resolved Fixed Jian Yu Jay Lan build Tue, 13 Oct 2020 16:39:22 +0000 Tue, 23 Jan 2024 07:56:36 +0000 Tue, 23 Jan 2024 07:56:36 +0000 Lustre 2.12.5 Lustre 2.12.5 0 6 <p>Jay</p> <p>Severity 1 is reserved for site down issues. It seems like this was not the intention of this ticket but please confirm</p> <p>Peter</p> <p>No, not site down. We would need it when we have new system installed. Please adjust the severity and priority as you think appropriate.</p> <p>BTW, since I build lustre rpms myself, I only need to have a working (for 2.12.5) patch 'submitted'. You handle release at your pace.</p> <p>Jay</p> <p>Are you able to build master clients ok? If I understand correctly, SLES15 SP2 requires a 5.3 kernel which I believe has been tested successfully with Ubuntu 18.04 HWE</p> <p>Peter</p> <p>I tried to compile master in sles15sp2 environment. It also failed. There were WARNING before the errors.</p> <p>log-rpms.master-sles15sp2-mofed512 is attached. <span class="nobr"><a href="https://jira.whamcloud.com/secure/attachment/36341/36341_log-rpms.master-sles15sp2-mofed512" title="log-rpms.master-sles15sp2-mofed512 attached to LU-14025">log-rpms.master-sles15sp2-mofed512^{<img class="rendericon" src="https://jira.whamcloud.com/images/icons/link_attachment_7.gif" height="7" width="7" align="absmiddle" alt="" border="0"/>}</a></span> </p> <p>Jian</p> <p>Could you please investigate?</p> <p>Thanks</p> <p>Peter</p> <p>Hi Jay,</p> <blockquote><p>I tried to compile master in sles15sp2 environment. It also failed. There were WARNING before the errors.</p></blockquote> <p>The warnings did not cause the build to fail. Have you tried to install the following packages and proceed with the build?</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>error: Failed build dependencies: binutils-devel is needed by lustre-client-2.13.56-1.x86_64 openmpi2-devel is needed by lustre-client-2.13.56-1.x86_64 </pre> </div></div> <p>FYI, with SLES15 SP2 client support patch <a href="https://review.whamcloud.com/40265" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/40265</a> applied to master branch, I can successfully build it with MLNX_OFED 5.1-2.3.7.1:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre># uname -r 5.3.18-24.24-default # rpm -qf /usr/src/ofa_kernel/default/ mlnx-ofa_kernel-devel-5.1-OFED.5.1.2.3.7.1.sles15sp2.x86_64 # cd lustre-release/ # sh ./autogen.sh # ./configure --disable-server --without-zfs --with-linux=/usr/src/linux-5.3.18-24.24 --with-linux-obj=/usr/src/linux-5.3.18-24.24-obj/x86_64/default --with-o2ib=/usr/src/ofa_kernel/default/ # make rpms &lt;~snip~&gt; Wrote: /tmp/rpmbuild-lustre-root-Pz1U9Cf0/RPMS/x86_64/lustre-client-2.13.56_23_g9e2f8a4-1.x86_64.rpm Wrote: /tmp/rpmbuild-lustre-root-Pz1U9Cf0/RPMS/x86_64/lustre-client-kmp-default-2.13.56_23_g9e2f8a4_k5.3.18_24.24-1.x86_64.rpm Wrote: /tmp/rpmbuild-lustre-root-Pz1U9Cf0/RPMS/x86_64/lustre-client-tests-2.13.56_23_g9e2f8a4-1.x86_64.rpm &lt;~snip~&gt; </pre> </div></div> <p>I will try Lustre b2_12 branch and figure out what patches are needed.</p> <p>Patches for <a href="https://jira.whamcloud.com/browse/LU-13209" title="SUSE 15 SP2 client build fix" class="issue-link" data-issue-key="LU-13209"><del>LU-13209</del></a> are needed on Lustre b2_12 branch.<br/> Patches for <a href="https://jira.whamcloud.com/browse/LU-12634" title="Support for linux kernel version 5.3" class="issue-link" data-issue-key="LU-12634"><del>LU-12634</del></a>, <a href="https://jira.whamcloud.com/browse/LU-12904" title="Support for linux kernel version 5.4" class="issue-link" data-issue-key="LU-12904"><del>LU-12904</del></a> and <a href="https://jira.whamcloud.com/browse/LU-13288" title="a performance regression on the single stream write" class="issue-link" data-issue-key="LU-13288"><del>LU-13288</del></a> are also needed.</p> <p>Hi Jay,<br/> It turns out more patches are needed. Here is the tip of the back-ported patch series: <a href="https://review.whamcloud.com/40266" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/40266</a>. With those patches applied to Lustre b2_12 branch, I can successfully build SLES15 SP2 client (kernel 5.3.18-24.24.1) with MLNX_OFED 5.1-2.3.7.1. The build needs to be tested.</p> <p>Hi Jian,</p> <p>I picked up these patches:<br/> <a href="https://jira.whamcloud.com/browse/LU-14029" title="kernel update [SLES15 SP2 5.3.18-24.24.1]" class="issue-link" data-issue-key="LU-14029"><del>LU-14029</del></a> kernel: new kernel <span class="error">&#91;SLES15 SP2 5.3.18-24.24.1&#93;</span><br/> <a href="https://jira.whamcloud.com/browse/LU-13288" title="a performance regression on the single stream write" class="issue-link" data-issue-key="LU-13288"><del>LU-13288</del></a> llite: Find account_page_dirtied on module init<br/> <a href="https://jira.whamcloud.com/browse/LU-12634" title="Support for linux kernel version 5.3" class="issue-link" data-issue-key="LU-12634"><del>LU-12634</del></a> gss: uid_keyring and session_keyring moved<br/> <a href="https://jira.whamcloud.com/browse/LU-12634" title="Support for linux kernel version 5.3" class="issue-link" data-issue-key="LU-12634"><del>LU-12634</del></a> libcfs: force_sig() removed task parameter<br/> <a href="https://jira.whamcloud.com/browse/LU-12634" title="Support for linux kernel version 5.3" class="issue-link" data-issue-key="LU-12634"><del>LU-12634</del></a> build: Recognize ELRepo -ml mainline kernel<br/> <a href="https://jira.whamcloud.com/browse/LU-12634" title="Support for linux kernel version 5.3" class="issue-link" data-issue-key="LU-12634"><del>LU-12634</del></a> llite: lm_compare_owner removed<br/> <a href="https://jira.whamcloud.com/browse/LU-12634" title="Support for linux kernel version 5.3" class="issue-link" data-issue-key="LU-12634"><del>LU-12634</del></a> osd-ldiskfs: bi_phys_segments removed from struct bio<br/> <a href="https://jira.whamcloud.com/browse/LU-12634" title="Support for linux kernel version 5.3" class="issue-link" data-issue-key="LU-12634"><del>LU-12634</del></a> build: kbuild changes in 5.3 drop subdir-m<br/> <a href="https://jira.whamcloud.com/browse/LU-13209" title="SUSE 15 SP2 client build fix" class="issue-link" data-issue-key="LU-13209"><del>LU-13209</del></a> build: SUSE 15 SP2 fix for KBUILD_SRC removed<br/> <a href="https://jira.whamcloud.com/browse/LU-13209" title="SUSE 15 SP2 client build fix" class="issue-link" data-issue-key="LU-13209"><del>LU-13209</del></a> build: Fix vvp_account_page_dirtied<br/> <a href="https://jira.whamcloud.com/browse/LU-13820" title="support for SLES 15 SP2" class="issue-link" data-issue-key="LU-13820"><del>LU-13820</del></a> kernel: new kernel <span class="error">&#91;SLES15 SP2 5.3.18-22.2&#93;</span></p> <p>Some errors:<br/> 1) /tmp/rpmbuild-lustre-jlan-bYemuTCT/BUILD/lustre-2.12.5/libcfs/libcfs/linux/linux-tracefile.c: In function 'cfs_trace_max_debug_mb':<br/> /tmp/rpmbuild-lustre-jlan-bYemuTCT/BUILD/lustre-2.12.5/libcfs/libcfs/linux/linux-tracefile.c:270:34: error: invalid operands to binary &gt;&gt; (have 'long unsigned int <img class="emoticon" src="https://jira.whamcloud.com/images/icons/emoticons/star_yellow.png" height="16" width="16" align="absmiddle" alt="" border="0"/>(void)' and 'int')<br/> int total_mb = (totalram_pages &gt;&gt; (20 - PAGE_SHIFT));<br/> ~~~~~~~~~~~~~~ ^~<br/> 2) /tmp/rpmbuild-lustre-jlan-bYemuTCT/BUILD/lustre-2.12.5/libcfs/libcfs/linux/linux-tracefile.c: At top level:<br/> cc1: error: unrecognized command line option '-Wno-stringop-truncation' <span class="error">&#91;-Werror&#93;</span><br/> 3) /tmp/rpmbuild-lustre-jlan-bYemuTCT/BUILD/lustre-2.12.5/lustre/include/lustre_compat.h:554:20: error: redefinition of 'inode_has_no_xattr'<br/> It was defined at<br/> lustre/include/lustre_compat.h:554:20 and<br/> <span class="error">&#91;linux source&#93;</span>/include/linux/fs.h:3504:20</p> <p>The log-rpms.20201027 would be uploaded.</p> <p>In sles15sp2 (linux 5.3), gcc version is 7.x. The '-Wno-stringop-truncation' problem seems to require gcc 8.</p> <p>Hi Jay,<br/> The patch series has 28 patches:<br/> <a href="https://jira.whamcloud.com/browse/LU-14029" title="kernel update [SLES15 SP2 5.3.18-24.24.1]" class="issue-link" data-issue-key="LU-14029"><del>LU-14029</del></a> kernel: new kernel <span class="error">&#91;SLES15 SP2 5.3.18-24.24.1&#93;</span><br/> <a href="https://jira.whamcloud.com/browse/LU-13344" title="Support for linux 5.6 clients" class="issue-link" data-issue-key="LU-13344"><del>LU-13344</del></a> lnet: stop using struct timeval<br/> <a href="https://jira.whamcloud.com/browse/LU-13210" title="Build failure with gcc8 -Werror=implicit-fallthrough=" class="issue-link" data-issue-key="LU-13210"><del>LU-13210</del></a> lnet: gcc8 add implicit-fallthrough decorator<br/> <a href="https://jira.whamcloud.com/browse/LU-12355" title="Support for linux kernel version 5.0" class="issue-link" data-issue-key="LU-12355"><del>LU-12355</del></a> llite: MS_* flags and SB_* flags split<br/> <a href="https://jira.whamcloud.com/browse/LU-12400" title="Support for linux kernel version 5.2" class="issue-link" data-issue-key="LU-12400"><del>LU-12400</del></a> libcfs: save_stack_trace_tsk if ARCH_STACKWALK<br/> <a href="https://jira.whamcloud.com/browse/LU-12400" title="Support for linux kernel version 5.2" class="issue-link" data-issue-key="LU-12400"><del>LU-12400</del></a> osd-ldiskfs: get rid of legacy 'get_ds()' function<br/> <a href="https://jira.whamcloud.com/browse/LU-12355" title="Support for linux kernel version 5.0" class="issue-link" data-issue-key="LU-12355"><del>LU-12355</del></a> llite: totalram_pages changed to atomic_long_t<br/> <a href="https://jira.whamcloud.com/browse/LU-13476" title="vvp_set_pagevec_dirty / vvp_page_completion_write lock ordering appears to trigger RCU stalls" class="issue-link" data-issue-key="LU-13476"><del>LU-13476</del></a> llite: Fix lock ordering in pagevec_dirty<br/> <a href="https://jira.whamcloud.com/browse/LU-13209" title="SUSE 15 SP2 client build fix" class="issue-link" data-issue-key="LU-13209"><del>LU-13209</del></a> build: SUSE 15 SP2 fix for KBUILD_SRC removed<br/> <a href="https://jira.whamcloud.com/browse/LU-13209" title="SUSE 15 SP2 client build fix" class="issue-link" data-issue-key="LU-13209"><del>LU-13209</del></a> build: Fix vvp_account_page_dirtied<br/> <a href="https://jira.whamcloud.com/browse/LU-13288" title="a performance regression on the single stream write" class="issue-link" data-issue-key="LU-13288"><del>LU-13288</del></a> llite: Find account_page_dirtied on module init<br/> <a href="https://jira.whamcloud.com/browse/LU-12904" title="Support for linux kernel version 5.4" class="issue-link" data-issue-key="LU-12904"><del>LU-12904</del></a> utils: zfs properly detect spa_multihost<br/> <a href="https://jira.whamcloud.com/browse/LU-12904" title="Support for linux kernel version 5.4" class="issue-link" data-issue-key="LU-12904"><del>LU-12904</del></a> build: account_page_dirtied is not exported<br/> <a href="https://jira.whamcloud.com/browse/LU-12634" title="Support for linux kernel version 5.3" class="issue-link" data-issue-key="LU-12634"><del>LU-12634</del></a> llite: Use __xa_set_mark if it is available<br/> <a href="https://jira.whamcloud.com/browse/LU-9920" title="Use pagevec for marking pages dirty" class="issue-link" data-issue-key="LU-9920"><del>LU-9920</del></a> vvp: dirty pages with pagevec<br/> <a href="https://jira.whamcloud.com/browse/LU-12904" title="Support for linux kernel version 5.4" class="issue-link" data-issue-key="LU-12904"><del>LU-12904</del></a> build: Support for gcc -Wimplicit-fallthrough<br/> <a href="https://jira.whamcloud.com/browse/LU-12904" title="Support for linux kernel version 5.4" class="issue-link" data-issue-key="LU-12904"><del>LU-12904</del></a> build: External <em>module</em> decorator removed<br/> <a href="https://jira.whamcloud.com/browse/LU-12634" title="Support for linux kernel version 5.3" class="issue-link" data-issue-key="LU-12634"><del>LU-12634</del></a> libcfs: force_sig() removed task parameter<br/> <a href="https://jira.whamcloud.com/browse/LU-12634" title="Support for linux kernel version 5.3" class="issue-link" data-issue-key="LU-12634"><del>LU-12634</del></a> build: Recognize ELRepo -ml mainline kernel<br/> <a href="https://jira.whamcloud.com/browse/LU-12634" title="Support for linux kernel version 5.3" class="issue-link" data-issue-key="LU-12634"><del>LU-12634</del></a> llite: lm_compare_owner removed<br/> <a href="https://jira.whamcloud.com/browse/LU-12634" title="Support for linux kernel version 5.3" class="issue-link" data-issue-key="LU-12634"><del>LU-12634</del></a> osd-ldiskfs: bi_phys_segments removed from struct bio<br/> <a href="https://jira.whamcloud.com/browse/LU-12634" title="Support for linux kernel version 5.3" class="issue-link" data-issue-key="LU-12634"><del>LU-12634</del></a> build: kbuild changes in 5.3 drop subdir-m<br/> <a href="https://jira.whamcloud.com/browse/LU-12635" title="gcc: -Werror=implicit-fallthrough" class="issue-link" data-issue-key="LU-12635"><del>LU-12635</del></a> lnet: Fix style issues for module.c conctl.c<br/> <a href="https://jira.whamcloud.com/browse/LU-12635" title="gcc: -Werror=implicit-fallthrough" class="issue-link" data-issue-key="LU-12635"><del>LU-12635</del></a> lnet: Fix deceptive indenting on for_each<br/> <a href="https://jira.whamcloud.com/browse/LU-12635" title="gcc: -Werror=implicit-fallthrough" class="issue-link" data-issue-key="LU-12635"><del>LU-12635</del></a> lnet: Fix style issues for selftest/rpc.c<br/> <a href="https://jira.whamcloud.com/browse/LU-12635" title="gcc: -Werror=implicit-fallthrough" class="issue-link" data-issue-key="LU-12635"><del>LU-12635</del></a> build: Support for gcc -Wimplicit-fallthrough<br/> <a href="https://jira.whamcloud.com/browse/LU-9859" title="libcfs simplification" class="issue-link" data-issue-key="LU-9859">LU-9859</a> libcfs: remove wi_data from cfs_workitem<br/> <a href="https://jira.whamcloud.com/browse/LU-9859" title="libcfs simplification" class="issue-link" data-issue-key="LU-9859">LU-9859</a> libcfs: use a workqueue for rehash work.<br/> Among the above patches, <a href="https://review.whamcloud.com/40339" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/40339</a> (<a href="https://jira.whamcloud.com/browse/LU-9859" title="libcfs simplification" class="issue-link" data-issue-key="LU-9859">LU-9859</a> libcfs: use a workqueue for rehash work.) is the first one needs to be applied, and <a href="https://review.whamcloud.com/40266" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/40266</a> (<a href="https://jira.whamcloud.com/browse/LU-14029" title="kernel update [SLES15 SP2 5.3.18-24.24.1]" class="issue-link" data-issue-key="LU-14029"><del>LU-14029</del></a> kernel: new kernel <span class="error">&#91;SLES15 SP2 5.3.18-24.24.1&#93;</span>) is the last one (the tip of the patch series).</p> <p>Hi Jian,</p> <p>Does any of patches you listed require gcc8 or kernel5.4? <a href="https://jira.whamcloud.com/browse/LU-12904" title="Support for linux kernel version 5.4" class="issue-link" data-issue-key="LU-12904"><del>LU-12904</del></a> "Support for linux kernel version 5.4" seems to suggest patches fro kernel 5.4?</p> <p>sles15sp2 is running kernel5.3 and gcc7.</p> <p>Thanks,<br/> Jay</p> <p>No, Jay.<br/> Here is the info on my SLES15 SP2 build node:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre># uname -r 5.3.18-24.24-default # gcc --version | head -1 gcc (SUSE Linux) 7.5.0 </pre> </div></div> <p>The commit messages for the above four <a href="https://jira.whamcloud.com/browse/LU-12904" title="Support for linux kernel version 5.4" class="issue-link" data-issue-key="LU-12904"><del>LU-12904</del></a> patches showed that two of the changes are needed for kernel 5.2 and 5.3. The other two patches are back-ported to resolve patch conflicts. </p> <p>Hi Jian,</p> <p>I needed to create a temporary workaround to address a synopsis change of rdma_reject() in mofed-5.1. The affected code is in lnet/klnds/o2iblnd/o2iblnd_cb.c.</p> <p>Otherwise, all worked well! I had rpms created. Thank you for your help!</p> <p>You're welcome, Jay.<br/> The rdma_reject() issue was fixed in <a href="https://review.whamcloud.com/39781" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/39781</a> on Lustre b2_12 branch. Could you please take a look if your b2_12 codes contain that patch?</p> <p>Ah, no, I did not have that patch in our nas-2.12.5 branch. Thank you.</p> Development Rank 1|i01c9b: Rank (Obsolete) 9223372036854775807 Severity