https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-14795 Lustre <p>Nvidia release a final version for the own GDS project.<br/> Lets add support in lustre code.</p> GDS 1.0 + MOFED 5.3 LU-14795

NVidia GDS support in lustre

New Feature Major Closed Won't Do Alexey Lyashkov Alexey Lyashkov Mon, 28 Jun 2021 11:55:27 +0000 Sun, 9 Jan 2022 23:29:33 +0000 Wed, 11 Aug 2021 03:42:05 +0000 0 13 <p>Alexey Lyashkov (alexey.lyashkov@hpe.com) uploaded a new patch: <a href="https://review.whamcloud.com/44093" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/44093</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-14795" title="NVidia GDS support in lustre" class="issue-link" data-issue-key="LU-14795"><del>LU-14795</del></a> lnet: use lnet_send_data as possible.<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: ec044b859e20a19e59b15074f5ae98d1da3d6bbd</p> <p>Alexey Lyashkov (alexey.lyashkov@hpe.com) uploaded a new patch: <a href="https://review.whamcloud.com/44094" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/44094</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-14795" title="NVidia GDS support in lustre" class="issue-link" data-issue-key="LU-14795"><del>LU-14795</del></a> lnet: add GDS configure<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: d108819dbbfb58a980f2323906e38f510401c1d1</p> <p>Alexey Lyashkov (alexey.lyashkov@hpe.com) uploaded a new patch: <a href="https://review.whamcloud.com/44095" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/44095</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-14795" title="NVidia GDS support in lustre" class="issue-link" data-issue-key="LU-14795"><del>LU-14795</del></a> lnet: export additional info<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 002f3c65b57aacf57bc23ca1a8fc4553ad0cdbe3</p> <p>Alexey Lyashkov (alexey.lyashkov@hpe.com) uploaded a new patch: <a href="https://review.whamcloud.com/44096" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/44096</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-14795" title="NVidia GDS support in lustre" class="issue-link" data-issue-key="LU-14795"><del>LU-14795</del></a> llite: detect an GPU transfer<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 632892ba1c00f44c48a9125bd364e81cbff7dd5a</p> <p>Alexey Lyashkov (alexey.lyashkov@hpe.com) uploaded a new patch: <a href="https://review.whamcloud.com/44097" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/44097</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-14795" title="NVidia GDS support in lustre" class="issue-link" data-issue-key="LU-14795"><del>LU-14795</del></a> ptlrpc: GDS bulk transfer support<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 647ce2dfab23de67b4b20b3187911db3419ff4ff</p> <p>Alexey Lyashkov (alexey.lyashkov@hpe.com) uploaded a new patch: <a href="https://review.whamcloud.com/44098" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/44098</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-14795" title="NVidia GDS support in lustre" class="issue-link" data-issue-key="LU-14795"><del>LU-14795</del></a> lnet: LNet GDS support<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 0013ef246d0c8b5e506cdf31a5a9879f3962fe6e</p> <p>Alexey Lyashkov (alexey.lyashkov@hpe.com) uploaded a new patch: <a href="https://review.whamcloud.com/44099" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/44099</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-14795" title="NVidia GDS support in lustre" class="issue-link" data-issue-key="LU-14795"><del>LU-14795</del></a> lnet: add GPU &lt;&gt; IB device affinity handing.<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 31e16fcdf48de3e59bd0156f9e7918d854380082</p> <p>This ticket "pre-dates" <a href="https://jira.whamcloud.com/browse/LU-14798" title="NVIDIA GPUDirect Storage Support" class="issue-link" data-issue-key="LU-14798"><del>LU-14798</del></a>.</p> <p>checked out head of <a href="https://jira.whamcloud.com/browse/LU-14795" title="NVidia GDS support in lustre" class="issue-link" data-issue-key="LU-14795"><del>LU-14795</del></a> <a href="https://review.whamcloud.com/#/c/44099" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/#/c/44099</a>, but build failed below.</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre># sh ./autogen.sh ; ./configure --with-o2ib=/usr/src/ofa_kernel/default --disable-server; make debs - snip - checking if Linux kernel was built with CONFIG_MODULES... yes checking if Linux kernel was built with CONFIG_MODVERSIONS... yes checking if Linux kernel was built with CONFIG_KALLSYMS... yes checking if Linux kernel module loading is possible... yes checking for /usr/src/nvidia-470.57.02/nvidia//nv-p2p.h... yes NVIDIA path is /usr/src/nvidia-470.57.02/nvidia/ checking for /usr/src/nvidia-fs-2.7.50/nvfs-dma.h... yes checking for /usr/src/nvidia-fs-2.7.50/config-host.h... no configure: error: GDS sources don't ready for build. run configure please make: *** No rule to make target 'debs'. Stop. </pre> </div></div> <p>testing GDS-1.0.1.3 which is part of latest cuda 11.4.1</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre># /usr/local/cuda-11.4/gds/tools/gdscheck -v GDS release version: 1.0.1.3 nvidia_fs version: 2.7 libcufile version: 2.4 </pre> </div></div> <p>Let's run a 'configure' script in the gds sources, it configure options removed after DKMS build.<br/> or make nv_configure in /usr/src/nvidia-fs-2.7.50/<br/> (i use an make all, but configure is enough).</p> <p>WC approach landed already.</p> Related LU-14798 LU-14212 Development Rank 1|i01xyv: Rank (Obsolete) 9223372036854775807