https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-3606 Lustre <p>The sys_fallocate() syscall was introduced to the linux kernel in the 2.6.24 kernel. There is also an ext4_fallocate() method added in this same kernel release. This has been available in vendor kernels since RHEL 5.4.</p> <p>We need to implement an fallocate() method for llite, and transport this to the OSTs to interface with the underlying OSD's fallocate() code (for ldiskfs, ZFS has no such method).</p> <p>The FALLOC_FL_PUNCH_HOLE functionality of fallocate() fits very nicely with the OST_PUNCH RPC, since this was anticipated in Lustre from the beginning. The actual space reservation might also use OST_PUNCH or OST_WRITE, or we might consider a new OST_PREALLOC RPC, depending on what fits best. </p> LU-3606

Implement fallocate() support for ldiskfs Lustre

New Feature Major Resolved Fixed Arshad Hussain Andreas Dilger medium patch Wed, 17 Jul 2013 17:06:31 +0000 Thu, 30 Nov 2023 21:58:17 +0000 Fri, 27 Nov 2020 13:54:11 +0000 Lustre 2.14.0 0 19 <p>The fallocate() API has been added to newer versions of the Lustre kernel to provide both persistent space reservation (block preallocation for a file, possibly beyond the file size, without having to write zeroes to the whole file), and for the reverse operation of hole punching (freeing allocated blocks in the middle or end of a file). However, Lustre predates these APIs and has not yet added support for them. Being able to preallocate space for a file is very useful for HPC applications that know the size of the output file in advance, and helps Lustre make better allocation decisions based on the file size.</p> <p>This project will touch all of the layers of the Lustre IO stack, from the Linux client VFS interface, through the file layout to the RPC interface, over to the storage server, and then interfacing with the back-end filesystems to use the space reservation (ext4 only) and hole punch (both ext4 and ZFS) methods. As a stretch goal, an ioctl() pass-through interface from the server mountpoint to the underlying filesystem would allow direct passing of FITRIM ioctl commands to the storage so that SSD-based filesystems can optimize their internal space management. </p> <p>I have uploaded the first patchset here: <a href="http://review.whamcloud.com/#/c/9275/" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#/c/9275/</a></p> <p>I will track the state of this ticket.</p> <p>Jinshan, can you see what is needed to get this patch across the finish line? Swap il is no longer available to work on it. </p> <p><span class="nobr"><a href="mailto:andreas.dilger@intel.com" class="external-link" target="_blank" rel="nofollow noopener">andreas.dilger@intel.com^{<img class="rendericon" src="https://jira.whamcloud.com/images/icons/mail_small.gif" height="12" width="13" align="absmiddle" alt="" border="0"/>}</a></span> I worked on the patch and have it working, including newer kernel versions. Most review comments against last patch addressed. Can post it against the same change-id if that works.</p> <p>Yes, please reuse the same Change-Id. That will allow comparing the new patch against the old one, and the old one is hopelessly outdated and not useful.</p> <p>Arshad Hussain (arshad.super@gmail.com) uploaded a new patch: <a href="https://review.whamcloud.com/37273" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/37273</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-3606" title="Implement fallocate() support for ldiskfs Lustre" class="issue-link" data-issue-key="LU-3606"><del>LU-3606</del></a> fsx: Add fallocate operation to fsx<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 2269c370864bd42a5fbf0608afecba0ec8f010db</p> <p>Arshad Hussain (arshad.super@gmail.com) uploaded a new patch: <a href="https://review.whamcloud.com/37277" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/37277</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-3606" title="Implement fallocate() support for ldiskfs Lustre" class="issue-link" data-issue-key="LU-3606"><del>LU-3606</del></a> lustre: Reserve fallocate opcode<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: a52c11c2c0935ce1b741dfde703bbaeb93c7c499</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/37273/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/37273/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-3606" title="Implement fallocate() support for ldiskfs Lustre" class="issue-link" data-issue-key="LU-3606"><del>LU-3606</del></a> fsx: Add fallocate operation to fsx<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 853d180121a6e313ffe678bbe5c107d7afc601d5</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/37277/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/37277/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-3606" title="Implement fallocate() support for ldiskfs Lustre" class="issue-link" data-issue-key="LU-3606"><del>LU-3606</del></a> lustre: Reserve OST_FALLOCATE(fallocate) opcode<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 46a11df089c92f0d18e6f3224eca76abfcd0ac67</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/9275/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/9275/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-3606" title="Implement fallocate() support for ldiskfs Lustre" class="issue-link" data-issue-key="LU-3606"><del>LU-3606</del></a> fallocate: Implement fallocate preallocate operation<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 48457868a02ae5a0407f304c9028f7e80e7fb574</p> <p>Do I understand correctly that fallocate support for ldiskfs has landed for 2.14 but fallocate support for ZFS needs more work? If so, shall we track the latter under a new ticket so that it is clear what capabilities are in each release?</p> <p>Yes, the patch <a href="https://review.whamcloud.com/36506" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/36506</a> "<tt><a href="https://jira.whamcloud.com/browse/LU-3606" title="Implement fallocate() support for ldiskfs Lustre" class="issue-link" data-issue-key="LU-3606"><del>LU-3606</del></a> zfs-osd: Add fallocate functionality for ZFS</tt>" needs to be refreshed. There also needs to be a separate patch to implement <tt>FALLOCATE_FL_PUNCH_HOLE</tt>.</p> <p>Opened <a href="https://jira.whamcloud.com/browse/LU-14157" title="Implement fallocate() support for zfs Lustre" class="issue-link" data-issue-key="LU-14157">LU-14157</a> to track fallocate support for zfs. Marking as fixed in 2.14 for ldiskfs support</p> <p>Thanks Andreas, Peter.</p> <p>&gt;Opened <a href="https://jira.whamcloud.com/browse/LU-14157" title="Implement fallocate() support for zfs Lustre" class="issue-link" data-issue-key="LU-14157">LU-14157</a> to track fallocate support for zfs. Marking as fixed in 2.14 for ldiskfs support<br/> <a href="https://jira.whamcloud.com/browse/LU-14160" title="Implement fallocate FALLOCATE_FL_PUNCH_HOLE support" class="issue-link" data-issue-key="LU-14160"><del>LU-14160</del></a> opened for ldiskfs fallocate punch support.</p> Cloners LU-14157 Related LU-14160 LU-14333 LU-15167 LU-15519 LU-10370 LU-8699 LU-13155 LU-17055 LU-17326 LUDOC-487 LU-14382 LU-10810 LU-14287 Development Rank 1|hzvvl3: Rank (Obsolete) 9150