https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-4865 Lustre <p>The ZFS VFS code does change blocksize as file grows, e.g. in zfs_write()-&gt;zfs_grow_blocksize(). But the osd-zfs just opens the objset and manipulate it directly by the DMU API, shortcutting ZFS VFS layers, so the zfs_grow_blocksize() logic will not work for us.</p> <p>Currently the osd-zfs set blocksize to 0 on MDS (then ZFS converts it to the default 512 byte block size), and to 128K on OSS. Neither is optimal setting. We may need to duplicate the ZPL functionality to grow block size dynamically as well.</p> LU-4865

osd-zfs: increase object block size dynamically as object grows

Improvement Minor Resolved Fixed Jinshan Xiong Isaac Huang RZ_LS zfs Sun, 6 Apr 2014 20:00:51 +0000 Wed, 22 Jun 2022 19:31:13 +0000 Mon, 29 Aug 2016 13:00:05 +0000 Lustre 2.9.0 0 13 <p>For large block support, the hard-coded 128K block size in osd_mkreg() should be changed.</p> <p>This really needs to be implemented in Lustre to ensure good performance over a range of file sizes. While you can't use zfs_grow_blocksize since it's part of the ZPL. This is really just a thin wrapper around dmu_object_set_blocksize() which does all the heavy lifting, and this is available. It will also properly enforce the maximum supported block size allowed by the feature flags.</p> <p>I'll very likely take a crack at this tomorrow since this code is fresh in my mind after working on <a href="https://jira.whamcloud.com/browse/LU-6152" title="zfs large block support" class="issue-link" data-issue-key="LU-6152"><del>LU-6152</del></a>.</p> <p>A comment from Eric: The first write to a file might be at a large offset if multiple clients are all racing to write at different offsets in a shared Lustre file, but the client could send a hint with that first write that this file looks like it's streaming.</p> <p>A write to any offset larger than the maximum block size needs to result in the maximum block size being used for the file.</p> <p>As an enhancement after the basic implementation of the "grow block size to match file size, and choose maximum blocksize" code is done is to hook the block size selection into the ladvise call (<a href="https://jira.whamcloud.com/browse/LU-4931" title="New feature of giving server/storage side advice of accessing file" class="issue-link" data-issue-key="LU-4931"><del>LU-4931</del></a>) to have LADVISE_RANDOM select a smaller blocksize, or even allow that call to specify the random IO blocksize along with the flag. That would avoid large read-modify-write operations with large blocksize, similar to how it is possible to specify the blocksize for a ZVOL. </p> <p>Jinshan Xiong (jinshan.xiong@intel.com) uploaded a new patch: <a href="http://review.whamcloud.com/15127" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/15127</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-4865" title="osd-zfs: increase object block size dynamically as object grows" class="issue-link" data-issue-key="LU-4865"><del>LU-4865</del></a> zfs: grow block size by write pattern<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 7fc0e68df9f917fb0f590f3169867dfa082f6373</p> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="http://review.whamcloud.com/15127/" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/15127/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-4865" title="osd-zfs: increase object block size dynamically as object grows" class="issue-link" data-issue-key="LU-4865"><del>LU-4865</del></a> zfs: grow block size by write pattern<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 3e4369135127b350dbc26a4a5dc94cfa46e394cf</p> <p>Landed for 2.8.0</p> <p>Andreas Dilger (andreas.dilger@intel.com) uploaded a new patch: <a href="http://review.whamcloud.com/17053" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/17053</a><br/> Subject: Revert "<a href="https://jira.whamcloud.com/browse/LU-4865" title="osd-zfs: increase object block size dynamically as object grows" class="issue-link" data-issue-key="LU-4865"><del>LU-4865</del></a> zfs: grow block size by write pattern"<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 2c3835d58993488669d15c2603d8fbbfd55f40d4</p> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="http://review.whamcloud.com/17053/" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/17053/</a><br/> Subject: Revert "<a href="https://jira.whamcloud.com/browse/LU-4865" title="osd-zfs: increase object block size dynamically as object grows" class="issue-link" data-issue-key="LU-4865"><del>LU-4865</del></a> zfs: grow block size by write pattern"<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: d690560bcef69cfe91747ffb8a7efb77c04800e6</p> <p>Jinshan Xiong (jinshan.xiong@intel.com) uploaded a new patch: <a href="http://review.whamcloud.com/18441" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/18441</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-4865" title="osd-zfs: increase object block size dynamically as object grows" class="issue-link" data-issue-key="LU-4865"><del>LU-4865</del></a> zfs: grow block size by write pattern<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: c2f3cb1cd31bb69b5ca08a56485e4fa4d7e03d7d</p> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="http://review.whamcloud.com/18441/" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/18441/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-4865" title="osd-zfs: increase object block size dynamically as object grows" class="issue-link" data-issue-key="LU-4865"><del>LU-4865</del></a> zfs: grow block size by write pattern<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: b738c4850935f3a9c483b3141cb37d6539557615</p> <p>Landed to master for 2.9.0</p> Related LU-6152 LU-15963 LU-7404 LU-8591 LU-7226 Development Rank 1|hzwjc7: Rank (Obsolete) 13423