[LU-4865] osd-zfs: increase object block size dynamically as object grows Created: 06/Apr/14 Updated: 22/Jun/22 Resolved: 29/Aug/16 |
|
| Status: | Resolved |
| Project: | Lustre |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | Lustre 2.9.0 |
| Type: | Improvement | Priority: | Minor |
| Reporter: | Isaac Huang (Inactive) | Assignee: | Jinshan Xiong (Inactive) |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | RZ_LS, zfs | ||
| Issue Links: |
|
||||||||||||||||||||||||
| Rank (Obsolete): | 13423 | ||||||||||||||||||||||||
| Description |
|
The ZFS VFS code does change blocksize as file grows, e.g. in zfs_write()->zfs_grow_blocksize(). But the osd-zfs just opens the objset and manipulate it directly by the DMU API, shortcutting ZFS VFS layers, so the zfs_grow_blocksize() logic will not work for us. Currently the osd-zfs set blocksize to 0 on MDS (then ZFS converts it to the default 512 byte block size), and to 128K on OSS. Neither is optimal setting. We may need to duplicate the ZPL functionality to grow block size dynamically as well. |
| Comments |
| Comment by Isaac Huang (Inactive) [ 23/Jan/15 ] |
|
For large block support, the hard-coded 128K block size in osd_mkreg() should be changed. |
| Comment by Brian Behlendorf [ 24/Jan/15 ] |
|
This really needs to be implemented in Lustre to ensure good performance over a range of file sizes. While you can't use zfs_grow_blocksize since it's part of the ZPL. This is really just a thin wrapper around dmu_object_set_blocksize() which does all the heavy lifting, and this is available. It will also properly enforce the maximum supported block size allowed by the feature flags. |
| Comment by Brian Behlendorf [ 28/Jan/15 ] |
|
I'll very likely take a crack at this tomorrow since this code is fresh in my mind after working on |
| Comment by Isaac Huang (Inactive) [ 19/Mar/15 ] |
|
A comment from Eric: The first write to a file might be at a large offset if multiple clients are all racing to write at different offsets in a shared Lustre file, but the client could send a hint with that first write that this file looks like it's streaming. |
| Comment by Brian Behlendorf [ 20/Apr/15 ] |
|
A write to any offset larger than the maximum block size needs to result in the maximum block size being used for the file. |
| Comment by Andreas Dilger [ 08/May/15 ] |
|
As an enhancement after the basic implementation of the "grow block size to match file size, and choose maximum blocksize" code is done is to hook the block size selection into the ladvise call ( |
| Comment by Gerrit Updater [ 03/Jun/15 ] |
|
Jinshan Xiong (jinshan.xiong@intel.com) uploaded a new patch: http://review.whamcloud.com/15127 |
| Comment by Gerrit Updater [ 10/Sep/15 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/15127/ |
| Comment by Joseph Gmitter (Inactive) [ 10/Sep/15 ] |
|
Landed for 2.8.0 |
| Comment by Gerrit Updater [ 05/Nov/15 ] |
|
Andreas Dilger (andreas.dilger@intel.com) uploaded a new patch: http://review.whamcloud.com/17053 |
| Comment by Gerrit Updater [ 13/Nov/15 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/17053/ |
| Comment by Gerrit Updater [ 13/Feb/16 ] |
|
Jinshan Xiong (jinshan.xiong@intel.com) uploaded a new patch: http://review.whamcloud.com/18441 |
| Comment by Gerrit Updater [ 29/Aug/16 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/18441/ |
| Comment by Joseph Gmitter (Inactive) [ 29/Aug/16 ] |
|
Landed to master for 2.9.0 |