https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-16613 Lustre <p>Prototype support for folio_batch support on page cache backed buffer I/O path.</p> <p>Relies on a patched kernel to provide:</p> <ul class="alternate" type="square"> <li>generic_perform_batch_write</li> <li>aops-&gt;write_batch_begin and aops-&gt;write_patch_end</li> <li>filemap_dirty_folio_batched</li> <li>grab_cache_folios_fast</li> </ul> LU-16613

Future: folio_batch BIO write path

Improvement Minor Open Unresolved Shaun Tancheff Shaun Tancheff Fri, 3 Mar 2023 07:03:36 +0000 Wed, 8 Mar 2023 10:01:40 +0000 0 5 <p>"Shaun Tancheff &lt;shaun.tancheff@hpe.com&gt;" uploaded a new patch: <a href="https://review.whamcloud.com/c/fs/lustre-release/+/50194" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/c/fs/lustre-release/+/50194</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-16613" title="Future: folio_batch BIO write path" class="issue-link" data-issue-key="LU-16613">LU-16613</a> clio: Use folio_batch write path<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 6dc4f8a03385bd507f5ebb5d67503c10e05053c7</p> <p>Shaun,</p> <p>This is very exciting, I didn’t know the requisite folio stuff existed yet - have you been able to benchmark this at all?  And is there some form of read batch available as well?  The read path is also stuck in page cache work as its main performance limitation.</p> <p>Ah, I see it requires a patched kernel.  Can you say more about this in general? Where are these patches from, are they expected upstream, etc, etc</p> <p>I have patches here:<br/> <a href="https://github.com/stancheff/linux/" class="external-link" target="_blank" rel="nofollow noopener">https://github.com/stancheff/linux/</a><br/> branch: dedupe-v1</p> <p>The basic idea is to enable 'batch' adding to the page cache (and batch dirty pages). I did the first pass on the write path.</p> <p>There is a device mapper that can be used to do local perf testing, ex:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>sudo mkdir -p /vol/fcp echo "0 `sudo blockdev --getsz /dev/vda` dedupe /dev/vda 0" | sudo dmsetup create dedupe sudo mkfs.ext4 -F -E 'lazy_itable_init=0 packed_meta_blocks=1' /dev/mapper/dedupe sudo mount /dev/mapper/dedupe /vol/fcp sudo fio --filename=/vol/fcp/fio.bin \ --size=100GB \ --buffered=1 \ --rw=write \ --bs=64k \ --buffer_pattern=0x00000000 \ --ioengine=sync \ --iodepth=64 \ --runtime=120 \ --numjobs=16 \ --time_based \ --group_reporting \ --name=throughput-test-job \ --eta-newline=1 </pre> </div></div> <p>Locally I see 2 G/s buffered. Still a long ways from direct I/O which is closer to 12G/s</p> <p>The read path is batch'd:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre> generic_file_read_iter() filemap_read() filemap_get_pages() </pre> </div></div> Development Rank 1|i03fl3: Rank (Obsolete) 9223372036854775807