https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-16789 Lustre <p>It would be useful to speed up "<tt>lfs migrate</tt>", "<tt>lfs mirror extend</tt>", and "<tt>lfs mirror resync</tt>" to use asynchronous Direct IO (AIO/DIO via libaio) to do the data copying</p> <p>This should use the same mechanism as <tt>lustre/tests/aiocp.c</tt> to have a producer/consumer queue and submits some number of AIO read requests, and then submit the write requests when the reads finish.</p> LU-16789

"lfs migrate" to use AIO/DIO

Improvement Minor Open Unresolved WC Triage Andreas Dilger lad23dd lug23dd medium Mon, 1 May 2023 17:49:24 +0000 Sat, 20 Jan 2024 00:50:58 +0000 0 6 <p>I think we should actually aim this at io_uring.  <a href="https://jira.whamcloud.com/secure/ViewProfile.jspa?name=simmonsja" class="user-hover" rel="simmonsja">simmonsja</a> says there is a library which allows you to write against io_uring and have it use AIO for backwards compatibility with older kernels (eg, RHEL 8 which is still a major target).</p> <p>Note for anyone doing this - These changes should be made to migrate_copy_data(), which can then be shared out.  See <a href="https://jira.whamcloud.com/browse/LU-17143" title="Use migrate_copy_data() for all data movement" class="issue-link" data-issue-key="LU-17143">LU-17143</a>.</p> <p>Just to leave a note, at ORNL we implemented a replacement for lfs mirgate using a library that uses IO ring under the hood. Need to ask Chris Brumgard.</p> <p><a href="https://jira.whamcloud.com/secure/ViewProfile.jspa?name=simmonsja" class="user-hover" rel="simmonsja">simmonsja</a> any word from Chris B?</p> <p>As per James, this work has not been tackled at ORNL yet so this task is still available for someone else to work on</p> Related LU-17143 Development Rank 1|i03kd3: Rank (Obsolete) 9223372036854775807