https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-12179 Lustre <p><a href="https://jira.whamcloud.com/browse/LU-11347" title="Do not use pagecache for SSD I/O when read/write cache are disabled" class="issue-link" data-issue-key="LU-11347"><del>LU-11347</del></a>: Do not use pagecache for SSD I/O when read/write cache are disabled, </p> <p>this feature allocates the physical pages per OSS I/O thread in need at the runtime, it may result that  the many allocated pages are  not physical continuous. This may be not good for I/O merge in the block layer due to the limited segements per block I/O request.</p> <p>For testing purposes, we should pre-allocate continuous pages when OSS thread started as much as possible when disable page chace.</p> LU-12179

allocate continuous pages when disabled page caches

Improvement Minor Open Unresolved Qian Yingjin Qian Yingjin Thu, 11 Apr 2019 09:32:40 +0000 Thu, 11 Jun 2020 08:09:23 +0000 0 5 <p>Yingjin Qian (qian@ddn.com) uploaded a new patch: <a href="https://review.whamcloud.com/34644" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/34644</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-12179" title="allocate continuous pages when disabled page caches" class="issue-link" data-issue-key="LU-12179">LU-12179</a> osd: allocate continuous pages when disabled page caches<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 8e8d1d32ad55786735823135d6ca126f7fadf40d</p> <p>while I understand that SG constraints can prevent large IOs thus limit throughput, I don't see how this is related to pagecache bypassing. i.e. how pagecache can be doing better in this context.</p> <p>IIRC, Ihara did a series of tests using obdfilter-survery,  we found that the performace is bad especially when using 64MB RPC size, and SFA backend can not get large I/Os. After analyzed the debug logs, we found the reason is that the physical pages per I/O thread are not continuous reaching SG constranints.</p> <p>Current pagechache bypassing allocates pages in need at runtime, when many OSS I/O threads perform IOs concurrently, it may cause that the page allocation is interleaving between each I/O thread, thus many allocated pages per OSS I/O thread is not physical continuous.</p> <p>If we allocate the pages at the start time of OSS threads, it can  avoid the interleaving page allocation, I think. Thus the pages per I/O thread is much more physical-continnous.</p> <p> </p> <p>Hi Ihara,</p> <p>Did you do any testing with this patch? any results?</p> <p> </p> <p>Regards,</p> <p>Qian</p> <p> </p> <p>&gt; Current pagechache bypassing allocates pages in need at runtime</p> <p>only once for each thread.</p> <p>From the code it is once for each I/O if the allocated pages are not enough before reach PTLRPC_MAX__BRW_PAGES.</p> <p>If we allocate pages at the start time of I/O threads and start max_threads during OSS setup, then PTLRPC_MAX_BRW_PAGES pages for each I/O thread are allocated together and it is not interrupt and interleaved by other threads, thus can get much more number of continuous physical pages, I think.</p> <p> </p> <p>If allocating pages during I/O at runtime, then there may be many I/O thread needing to allocate page (via call __page_cache_alloc) for IO concurrently...  and obdfilter-suvery may test with I/O size varying from 1M to 64M, these all will make the allocated pyhiscal pages per I/O thread are not continuous...</p> <p>Now that there are preallocated BRW pages to avoid page cache overhead, since patch <a href="https://review.whamcloud.com/32875" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/32875</a> "<tt><a href="https://jira.whamcloud.com/browse/LU-11347" title="Do not use pagecache for SSD I/O when read/write cache are disabled" class="issue-link" data-issue-key="LU-11347"><del>LU-11347</del></a> osd: do not use pagecache for I/O</tt>", it probably makes sense to update the patch here so those pages can be allocated in larger chunks so that IO submission to the lower storage is more efficient.</p> Related LU-11347 LU-12071 LU-13309 Development Rank 1|i00erj: Rank (Obsolete) 9223372036854775807