Whamcloud Community JIRA

Whamcloud Community JIRA https://jira.whamcloud.com This file is an XML representation of an issue en-us

9.4.14

940014

05-12-2023

[LU-7181] Submitting random writes using 4MB RPC https://jira.whamcloud.com/browse/LU-7181 Lustre <p>After ran "lctl set_param osc.*.max_pages_per_rpc=4M", we ran 4k random write test with IOR and easily hit this problem.</p> <p>After some test, we found that the biggest value of msgsize is 16384 for bulk write, while req->rq_reqbuf_len is only 8192.</p> LU-7181

Submitting random writes using 4MB RPC

Improvement Minor Resolved Fixed WC Triage Andreas Dilger Thu, 17 Sep 2015 22:32:55 +0000 Wed, 8 Feb 2017 19:53:13 +0000 Wed, 8 Feb 2017 19:53:13 +0000 Lustre 2.7.0 Lustre 2.8.0 Lustre 2.9.0 0 4 <p>The patch from <a href="https://jira.whamcloud.com/browse/LU-4755" title="ASSERTION( req->rq_reqbuf_len >= msgsize ) failed when using 4MB RPC" class="issue-link" data-issue-key="LU-4755"><del>LU-4755</del></a> increased the OST_MAXREQSIZE to accommodate very large numbers of niobufs in a single request (up to 1024 with a 4MB RPC). However, per my last comments in <a href="https://jira.whamcloud.com/browse/LU-4755" title="ASSERTION( req->rq_reqbuf_len >= msgsize ) failed when using 4MB RPC" class="issue-link" data-issue-key="LU-4755"><del>LU-4755</del></a> it isn't clear whether there is an advantage to having so many small IOs in a single large RPC vs. having multiple separate RPCs in parallel.</p> <blockquote> <p>t is worthwhile to ask if there is any performance improvement from sending 4096 random pages in one RPC compared to 16 x 256 random pages in separate RPCs? It might even be faster to send parallel RPCs due to checksums running on separate cores and being handled in parallel on the OST. If there is no improvement from many random pages in one RPC, it is better to just limit the number of niobufs that the client sends in one RPC.</p></blockquote> <p>It would be useful to test a random write workload with 1MB, 4MB, and other RPC sizes to see if there is an improvement from sending multiple 1MB RPCs in parallel vs. larger single RPCs.</p> <p>This was fixed as part of patch <a href="http://review.whamcloud.com/22369" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/22369</a> "<a href="https://jira.whamcloud.com/browse/LU-8135" title="sanity test_101g fails with 'not all RPCs are 16 MiB BRW rpcs' " class="issue-link" data-issue-key="LU-8135"><del>LU-8135</del></a> osc: limits the number of chunks in write RPC".</p> <p>Landed as commit v2_8_58_0-8-g7f2aae8.</p> Related LU-4755 LU-8135 Development Rank 1|hzxo1j: Rank (Obsolete) 9223372036854775807 Severity