[LU-7181] Submitting random writes using 4MB RPC Created: 17/Sep/15  Updated: 08/Feb/17  Resolved: 08/Feb/17

Status: Resolved
Project: Lustre
Component/s: None
Affects Version/s: Lustre 2.7.0, Lustre 2.8.0
Fix Version/s: Lustre 2.9.0

Type: Improvement Priority: Minor
Reporter: Andreas Dilger Assignee: WC Triage
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
Related
is related to LU-4755 ASSERTION( req->rq_reqbuf_len >= msgs... Resolved
is related to LU-8135 sanity test_101g fails with 'not all ... Resolved
Severity: 3
Rank (Obsolete): 9223372036854775807

 Description   

After ran "lctl set_param osc.*.max_pages_per_rpc=4M", we ran 4k random write test with IOR and easily hit this problem.

After some test, we found that the biggest value of msgsize is 16384 for bulk write, while req->rq_reqbuf_len is only 8192.



 Comments   
Comment by Andreas Dilger [ 17/Sep/15 ]

The patch from LU-4755 increased the OST_MAXREQSIZE to accommodate very large numbers of niobufs in a single request (up to 1024 with a 4MB RPC). However, per my last comments in LU-4755 it isn't clear whether there is an advantage to having so many small IOs in a single large RPC vs. having multiple separate RPCs in parallel.

t is worthwhile to ask if there is any performance improvement from sending 4096 random pages in one RPC compared to 16 x 256 random pages in separate RPCs? It might even be faster to send parallel RPCs due to checksums running on separate cores and being handled in parallel on the OST. If there is no improvement from many random pages in one RPC, it is better to just limit the number of niobufs that the client sends in one RPC.

It would be useful to test a random write workload with 1MB, 4MB, and other RPC sizes to see if there is an improvement from sending multiple 1MB RPCs in parallel vs. larger single RPCs.

Comment by Andreas Dilger [ 08/Feb/17 ]

This was fixed as part of patch http://review.whamcloud.com/22369 "LU-8135 osc: limits the number of chunks in write RPC".

Comment by Andreas Dilger [ 08/Feb/17 ]

Landed as commit v2_8_58_0-8-g7f2aae8.

Generated at Sat Feb 10 02:06:41 UTC 2024 using Jira 9.4.14#940014-sha1:734e6822bbf0d45eff9af51f82432957f73aa32c.