https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-11618 Lustre <p>Originally, the maximum RPC size for bulk I/O can be controlled by<br/> per-OSC parameter max_pages_per_rpc. Whenever it is possible, the<br/> OSC will do bulk I/O as large as max_pages_per_rpc for better<br/> performance. Thus, changing the value of max_pages_per_rpc usually<br/> affects the I/O performance a lot. However, due to the I/O pattern<br/> difference, not all applications can get the best performance with<br/> the same value of max_pages_per_rpc.</p> <p>We want to add a new type of ladvise to enabling applications to<br/> set different RPC sizes for different files. max_pages_per_rpc is<br/> still the upper limit of the RPC size in all cases. And new<br/> parameter default_pages_per_rpc has been added and its value is<br/> the default RPC size. If a ladvise of rpc_size is given to a file,<br/> the RPC size of the file will be changed according to the ladvise.<br/> But the maximum RPC size will still limited by max_pages_per_rpc.</p> <p>The RPC size of a file configured by ladivse is neither a global<br/> attribute nor a persistent attribute. Each client may have<br/> different RPC size for the same file. And the RPC size of the file<br/> will change back to default_pages_per_rpc when the hint kept in<br/> memory is lost due to memory shrinkage.</p> LU-11618

implement ladvise rpc_size for optimized performance

New Feature Minor Open Unresolved Li Xi Li Xi Mon, 5 Nov 2018 02:19:51 +0000 Wed, 7 Nov 2018 07:05:49 +0000 0 5 <p>Li Xi (lixi@ddn.com) uploaded a new patch: <a href="https://review.whamcloud.com/33573" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/33573</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11618" title="implement ladvise rpc_size for optimized performance" class="issue-link" data-issue-key="LU-11618">LU-11618</a> osc: implement ladvise rpc_size<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: ce387d26f369dec24496eaed4f9213b76bb3f47f</p> <p>In the past, the RPC size was also affected by the stripe size, so that applications could specify this on a per-file basis. Also, storing the stripe_size in the file is persistent, and does not require the application to be modified. </p> <p>Thanks Andreas. Has this mechanism of stripe-based-RPC-size being changed? I can't find any related codes now.</p> <p>We are thinking of using ladvise hint through MPI ROMIO like <a href="https://jira.whamcloud.com/browse/LU-6179" title="Lock ahead - Request extent locks from userspace" class="issue-link" data-issue-key="LU-6179"><del>LU-6179</del></a>. That would require some arguments when calling MPI run command, but doesn't require modification of the appliation. And we are tired of tuning the global RPC size parameter to get good performance.</p> <p> </p> <p>Following is how to use it:</p> <p>MGS<br/> $ lctl conf_param lipe1-OST*.obdfilter.brw_size=16<br/> $ lctl conf_param lipe1-OST*.osc.max_pages_per_rpc=16M</p> <p> Client<br/> $ mount -t lustre 10.0.1.148@tcp:10.0.1.149@tcp:/lipe1 /mnt/lustre_lipe1<br/> $ cat /proc/fs/lustre/osc/lipe1-OST0000-osc-*/default_pages_per_rpc<br/> 4096<br/> $ cat /proc/fs/lustre/osc/lipe1-OST0000-osc-*/max_pages_per_rpc <br/> 4096<br/> $ lfs setstripe -c 1 -i 0 /mnt/lustre_lipe1/file<br/> $ dd if=/dev/zero of=/mnt/lustre_lipe1/file bs=1048576 count=10<br/> $ cat /proc/fs/lustre/osc/lipe1-OST0000-osc-*/rpc_stats<br/> ...<br/> read write<br/> pages per rpc rpcs % cum % | rpcs % cum %<br/> 1: 0 0 0 | 0 0 0<br/> 2: 0 0 0 | 0 0 0<br/> 4: 0 0 0 | 0 0 0<br/> 8: 0 0 0 | 0 0 0<br/> 16: 0 0 0 | 0 0 0<br/> 32: 0 0 0 | 0 0 0<br/> 64: 0 0 0 | 0 0 0<br/> 128: 0 0 0 | 0 0 0<br/> 256: 0 0 0 | 0 0 0<br/> 512: 0 0 0 | 0 0 0<br/> 1024: 0 0 0 | 0 0 0<br/> 2048: 0 0 0 | 0 0 0<br/> 4096: 0 0 0 | 1 100 100<br/> ...<br/> $ echo &gt; /proc/fs/lustre/osc/lipe1-OST0000-osc-*/rpc_stats<br/> $ echo 256 &gt; /proc/fs/lustre/osc/lipe1-OST0000-osc-*/default_pages_per_rpc<br/> $ dd if=/dev/zero of=/mnt/lustre_lipe1/file bs=1048576 count=10<br/> $ cat /proc/fs/lustre/osc/lipe1-OST0000-osc-*/rpc_stats<br/> ...<br/> read write<br/> pages per rpc rpcs % cum % | rpcs % cum %<br/> 1: 0 0 0 | 0 0 0<br/> 2: 0 0 0 | 0 0 0<br/> 4: 0 0 0 | 0 0 0<br/> 8: 0 0 0 | 0 0 0<br/> 16: 0 0 0 | 0 0 0<br/> 32: 0 0 0 | 0 0 0<br/> 64: 0 0 0 | 0 0 0<br/> 128: 0 0 0 | 0 0 0<br/> 256: 0 0 0 | 10 100 100<br/> ...<br/> $ echo &gt; /proc/fs/lustre/osc/lipe1-OST0000-osc-*/rpc_stats<br/> $ lfs ladvise -a rpcsize -r 16M /mnt/lustre_lipe1/file<br/> $ dd if=/dev/zero of=/mnt/lustre_lipe1/file bs=1048576 count=16<br/> $ cat /proc/fs/lustre/osc/lipe1-OST0000-osc-*/rpc_stats<br/> ...<br/> read write<br/> pages per rpc rpcs % cum % | rpcs % cum %<br/> 1: 0 0 0 | 0 0 0<br/> 2: 0 0 0 | 0 0 0<br/> 4: 0 0 0 | 0 0 0<br/> 8: 0 0 0 | 0 0 0<br/> 16: 0 0 0 | 0 0 0<br/> 32: 0 0 0 | 0 0 0<br/> 64: 0 0 0 | 0 0 0<br/> 128: 0 0 0 | 0 0 0<br/> 256: 0 0 0 | 0 0 0<br/> 512: 0 0 0 | 0 0 0<br/> 1024: 0 0 0 | 0 0 0<br/> 2048: 0 0 0 | 0 0 0<br/> 4096: 0 0 0 | 1 100 100<br/> ...<br/> $ echo &gt; /proc/fs/lustre/osc/lipe1-OST0000-osc-*/rpc_stats<br/> $ lfs ladvise -a rpcsize -r 4M /mnt/lustre_lipe1/file<br/> $ dd if=/dev/zero of=/mnt/lustre_lipe1/file bs=1048576 count=16<br/> $ cat /proc/fs/lustre/osc/lipe1-OST0000-osc-*/rpc_stats<br/> ...<br/> read write<br/> pages per rpc rpcs % cum % | rpcs % cum %<br/> 1: 0 0 0 | 0 0 0<br/> 2: 0 0 0 | 0 0 0<br/> 4: 0 0 0 | 0 0 0<br/> 8: 0 0 0 | 0 0 0<br/> 16: 0 0 0 | 0 0 0<br/> 32: 0 0 0 | 0 0 0<br/> 64: 0 0 0 | 0 0 0<br/> 128: 0 0 0 | 0 0 0<br/> 256: 0 0 0 | 0 0 0<br/> 512: 0 0 0 | 0 0 0<br/> 1024: 0 0 0 | 4 100 100<br/> ...</p> Related LU-6179 Development Rank 1|i005p3: Rank (Obsolete) 9223372036854775807