[LU-983] On a 1.8.6 client/host, tar is performing 10K reads, and the RPCs are typically one page in size for small files==tar is slow for small files using 1.8.6 vs 1.8.5

Fri, 1 Jun 2012 14:31:33 +0000

This is all with respect to small files. Large files seem to be ok.

We have been focusing on tar, but suspect the problem is with small
file read performance in general on 1.8.6 clients. We are aware that
WC has a version of tar (~~LU-682~~), which we tried but it didn't help at all.

On a 1.8.5 host, tar is performing 10K reads, and the RPCs
are typically falling into the 128 page bin of the histogram. Tar performs
as expected.

On a 1.8.6 host, tar is performing 10K reads, and the RPCs are typically
one page in size. That's bad.

We are having some problems with "collectl" on 1.8.6 filesystems or we'd
have better data for you.

As far as real world issues, we have several complaints form user that are
doing small IO. Jay lan reported this to wc-discuss but we havne't seen
antyhing come across yet. Here is what he reported:

Our users reported a serious performance issue in 1.8.6. The time
needed to tar a directory of 14k files (total size 6.3G, tar file
is stripe size 30) is ~6 minutes in lustre1.8.5, but ~120 minutes
using 1.8.6. Our nas-1.8.6 is very close to 1.8.6-wc release, but
our nas-1.8.5 was based on LLNL's version.

Is there a known issue on tar'ing large number of small files in
1.8.6? I am aware of the lustre-tar and downloaded the rpm
from the Whamcloud site for our admin. That version does
not seem to help.

Whamcloud Community JIRA

[LU-983] On a 1.8.6 client/host, tar is performing 10K reads, and the RPCs are typically one page in size for small files==tar is slow for small files using 1.8.6 vs 1.8.5