[LU-4080] Troubleshooting poor filesystem performance

• Type: Task
• Resolution: Fixed
• Priority: Major
• Affects Version/s: Lustre 2.1.3
• Environment: RHEL 6.4, Lustre client kernel 2.6.32-279.2.1_x86-64; 96TB filesystem comprised of 6 x 16 TB OSTs on 2 OSS - 3 OSTs per OSS
• Assignee: Oleg Drokin
• Reporter: George Jackson (Inactive)

Description

On a Lustre client with 96 TB filesystem mounted over Infiniband, we seem to be having write performance issues and need assistance with troubleshooting. On this particular client, we receive ~600,000 binary data files daily from an external source. Each file is anywhere from 100 bytes to 1 MB. These small files are then appended to data files already on the system from previous writes/appends.

The speed at which the files are appended is very poor compared to a less powerful system we have that runs an old version of HP's IBRIX filesystem. What commands or troubleshooting steps can I take to find out where the bottleneck is in processing?

Note this is a DOD classified system no connected to the internet so supplying logs may be problematic. I will be glad to answer any questions about the filesystem layout and configuration itself as well as discuss output from any commands you recommend I run.

Thanks,
George Jackson