[LU-7828] Traversal directory is very slow

• Status: Resolved
• Project: Lustre
• Affects Version/s: Lustre 2.7.0, Lustre 2.8.0
• Fix Version/s: Lustre 2.9.0
• Type: Bug
• Priority: Critical
• Reporter: nasf (Inactive)
• Assignee: nasf (Inactive)
• Resolution: Fixed
• Severity: 3

Description

Some customer found that:
Several customers said "ls" is slow with latest Lustre codes.
We tried to figure out detail and the following codes are easy to reproudce at the cutomer site. But, if we changed "ls -F" with "ls -F *", the problem didn't reproduce.

#!/bin/bash

for ((i=0;i<2583;i++)); do 
        dd if=/dev/zero of=file$i.dat bs=1024 count=400 > /dev/null 2>&1 
done

for ((i=0;i<20;i++)); do
        (time ls -F > /dev/null) >> lscheck.txt
done

rm -f file*.dat
echo ""
cat lscheck.txt
rm -f lscheck.txt
exit

Above test is just repeating "ls -F" 10 times after 2583 file creation.
Please see below. The most of cases finish quickly (less than 1 sec), but sevral cases, it took 30 sec.
And, one of strange thing, alltime it takes 30 sec or 30 sec x N.

This is 100% reproducable and we trying strace or ftrace to find where is most highest cost of system call.
However, when if we run "strace -f ls -F", the problem never reproduced which is also strange.

But, the problem only reproduced on the Lustre..

Comments

[nasf (Inactive)]

Here is the the patch:
http://review.whamcloud.com/18499

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch http://review.whamcloud.com/18499/
Subject: LU-7828 statahead: set sai_index_wait with lli_sa_lock held
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 35fb132546ff45cd4b3f08dd9feaa6f69ad02b3a

[Joseph Gmitter (Inactive)]

Landed for 2.9.0