[LU-12151] metadata performance difference on root and non-root user

• Type: Bug
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.13.0, Lustre 2.12.1
• Affects Version/s: Lustre 2.10.5
• Environment: lustre-2.10.5-RC2/ldiskfs
• Severity: 3
• Assignee: Wang Shilong (Inactive)
• Reporter: Shuichi Ihara

Description

We found a huge performance difference on file creation with root user and non-root user .

• 1 x MDS(1 x Platinum 8160, 96GB memory, EDR)
• 32 x client(2 x E5-2650 v4, 128GB memory, EDR)
• 1 x ES14K (40 x SSD)

root user

[root@c01 ~]# salloc -N 32 --ntasks-per-node=20 mpirun --allow-run-as-root /work/tools/bin/mdtest -n 1000 -F -v -u -d /scratch0/bmuser/ -C
SUMMARY: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation     :     151328.449     151328.449     151328.449          0.000
   File stat         :          0.000          0.000          0.000          0.000
   File read         :          0.000          0.000          0.000          0.000
   File removal      :          0.000          0.000          0.000          0.000
   Tree creation     :         42.057         42.057         42.057          0.000
   Tree removal      :          0.000          0.000          0.000          0.000
V-1: Entering timestamp...

Non-root user

[bmuser@c01 ~]$ salloc -N 32 --ntasks-per-node=20 mpirun --allow-run-as-root /work/tools/bin/mdtest -n 1000 -F -v -u -d /scratch0/bmuser/ -C
SUMMARY: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation     :     102825.662     102825.662     102825.662          0.000
   File stat         :          0.000          0.000          0.000          0.000
   File read         :          0.000          0.000          0.000          0.000
   File removal      :          0.000          0.000          0.000          0.000
   Tree creation     :         30.589         30.589         30.589          0.000
   Tree removal      :          0.000          0.000          0.000          0.000
V-1: Entering timestamp...

150K (root) vs 100K (non-root)

Issue Links

• is related to: LU-13239 ldiskfs: pass initial inode attributes at creation time

Comments

[Wang Shilong (Inactive)]

Hi,

Yup, that would be nice.

Thank you,
Shilong

[Andrew Perepechko]

Hi Wang Shilong,

unfortunately the patch dropped from the porting list and was forgotten for a while.

I'll measure how the xtime optimization improves performance in addition to the owner optimization and open a new ticket.

Are you ok with that?

Thank you

[Wang Shilong (Inactive)]

Hi Andrew Perepechko,

Yup, you guys have similar optimizations three years ago It is a pity that our Lustre upstream did not have similar thing for a long time.

Passing down xtime down could avoid us an extra ext4 inode dirty operation(which reduce jbd2 memory operations) even not huge improvements like this uid/gid but deserve us to do.

Do you agree a separate ticket for that?

Thank you,
Shilong

[Andrew Perepechko]

Passing xtimes (even with as low as 1 s resolution) can sometimes be beneficial as well: https://github.com/Xyratex/lustre-stable/commit/7ab00b00eb057f6963c0b5641686240ef95e1388#diff-89ce3dab611fea06ce62efa5bed4ae63

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/34685/
Subject: LU-12151 osd-ldiskfs: pass owner down rather than transfer it
Project: fs/lustre-release
Branch: b2_12
Current Patch Set:
Commit: f3d83215acd79ad062d3c605ca7dc8ba373be65d

[Gerrit Updater]

Minh Diep ([e-mail]) uploaded a new patch: https://review.whamcloud.com/34685
Subject: LU-12151 osd-ldiskfs: pass owner down rather than transfer it
Project: fs/lustre-release
Branch: b2_12
Current Patch Set: 1
Commit: a6996b311a4c852a8eeb68b684c046d00fbac127