[LU-11623] Allow caching of open-created dentries Created: 06/Nov/18  Updated: 22/Jan/24

Status: Reopened
Project: Lustre
Component/s: None
Affects Version/s: Lustre 2.10.0, Lustre 2.11.0
Fix Version/s: Lustre 2.14.0, Lustre 2.12.6, Lustre 2.16.0

Type: Improvement Priority: Major
Reporter: Oleg Drokin Assignee: Lai Siyao
Resolution: Unresolved Votes: 1
Labels: None

Issue Links:
Duplicate
Related
is related to LU-10948 client cache open lock after N opens Open
is related to LU-15725 Client side Mdtest File Read Regressi... Open
Rank (Obsolete): 9223372036854775807

 Description   

Currently when creating/opening a file, we do not let the client hash the name so next access (frequently stat?) goes to talk on the MDS. This is also seen in various tests like mdtest.

We should consider returning more bits from the open call. This is in addition to LU-10948 that handles similar situation for recurring non-creating opens.

 Comments   
Comment by Gerrit Updater [ 06/Nov/18 ]

Oleg Drokin (green@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/33584
Subject: LU-11623 llite: hash jsut created files if lock allows
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: a72ed0f38312f3a036a51acdd4f0a9e93b1b3228

Comment by Gerrit Updater [ 06/Nov/18 ]

Oleg Drokin (green@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/33585
Subject: LU-11623 mdt: Opportunistically return UPDATE and PERM bits on open
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 1a59d230c86e5894d2d1148f786b9fa37ab904c5

Comment by Shuichi Ihara [ 28/May/19 ]

https://review.whamcloud.com/#/c/32157
https://review.whamcloud.com/#/c/33584
https://review.whamcloud.com/#/c/33585

The above three patches(mainly two patches 32157 and 33585) can significant improve 'stat' operation at one of workload that is open-create, then stat on same client.
This is one of common workload many people are doing on a single client. (e.g. create many files, then 'ls' command just for created files)
Here is test results with/without those three patches.
Test case is simple that create 240,000 x 32K files into a single directory, then 'ls -l'. I also ran mdtest in parallel for 'create' and 'stat' for those 1.2M files in addition to 'ls -l' test.

Without patch(master branch commit 26a7abe)

[root@c01 ~]# mpirun -np 24 --allow-run-as-root /work/tools/bin/mdtest -n 10000 -d /cache1/out/ -F -C -v -w 32k 
[root@c01 ~]# time ls -l /cache1/out/#test-dir.0-0/mdtest_tree.0 > /dev/null

real	0m23.447s
user	0m1.010s
sys	0m11.916s

[root@c01 ~]# mpirun -np 24 --allow-run-as-root /work/tools/bin/mdtest -n 50000 -d /cache1/out/ -F -C -T -v -w 32k
SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation     :       3838.205       3838.204       3838.204          0.000
   File stat         :      33459.289      33459.249      33459.271          0.011
   File read         :          0.000          0.000          0.000          0.000
   File removal      :          0.000          0.000          0.000          0.000
   Tree creation     :       3146.841       3146.841       3146.841          0.000
   Tree removal      :          0.000          0.000          0.000          0.000

With three patches agianst master (commit 26a7abe)

[root@c01 ~]# mpirun -np 24 --allow-run-as-root /work/tools/bin/mdtest -n 10000 -d /cache1/out/ -F -C -v -w 32k 
[root@c01 ~]# time ls -l /cache1/out/#test-dir.0-0/mdtest_tree.0 > /dev/null

real	0m5.780s
user	0m0.635s
sys	0m4.996s

[root@c01 ~]#  mpirun -np 24 --allow-run-as-root /work/tools/bin/mdtest -n 50000 -d /cache1/out/ -F -C -T -v -w 32k
SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation     :       3822.440       3822.439       3822.440          0.000
   File stat         :     350620.140     350615.980     350617.193          1.051
   File read         :          0.000          0.000          0.000          0.000
   File removal      :          0.000          0.000          0.000          0.000
   Tree creation     :       2076.727       2076.727       2076.727          0.000
   Tree removal      :          0.000          0.000          0.000          0.000
Comment by Gerrit Updater [ 01/Jun/19 ]

Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/33584/
Subject: LU-11623 llite: hash just created files if lock allows
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: fc42cbe0e2e5d1d87d0edca73986b831ac718301

Comment by Gerrit Updater [ 05/Jun/19 ]

Oleg Drokin (green@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/35067
Subject: LU-11623 tests: Fix sanity 27E to ensure getattr RPC
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 5f8705f8164e4cdaf22d96e5b37ae1bd362b7d3f

Comment by Gerrit Updater [ 16/Jun/19 ]

Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/35067/
Subject: LU-11623 tests: Fix sanity 27E to ensure getattr RPC
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 892c2251d854f480c272ab2999cdc6f4c87fecf0

Comment by Gerrit Updater [ 28/May/20 ]

Lai Siyao (lai.siyao@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/38750
Subject: LU-11623 llite: hash just created files if lock allows
Project: fs/lustre-release
Branch: b2_12
Current Patch Set: 1
Commit: 72ba58af89946e0e4b3743262bacc6af315483fa

Comment by Gerrit Updater [ 29/May/20 ]

Lai Siyao (lai.siyao@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/38763
Subject: LU-11623 llite: hash just created files if lock allows
Project: fs/lustre-release
Branch: b2_12
Current Patch Set: 1
Commit: 1922e2b202c9758f60dd1d72b2099302c0c0809c

Comment by Gerrit Updater [ 30/Jun/20 ]

Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/38763/
Subject: LU-11623 llite: hash just created files if lock allows
Project: fs/lustre-release
Branch: b2_12
Current Patch Set:
Commit: 7554658056cddae71fd90ef4a6dd639ddbe3f89b

Comment by Peter Jones [ 11/Jul/20 ]

Do I understand correctly that the work tracked under this ticket landed to 2.13 and 2.12.6?

Comment by Peter Jones [ 09/Dec/20 ]

Landed for 2.14 and 2.12.6

Comment by Shuichi Ihara [ 02/Jul/21 ]

I would open this ticket since patch https://review.whamcloud.com/#/c/33585 was not merged yet. without this patch, what I showed the performance numbers can't be demonstrated.

Comment by Gerrit Updater [ 28/Apr/22 ]

"Lai Siyao <lai.siyao@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/47163
Subject: LU-11623 mdt: return UPDATE|PERM on open
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 509656f89bd967395da859ea123875ca5db9931c

Comment by Shuichi Ihara [ 20/Apr/23 ]

latest patch set (patch set 29) of https://review.whamcloud.com/#/c/fs/lustre-release/+/33585/ patch still demonstrated and promised 9x (86,466 ops vs 789,729 ops) performance boost on open()-create()-stat() workload. And this is same as we are seeing same cached performance on stat()-stat() workload toady. Here is performance comparisons with/without patch.

without patch(commit:8beeec77c3)

mpirun -np 16 mdtest -n 10000 -d /exafs/d0/d1/d2/ -F -C -v -w 32k
mpirun -np 16 mdtest -n 10000 -d /exafs/d0/d1/d2/ -F -T -v -w 32k
SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :      86466.293      86466.272      86466.285          0.006
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

mpirun -np 16 mdtest -n 10000 -d /exafs/d0/d1/d2/ -F -T -v -w 32k
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :     779470.812     779428.353     779442.415         18.312
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

With patch (https://review.whamcloud.com/#/c/fs/lustre-release/+/33585/)

mpirun -np 16 mdtest -n 10000 -d /exafs/d0/d1/d2/ -F -C -v -w 32k
mpirun -np 16 mdtest -n 10000 -d /exafs/d0/d1/d2/ -F -T -v -w 32k
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :     789729.392     789726.612     789727.879          0.702
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
mpirun -np 16 mdtest -n 10000 -d /exafs/d0/d1/d2/ -F -T -v -w 32k
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :     790002.623     790000.712     790001.644          0.633
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
Comment by Gerrit Updater [ 19/May/23 ]

"Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/33585/
Subject: LU-11623 mdt: return UPDATE|PERM on open
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 4b44204930cf4d312425dd95e51720fe5d3e801b

Comment by Peter Jones [ 19/May/23 ]

Latest patch merged for 2.16. TBH it seems like it would have been better to track this latest work under a new linked Jira than to use this several years old ticket.

Generated at Sat Feb 10 02:45:29 UTC 2024 using Jira 9.4.14#940014-sha1:734e6822bbf0d45eff9af51f82432957f73aa32c.