[LU-9782] High CPU usage with random IO test. Created: 19/Jul/17  Updated: 26/Oct/17  Resolved: 24/Oct/17

Status: Resolved
Project: Lustre
Component/s: None
Affects Version/s: Lustre 2.7.0, Lustre 2.8.0, Lustre 2.9.0, Lustre 2.10.0
Fix Version/s: Lustre 2.11.0, Lustre 2.10.2

Type: Bug Priority: Critical
Reporter: Alexey Lyashkov Assignee: WC Triage
Resolution: Fixed Votes: 0
Labels: ldiskfs
Environment:

any lustre with commit 144b5a65c16, and likely before.


Issue Links:
Related
Severity: 3
Rank (Obsolete): 9223372036854775807

 Description   

osd-ldiskfs have a several osd_is_mapped calls when extent tree scanned to get decission about error for rewrite case. But extent tree is huge with while random io write test and search is CPU expensive in this case. typical perf output is

|--75.99%-- rb_next
            |          |          
            |          |--94.49%-- ldiskfs_es_find_delayed_extent_range
            |          |          ldiskfs_fiemap
            |          |          osd_is_mapped
            |          |          osd_declare_write_commit
            |          |          ofd_commitrw_write.isra.32
            |          |          ofd_commitrw
            |          |          obd_commitrw.constprop.39
            |          |          tgt_brw_write
            |          |          tgt_request_handle
            |          |          ptlrpc_server_handle_request
            |          |          ptlrpc_main
            |          |          kthread
            |          |          ret_from_fork
            |          |          
            |          |--5.49%-- ldiskfs_fiemap
            |          |          osd_is_mapped
            |          |          osd_declare_write_commit
            |          |          ofd_commitrw_write.isra.32
            |          |          ofd_commitrw
            |          |          obd_commitrw.constprop.39
            |          |          tgt_brw_write
            |          |          tgt_request_handle
            |          |          ptlrpc_server_handle_request
            |          |          ptlrpc_main
            |          |          kthread
            |          |          ret_from_fork
            |           --0.02%-- [...]
            |          
            |--21.80%-- ldiskfs_es_find_delayed_extent_range
            |          |          
            |          |--100.00%-- ldiskfs_fiemap
            |          |          osd_is_mapped
            |          |          osd_declare_write_commit
            |          |          ofd_commitrw_write.isra.32
            |          |          ofd_commitrw
            |          |          obd_commitrw.constprop.39
            |          |          tgt_brw_write
            |          |          tgt_request_handle
            |          |          ptlrpc_server_handle_request
            |          |          ptlrpc_main
            |          |          kthread
            |          |          ret_from_fork
            |           --0.00%-- [...]

avoid second search by caching result on hight bits of lnb_flags which not used in wire increase a performance dramatically

  • without patch with default mkfsoptions: CPU usage 100%​ with 55-60K random 4K writes
  • with the patch: CPU usage 50-60% with 250K random 4K writes


 Comments   
Comment by Gerrit Updater [ 20/Jul/17 ]

Parinay Kondekar (parinay.kondekar@seagate.com) uploaded a new patch: https://review.whamcloud.com/28145
Subject: LU-9782 osd-ldiskfs: avoid extra search
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: b0463e3ad8dd664ac409f10b80187d2cfcbbbf81

Comment by Gerrit Updater [ 24/Oct/17 ]

Oleg Drokin (oleg.drokin@intel.com) merged in patch https://review.whamcloud.com/28145/
Subject: LU-9782 osd-ldiskfs: avoid extra search
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 1eb0573fdeb7ef8ed667974fd30d4d110b8166c9

Comment by Peter Jones [ 24/Oct/17 ]

Landed for 2.11

Comment by Gerrit Updater [ 24/Oct/17 ]

Minh Diep (minh.diep@intel.com) uploaded a new patch: https://review.whamcloud.com/29739
Subject: LU-9782 osd-ldiskfs: avoid extra search
Project: fs/lustre-release
Branch: b2_10
Current Patch Set: 1
Commit: 90140d6851c86802aed08fba9df0454c229cf39d

Comment by Gerrit Updater [ 26/Oct/17 ]

John L. Hammond (john.hammond@intel.com) merged in patch https://review.whamcloud.com/29739/
Subject: LU-9782 osd-ldiskfs: avoid extra search
Project: fs/lustre-release
Branch: b2_10
Current Patch Set:
Commit: 7d450f590df3deafde7d90d1a1bff6398df0b99b

Generated at Sat Feb 10 02:29:12 UTC 2024 using Jira 9.4.14#940014-sha1:734e6822bbf0d45eff9af51f82432957f73aa32c.