Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-12651

High kworker CPU usage (osc_grant_work_handler) on IDLE connections

Details

    • 3
    • 9223372036854775807

    Description

      We discovered that on our systems with lustre mounted, the kworker is using signifficant amount of CPU.
      perf top shows on an idle system:

       39.44%  [kernel]                  [k] osc_should_shrink_grant
        12.14%  [kernel]                  [k] osc_grant_work_handler
         2.81%  [kernel]                  [k] process_one_work
         2.64%  [kernel]                  [k] __queue_work
         2.56%  [kernel]                  [k] read_tsc
      

      We currently have grant_shrink=0 on this system.

      Looks like doing just du -hs /fs makes the problem go away for some time.
      Also unmounting the filesystem makes the problem go away.
      This is Centos 7.6 system with Lustre 2.12.0

      Attachments

        Issue Links

          Activity

            [LU-12651] High kworker CPU usage (osc_grant_work_handler) on IDLE connections
            pjones Peter Jones made changes -
            Link New: This issue is related to JFC-20 [ JFC-20 ]
            pjones Peter Jones made changes -
            Link Original: This issue is related to JFC-20 [ JFC-20 ]
            pjones Peter Jones made changes -
            Link New: This issue is related to JFC-20 [ JFC-20 ]
            pjones Peter Jones made changes -
            Fix Version/s New: Lustre 2.12.5 [ 14696 ]
            pjones Peter Jones made changes -
            Resolution New: Fixed [ 1 ]
            Status Original: Open [ 1 ] New: Resolved [ 5 ]
            pjones Peter Jones made changes -
            Labels New: LTS12
            pjones Peter Jones made changes -
            Fix Version/s New: Lustre 2.14.0 [ 14490 ]
            adilger Andreas Dilger made changes -
            Link New: This issue is related to LU-8708 [ LU-8708 ]
            adilger Andreas Dilger made changes -
            Link New: This issue is duplicated by DDN-1032 [ DDN-1032 ]
            pjones Peter Jones made changes -
            Assignee Original: WC Triage [ wc-triage ] New: Alexander Zarochentsev [ zam ]
            Tomaka Jacek Tomaka (Inactive) created issue -

            People

              zam Alexander Zarochentsev
              Tomaka Jacek Tomaka (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: