Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-13413

Lustre soft lockups with peer credit exhaustion

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Major
    • None
    • Lustre 2.12.4
    • RHEL7.7 running Lustre 2.12 LTS
    • 3
    • 9223372036854775807

    Description

      Currently our production file system is experience peer credit exhaustion which is leading to soft locks. The back trace are attached. Instead of a soft lockups we should be having evicts.

      [2642432.333239]  [<ffffffff9477544a>] queued_spin_lock_slowpath+0xb/0xf
      [2642432.333241]  [<ffffffff94783330>] _raw_spin_lock+0x20/0x30
      [2642432.333258]  [<ffffffffc146202c>] lock_res_and_lock+0x2c/0x50 [ptlrpc]
      [2642432.333273]  [<ffffffffc1469c61>] ldlm_lock_enqueue+0x1b1/0xa20 [ptlrpc]
      [2642432.333294]  [<ffffffffc14b9891>] ? lustre_pack_reply+0x11/0x20 [ptlrpc]
      [2642432.333311]  [<ffffffffc1492506>] ldlm_handle_enqueue0+0xa56/0x15f0 [ptlrpc]
      [2642432.333332]  [<ffffffffc14bb300>] ? lustre_swab_ldlm_lock_desc+0x30/0x30 [ptlrpc]
      [2642432.333357]  [<ffffffffc151acf2>] tgt_enqueue+0x62/0x210 [ptlrpc]
      [2642432.333381]  [<ffffffffc1521b0a>] tgt_request_handle+0xada/0x1570 [ptlrpc]
      [2642432.333404]  [<ffffffffc14fb021>] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc]
      [2642432.333408]  [<ffffffffc103fbde>] ? ktime_get_real_seconds+0xe/0x10 [libcfs]
      [2642432.333428]  [<ffffffffc14c646b>] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc]
      [2642432.333449]  [<ffffffffc14c3285>] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc]
      [2642432.333451]  [<ffffffff940d3903>] ? __wake_up+0x13/0x20
      [2642432.333471]  [<ffffffffc14c9dd4>] ptlrpc_main+0xb34/0x1470 [ptlrpc]
      

       

      Attachments

        1. vmcore-dmesg.txt
          1.00 MB
          James A Simmons

        Issue Links

          Activity

            People

              adilger Andreas Dilger
              simmonsja James A Simmons
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: