Details

    • Bug
    • Resolution: Fixed
    • Major
    • Lustre 2.12.0
    • Lustre 2.12.0
    • None
    • 3
    • 9223372036854775807

    Description

      Had this trigger in racer in current master-next, but does not appear to be caused by anything unlanded.

      [ 2800.273242] LustreError: 20863:0:(tgt_grant.c:151:tgt_check_export_grants()) lustre-OST0002: cli e57bac33-ee31-2bdc-225e-2658736d80ff/ffff880251df1800 ted_grant(1142554624) + ted_pending(0) > maxsize(250609664)
      [ 2800.312893] LustreError: 20863:0:(tgt_grant.c:223:tgt_grant_sanity_check()) LBUG
      [ 2800.319055] Pid: 20863, comm: ll_ost_create07 3.10.0-7.5-debug #1 SMP Sun Jun 3 13:35:38 EDT 2018
      [ 2800.321344] Call Trace:
      [ 2800.322479]  [<ffffffffa01cd7dc>] libcfs_call_trace+0x8c/0xc0 [libcfs]
      [ 2800.327468]  [<ffffffffa01cd88c>] lbug_with_loc+0x4c/0xa0 [libcfs]
      [ 2800.328911]  [<ffffffffa0bbbf7c>] tgt_grant_sanity_check+0x51c/0x550 [ptlrpc]
      [ 2800.333566]  [<ffffffffa127dc14>] ofd_statfs+0x104/0x480 [ofd]
      [ 2800.334982]  [<ffffffffa12712e0>] ofd_statfs_hdl+0x70/0x280 [ofd]
      [ 2800.335938] LustreError: 3246:0:(tgt_grant.c:151:tgt_check_export_grants()) lustre-OST0002: cli e57bac33-ee31-2bdc-225e-2658736d80ff/ffff880251df1800 ted_grant(1142554624) + ted_pending(0) > maxsize(250609664)
      [ 2800.335961] LustreError: 3246:0:(tgt_grant.c:223:tgt_grant_sanity_check()) LBUG
      [ 2800.344690]  [<ffffffffa0ba0705>] tgt_request_handle+0xaf5/0x1590 [ptlrpc]
      [ 2800.348268]  [<ffffffffa0b44e26>] ptlrpc_server_handle_request+0x256/0xad0 [ptlrpc]
      [ 2800.350975]  [<ffffffffa0b48c1e>] ptlrpc_main+0xabe/0x1f80 [ptlrpc]
      [ 2800.352543]  [<ffffffff810ae864>] kthread+0xe4/0xf0
      [ 2800.354434]  [<ffffffff81783777>] ret_from_fork_nospec_end+0x0/0x39
      [ 2800.356861]  [<ffffffffffffffff>] 0xffffffffffffffff
      [ 2800.360929] Kernel panic - not syncing: LBUG
      [ 2800.360945] Pid: 3246, comm: ll_ost_create00 3.10.0-7.5-debug #1 SMP Sun Jun 3 13:35:38 EDT 2018
      [ 2800.360945] Call Trace:
      [ 2800.360972]  [<ffffffffa01cd7dc>] libcfs_call_trace+0x8c/0xc0 [libcfs]
      [ 2800.360977]  [<ffffffffa01cd88c>] lbug_with_loc+0x4c/0xa0 [libcfs]
      [ 2800.361073]  [<ffffffffa0bbbf7c>] tgt_grant_sanity_check+0x51c/0x550 [ptlrpc]
      [ 2800.361090]  [<ffffffffa127dc14>] ofd_statfs+0x104/0x480 [ofd]
      [ 2800.361093]  [<ffffffffa12712e0>] ofd_statfs_hdl+0x70/0x280 [ofd]
      [ 2800.361125]  [<ffffffffa0ba0705>] tgt_request_handle+0xaf5/0x1590 [ptlrpc]
      [ 2800.361152]  [<ffffffffa0b44e26>] ptlrpc_server_handle_request+0x256/0xad0 [ptlrpc]
      [ 2800.361190]  [<ffffffffa0b48c1e>] ptlrpc_main+0xabe/0x1f80 [ptlrpc]
      [ 2800.361196]  [<ffffffff810ae864>] kthread+0xe4/0xf0
      [ 2800.361200]  [<ffffffff81783777>] ret_from_fork_nospec_end+0x0/0x39
      [ 2800.361203]  [<ffffffffffffffff>] 0xffffffffffffffff
      

      Attachments

        Issue Links

          Activity

            [LU-11288] tgt_grant_sanity_check()) LBUG
            pjones Peter Jones added a comment -

            Landed for 2.12

            pjones Peter Jones added a comment - Landed for 2.12

            Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/33226/
            Subject: LU-11288 osc: re-check target versus available grant
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: fcbd8c981239dc2be4bcf55e9a3e40b72d939700

            gerrit Gerrit Updater added a comment - Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/33226/ Subject: LU-11288 osc: re-check target versus available grant Project: fs/lustre-release Branch: master Current Patch Set: Commit: fcbd8c981239dc2be4bcf55e9a3e40b72d939700

            Alex Zhuravlev (bzzz@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/33226
            Subject: LU-11288 osc: check target versus available grant
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: ec68ff8b3d389c86da6cf74fad49a513e4cfd255

            gerrit Gerrit Updater added a comment - Alex Zhuravlev (bzzz@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/33226 Subject: LU-11288 osc: check target versus available grant Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: ec68ff8b3d389c86da6cf74fad49a513e4cfd255

            any details on how to reproduce that?

             

            bzzz Alex Zhuravlev added a comment - any details on how to reproduce that?  
            pjones Peter Jones added a comment -

            Ok then let's have Alex handle this - thanks for your analysis so far!

            pjones Peter Jones added a comment - Ok then let's have Alex handle this - thanks for your analysis so far!
            bobijam Zhenyu Xu added a comment -

            not yet, this code path hasn't been used for a long time, and I'm not familiar with grant as well.

            bobijam Zhenyu Xu added a comment - not yet, this code path hasn't been used for a long time, and I'm not familiar with grant as well.
            pjones Peter Jones added a comment -

            Bobijam

            Have you been able to make any progress on identifying the problem with the grant shrink algorithm?

            Peter

            pjones Peter Jones added a comment - Bobijam Have you been able to make any progress on identifying the problem with the grant shrink algorithm? Peter
            pjones Peter Jones added a comment -

            Oleg has confirmed that reverting the patch to enable grant shrink stops this failure appearing in his testing. However, based on Bobijam's analysis that means that the bug is still there, just not being exposed.

            pjones Peter Jones added a comment - Oleg has confirmed that reverting the patch to enable grant shrink stops this failure appearing in his testing. However, based on Bobijam's analysis that means that the bug is still there, just not being exposed.
            green Oleg Drokin added a comment -

            ok, thanks. I'll try to revert that patch locally and see if it makes any difference.

            green Oleg Drokin added a comment - ok, thanks. I'll try to revert that patch locally and see if it makes any difference.
            bobijam Zhenyu Xu added a comment -

            LU-8708 enables grant shrink by default, wondering whether there is issues with the grant shrinking algorithm that causes this.

            bobijam Zhenyu Xu added a comment - LU-8708 enables grant shrink by default, wondering whether there is issues with the grant shrinking algorithm that causes this.

            People

              bzzz Alex Zhuravlev
              green Oleg Drokin
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: