Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-10160

LBUG ldlm_lock.c:2259:__ldlm_reprocess_all()) ASSERTION( list_empty(&rpc_list) ) failed:

    XMLWordPrintable

Details

    • Bug
    • Resolution: Unresolved
    • Major
    • None
    • Lustre 2.11.0
    • Soak cluster version=2.10.54_13_g84f690e
      lustre-master-ib build 6 with DNE configured
    • 3
    • 9223372036854775807

    Description

      After OSS failover and recovery:

      Oct 25 08:41:02 soak-2 kernel: Lustre: soaked-OST0012: Recovery over after 0:35, of 33 clients 33 recovered and 0 were evicted.
      Oct 25 08:41:03 soak-2 kernel: LustreError: 26237:0:(ofd_io.c:616:ofd_preprw_write()) soaked-OST0012: BRW to missing obj 0x0:1943200
      Oct 25 08:41:03 soak-2 kernel: LustreError: 26369:0:(lustre_dlm.h:1371:ldlm_res_lvbo_update()) delayed lvb init failed (rc -2)
      Oct 25 08:41:03 soak-2 kernel: format at lustre_dlm.h:1097:ldlm_lvbo_fill doesn't end in newline
      Oct 25 08:41:03 soak-2 kernel: LustreError: 26237:0:(ofd_io.c:616:ofd_preprw_write()) Skipped 3 previous similar messages
      Oct 25 08:41:03 soak-2 kernel: format at lustre_dlm.h:1097:ldlm_lvbo_fill doesn't end in newline
      Oct 25 08:41:03 soak-2 kernel: LustreError: 12692:0:(ofd_io.c:616:ofd_preprw_write()) soaked-OST0012: BRW to missing obj 0x780000400:835459
      Oct 25 08:41:03 soak-2 kernel: LustreError: 12692:0:(ofd_io.c:616:ofd_preprw_write()) Skipped 1 previous similar message
      Oct 25 08:41:03 soak-2 kernel: Lustre: soaked-OST0006: Connection restored to e5f19f2d-79a3-0744-c0c5-e9d37cbaf536 (at 192.168.1.118@o2ib)
      Oct 25 08:41:03 soak-2 kernel: Lustre: soaked-OST0000: Connection restored to e5f19f2d-79a3-0744-c0c5-e9d37cbaf536 (at 192.168.1.118@o2ib)
      Oct 25 08:41:03 soak-2 kernel: Lustre: Skipped 43 previous similar messages
      Oct 25 08:41:03 soak-2 kernel: Lustre: Skipped 43 previous similar messages
      Oct 25 08:41:03 soak-2 kernel: LustreError: 26369:0:(lustre_dlm.h:1097:ldlm_lvbo_fill()) lock ffff8800a4621c00: delayed lvb init failed (rc -2)
      Oct 25 08:41:03 soak-2 kernel: LustreError: 26469:0:(lustre_dlm.h:1097:ldlm_lvbo_fill()) lock ffff8800afe75200: delayed lvb init failed (rc -2)
      Oct 25 08:41:03 soak-2 kernel: LustreError: 26469:0:(ldlm_lock.c:2259:__ldlm_reprocess_all()) ASSERTION( list_empty(&rpc_list) ) failed:
      Oct 25 08:41:04 soak-2 kernel: LustreError: 26469:0:(ldlm_lock.c:2259:__ldlm_reprocess_all()) LBUG
      Oct 25 08:41:04 soak-2 kernel: Pid: 26469, comm: ldlm_cn00_003
      Oct 25 08:41:04 soak-2 kernel: #012Call Trace:
      Oct 25 08:41:04 soak-2 kernel: [<ffffffffc0d3e7ae>] libcfs_call_trace+0x4e/0x60 [libcfs]
      Oct 25 08:41:04 soak-2 kernel: [<ffffffffc0d3e83c>] lbug_with_loc+0x4c/0xb0 [libcfs]
      Oct 25 08:41:04 soak-2 kernel: [<ffffffffc112a88b>] __ldlm_reprocess_all+0x32b/0x3a0 [ptlrpc]
      Oct 25 08:41:04 soak-2 kernel: [<ffffffffc112a910>] ldlm_reprocess_all+0x10/0x20 [ptlrpc]
      Oct 25 08:41:04 soak-2 kernel: [<ffffffffc114dc8d>] ldlm_request_cancel+0x46d/0x720 [ptlrpc]
      Oct 25 08:41:04 soak-2 kernel: [<ffffffffc115417a>] ldlm_handle_cancel+0xba/0x250 [ptlrpc]
      Oct 25 08:41:04 soak-2 kernel: [<ffffffffc1154451>] ldlm_cancel_handler+0x141/0x490 [ptlrpc]
      Oct 25 08:41:05 soak-2 kernel: [<ffffffffc118537e>] ptlrpc_server_handle_request+0x24e/0xab0 [ptlrpc]
      Oct 25 08:41:05 soak-2 kernel: [<ffffffffc1182248>] ? ptlrpc_wait_event+0x98/0x340 [ptlrpc]
      Oct 25 08:41:05 soak-2 kernel: [<ffffffff810c4822>] ? default_wake_function+0x12/0x20
      Oct 25 08:41:05 soak-2 kernel: [<ffffffff810ba588>] ? __wake_up_common+0x58/0x90
      Oct 25 08:41:05 soak-2 kernel: [<ffffffffc1188b22>] ptlrpc_main+0xa92/0x1e40 [ptlrpc]
      Oct 25 08:41:05 soak-2 kernel: [<ffffffff81029557>] ? __switch_to+0xd7/0x510
      Oct 25 08:41:05 soak-2 kernel: [<ffffffff816a8f01>] ? __schedule+0x311/0x8b0
      Oct 25 08:41:05 soak-2 kernel: [<ffffffffc1188090>] ? ptlrpc_main+0x0/0x1e40 [ptlrpc]
      Oct 25 08:41:05 soak-2 kernel: [<ffffffff810b098f>] kthread+0xcf/0xe0
      Oct 25 08:41:05 soak-2 kernel: [<ffffffff810b08c0>] ? kthread+0x0/0xe0
      Oct 25 08:41:05 soak-2 kernel: [<ffffffff816b4f18>] ret_from_fork+0x58/0x90
      Oct 25 08:41:05 soak-2 kernel: [<ffffffff810b08c0>] ? kthread+0x0/0xe0
      

      Lustre-log dump attached

      Attachments

        1. soak-2-lustre-log.1508920865.26469.txt.gz
          883 kB
          Cliff White
        2. vmcore-dmesg.txt
          362 kB
          Cliff White

        Issue Links

          Activity

            People

              green Oleg Drokin
              cliffw Cliff White (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated: