Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-547

1.8<->2.1 / 2.3<->2.4 interop: dbench kept doing cleanup

    Details

    • Severity:
      3
    • Rank (Obsolete):
      6058

      Description

      While running sanity-benchmark test suite, the dbench test kept doing cleanup and could not stop:

      <~snip~>
         1    737819     5.51 MB/sec  execute 598 sec  latency 38819.862 ms
         1    737819     5.50 MB/sec  execute 599 sec  latency 39820.034 ms
         1  cleanup 600 sec
         1  cleanup 601 sec
         ......
         1  cleanup 2897 sec
         1  cleanup 2898 sec
         ......
      

      On client node fat-amd-3-ib:

      # ps auxww
      <~snip~>
      500      18628  0.0  0.0 107264  2552 pts/0    S+   00:50   0:00 bash rundbench -D /mnt/lustre/d0.fat-amd-3-ib 1
      500      18636  0.0  0.0   6092   596 pts/0    S+   00:50   0:00 dbench -c client.txt 1
      500      18637  2.2  0.0   8140  2540 pts/0    S+   00:50   2:00 dbench -c client.txt 1
      
      # echo t > /proc/sysrq-trigger
      <~snip~>
      dbench        S 0000000000000009     0 18636  18628 0x00000080
       ffff880219239e78 0000000000000086 0000000000000000 ffffffff812688c5
       ffff880323c50f98 ffff880218872068 0000000000000000 0000000100b91979
       ffff880218af3a78 ffff880219239fd8 000000000000f598 ffff880218af3a78
      Call Trace:
       [<ffffffff812688c5>] ? rb_insert_color+0x125/0x160
       [<ffffffff8106b325>] do_wait+0x1c5/0x250
       [<ffffffff8106b453>] sys_wait4+0xa3/0x100
       [<ffffffff810699b0>] ? child_wait_callback+0x0/0x70
       [<ffffffff8100b425>] ? int_check_syscall_exit_work+0x34/0x3d
       [<ffffffff8100b172>] system_call_fastpath+0x16/0x1b
      dbench        S 000000000000000c     0 18637  18636 0x00000080
       ffff880419933a98 0000000000000082 0000000000000000 000000004e31176c
       ffff880419933a78 ffffffffa04a2cac ffff880419933a78 ffff8803191cb000
       ffff8804197b05f8 ffff880419933fd8 000000000000f598 ffff8804197b05f8
      Call Trace:
       [<ffffffffa04a2cac>] ? ptlrpc_at_adj_net_latency+0x7c/0x230 [ptlrpc]
       [<ffffffffa049c00d>] ldlm_flock_completion_ast+0x61d/0x9f0 [ptlrpc]
       [<ffffffffa046c15f>] ? lock_res_and_lock+0x5f/0xe0 [ptlrpc]
       [<ffffffff8105dc20>] ? default_wake_function+0x0/0x20
       [<ffffffffa048a565>] ldlm_cli_enqueue_fini+0x6c5/0xba0 [ptlrpc]
       [<ffffffff8105dc20>] ? default_wake_function+0x0/0x20
       [<ffffffffa048e074>] ldlm_cli_enqueue+0x344/0x7a0 [ptlrpc]
       [<ffffffffa0677edd>] ll_file_flock+0x47d/0x6b0 [lustre]
       [<ffffffffa049b9f0>] ? ldlm_flock_completion_ast+0x0/0x9f0 [ptlrpc]
       [<ffffffff811bc243>] vfs_lock_file+0x23/0x40
       [<ffffffff811bc497>] fcntl_setlk+0x177/0x320
       [<ffffffff8107ff06>] ? group_send_sig_info+0x26/0x70
       [<ffffffff81080212>] ? sys_kill+0x92/0x190
       [<ffffffff811845f7>] sys_fcntl+0x197/0x530
       [<ffffffff8100b172>] system_call_fastpath+0x16/0x1b
      

      Maloo report: https://maloo.whamcloud.com/test_sets/4f04bb24-b8f5-11e0-8bdf-52540025f9af

      Please find the attached dbench-debug_log.tar.bz2 for debug logs.

        Attachments

          Activity

            People

            • Assignee:
              green Oleg Drokin
              Reporter:
              yujian Jian Yu
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: