Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-20986

sanity test_119a: Timeout occurred after 96 minutes, last suite running was sanity

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Medium Medium
    • None
    • None
    • None
    • 3
    • 9223372036854775807

      This issue was created by maloo for Xiao Yang <xyang@ddn.com>

      This issue relates to the following test suite run: https://testing.whamcloud.com/test_sets/a829da93-09fb-4087-a36e-02a2a06d6f6e

      Test session details:

      Interop

      Client
      Lustre: 2.17.59.69
      Branch: master
      Build: https://build.whamcloud.com/job/lustre-reviews/133773
      Kernel: 5.14.0-611.55.1.el9_7.x86_64

      Server
      Lustre: 2.15.8
      Branch: b2_15
      Build: https://build.whamcloud.com/job/lustre-b2_15/117
      Kernel: 4.18.0-553.82.1.el8_lustre.x86_64

      <<Please provide additional information about the failure here>>
      dmesg:
      [Sat Oct 17 15:44:21 2026] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 119a: Short directIO read must return actual read amount ========================================================== 15:44:22 (1792251862)
      [Sat Oct 17 15:44:22 2026] Lustre: DEBUG MARKER: == sanity test 119a: Short directIO read must return actual read amount ========================================================== 15:44:22 (1792251862)
      [Sat Oct 17 15:44:22 2026] LustreError: 3578:0:(events.c:193:client_bulk_callback()) event type 1, status -5, req ff286b7683e9ad80 desc ff286b7696547000 mbits 1879312075616512
      [Sat Oct 17 15:44:22 2026] Lustre: 3586:0:(client.c:2508:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1792251863/real 1792251863] req@ff286b7683e9ad80 x1879312075616512/t0(0) o4->lustre-OST0005-osc-ff286b7683ef3800@10.240.29.126@tcp:6/4 lens 488/448 e 0 to 1 dl 1792251879 ref 2 fl Rpc:eXQr/600/ffffffff rc 0/-1 job:'directio.0' uid:0 gid:0 projid:0
      [Sat Oct 17 15:44:22 2026] Lustre: lustre-OST0005-osc-ff286b7683ef3800: Connection to lustre-OST0005 (at 10.240.29.126@tcp) was lost; in progress operations using this service will wait for recovery to complete
      [Sat Oct 17 15:44:22 2026] LNet: 1 local NIs in recovery (showing 1): 10.240.31.235@tcp
      [Sat Oct 17 15:44:27 2026] Lustre: 3585:0:(client.c:2508:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1792251868/real 1792251868] req@ff286b7683e9b740 x1879312075617408/t0(0) o400->lustre-OST0004-osc-ff286b7683ef3800@10.240.29.126@tcp:28/4 lens 224/224 e 0 to 1 dl 1792251884 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
      [Sat Oct 17 15:44:27 2026] Lustre: lustre-OST0004-osc-ff286b7683ef3800: Connection to lustre-OST0004 (at 10.240.29.126@tcp) was lost; in progress operations using this service will wait for recovery to complete
      [Sat Oct 17 15:44:52 2026] Lustre: lustre-OST0005-osc-ff286b7683ef3800: Connection restored to 10.240.29.126@tcp (at 10.240.29.126@tcp)
      [Sat Oct 17 15:44:52 2026] LustreError: 3578:0:(events.c:193:client_bulk_callback()) event type 1, status -5, req ff286b7683e9ad80 desc ff286b7696547000 mbits 1879312075622272
      [Sat Oct 17 15:44:52 2026] Lustre: 3586:0:(client.c:2508:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1792251893/real 1792251893] req@ff286b7683e9ad80 x1879312075616512/t0(0) o4->lustre-OST0005-osc-ff286b7683ef3800@10.240.29.126@tcp:6/4 lens 488/448 e 0 to 1 dl 1792251909 ref 2 fl Rpc:eXQr/602/ffffffff rc 0/-1 job:'directio.0' uid:0 gid:0 projid:0
      [Sat Oct 17 15:44:52 2026] Lustre: lustre-OST0005-osc-ff286b7683ef3800: Connection to lustre-OST0005 (at 10.240.29.126@tcp) was lost; in progress operations using this service will wait for recovery to complete
      [Sat Oct 17 15:44:58 2026] Lustre: lustre-OST0005-osc-ff286b7683ef3800: Connection restored to 10.240.29.126@tcp (at 10.240.29.126@tcp)
      [Sat Oct 17 15:44:58 2026] Lustre: Skipped 1 previous similar message
      [Sat Oct 17 15:44:58 2026] LustreError: 3578:0:(events.c:193:client_bulk_callback()) event type 1, status -5, req ff286b7683e9ad80 desc ff286b7696547000 mbits 1879312075623424
      [Sat Oct 17 15:44:58 2026] Lustre: 3586:0:(client.c:2508:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1792251898/real 1792251898] req@ff286b7683e9ad80 x1879312075616512/t0(0) o4->lustre-OST0005-osc-ff286b7683ef3800@10.240.29.126@tcp:6/4 lens 488/448 e 0 to 1 dl 1792251914 ref 2 fl Rpc:eXQr/602/ffffffff rc 0/-1 job:'directio.0' uid:0 gid:0 projid:0
      [Sat Oct 17 15:44:58 2026] Lustre: lustre-OST0005-osc-ff286b7683ef3800: Connection to lustre-OST0005 (at 10.240.29.126@tcp) was lost; in progress operations using this service will wait for recovery to complete
      [Sat Oct 17 15:45:03 2026] Lustre: lustre-OST0005-osc-ff286b7683ef3800: Connection restored to 10.240.29.126@tcp (at 10.240.29.126@tcp)
      [Sat Oct 17 15:45:03 2026] LustreError: 3578:0:(events.c:193:client_bulk_callback()) event type 1, status -5, req ff286b7683e9ad80 desc ff286b7696547000 mbits 1879312075624576
      [Sat Oct 17 15:45:03 2026] Lustre: 3586:0:(client.c:2508:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1792251903/real 1792251903] req@ff286b7683e9ad80 x1879312075616512/t0(0) o4->lustre-OST0005-osc-ff286b7683ef3800@10.240.29.126@tcp:6/4 lens 488/448 e 0 to 1 dl 1792251919 ref 2 fl Rpc:eXQr/602/ffffffff rc 0/-1 job:'directio.0' uid:0 gid:0 projid:0
      [Sat Oct 17 15:45:03 2026] Lustre: lustre-OST0005-osc-ff286b7683ef3800: Connection to lustre-OST0005 (at 10.240.29.126@tcp) was lost; in progress operations using this service will wait for recovery to complete
      [Sat Oct 17 15:45:08 2026] Lustre: lustre-OST0005-osc-ff286b7683ef3800: Connection restored to 10.240.29.126@tcp (at 10.240.29.126@tcp)
      [Sat Oct 17 15:45:08 2026] LustreError: 3578:0:(events.c:193:client_bulk_callback()) event type 1, status -5, req ff286b7683e9ad80 desc ff286b7696547000 mbits 1879312075625728
      [Sat Oct 17 15:45:13 2026] Lustre: lustre-OST0004-osc-ff286b7683ef3800: disconnect after 21s idle
      [Sat Oct 17 15:45:13 2026] Lustre: Skipped 7 previous similar messages
      ...

            wc-triage WC Triage
            maloo Maloo
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

              Created:
              Updated: