Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-9340

PFL fails performance testsSpirit

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: Lustre 2.10.0
    • Fix Version/s: Lustre 2.10.0
    • Labels:
    • Environment:
      Spirit performance cluster
    • Severity:
      3
    • Rank (Obsolete):
      9223372036854775807

      Description

      Attempting to run P02 and P03 performance tests, with striping set as:
      $LFS setstripe $testdir --pool $ior_ostPool -E 64M -c 1 -E 4G -c 4 -E -1 -c -1I

      Immediate MPI failures with IOR

       Commencing write performance test: Thu Apr 13 21:04:16 2017
      024: ior ERROR: write() failed, errno 61, No data available (aiori-POSIX.c:335)
      024: --------------------------------------------------------------------------
      024: MPI_ABORT was invoked on rank 24 in communicator MPI_COMM_WORLD
      --
      ..........
      231: ior ERROR: write() failed, errno 61, No data available (aiori-POSIX.c:335)
      088: In: PMI_Abort(-1, N/A)
      287: ior ERROR: write() failed, errno 61, No data available (aiori-POSIX.c:335)
      134: In: PMI_Abort(-1, N/A)
      057: --------------------------------------------------------------------------
      057: MPI_ABORT was invoked on rank 57 in communicator MPI_COMM_WORLD 
      --
      057: 
      057: NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
      057: You may or may not see output from other processes, depending on
      057: exactly when Open MPI kills them.
      057: -------------------------------------------
      

      Lustre Errors on all nodes attached.

        Attachments

        1. ior-stripe.txt
          3 kB
        2. pfl.errors.txt
          17 kB
        3. spirit-10.lustre.dump.gz
          3.48 MB
        4. spirit-29.lustre.dump.gz
          963 kB
        5. spirit-30.lustre.dump.gz
          955 kB
        6. spirit-7.lustre.dump.gz
          3.26 MB
        7. spirit-8.lustre.dump.gz
          3.48 MB
        8. spirit-9.lustre.dump.gz
          3.65 MB

          Issue Links

            Activity

              People

              • Assignee:
                jay Jinshan Xiong (Inactive)
                Reporter:
                cliffw Cliff White (Inactive)
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: