Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-7938

Recovery on secondary OSS node stalled

Details

    • Bug
    • Resolution: Not a Bug
    • Blocker
    • Lustre 2.9.0
    • Lustre 2.8.0, Lustre 2.9.0
    • lola
      build: 2.8 GA + patches
    • 3
    • 9223372036854775807

    Description

      Description

      Error happens during soak testing of build '20160324' (see https://wiki.hpdd.intel.com/display/Releases/Soak+Testing+on+Lola#SoakTestingonLola-20160324). DNE is enabled. MDTs had been formatted with ldiskfs, OSTs using zfs. MDS and OSS nodes are configured in HA active-active failover configuration. MDS nodes operated wiht 1 MDT per MDS, while OSSes running 4 OST per node.
      Nodes lola-4 and lola-5 form a HA cluster.

      Event history

      • 2016-03-29 07:48:04,307:fsmgmt.fsmgmt:INFO triggering fault oss_failover of node lola-4
        powercyle node
      • 2016-03-29 07:52:27,876:fsmgmt.fsmgmt:INFO lola-4 is up
      • 2016-03-29 07:53:09,557:fsmgmt.fsmgmt:INFO zpool import and mount of lola-4's OSTs complete
      • Recovery don't complete after recovery_time is zero
      • 2016-03-29 08:03 (approximately) Aborted recovery manually (lctl --device ... abort_recovery)

      Attached files:
      messages, console and debug log before (lustre-log-20160329-0759-recovery-stalled) and after recovery was aborted (lustre-log-20160329-0803-recovery-aborted)

      Attachments

        Activity

          [LU-7938] Recovery on secondary OSS node stalled
          jgmitter Joseph Gmitter (Inactive) made changes -
          Resolution New: Not a Bug [ 6 ]
          Status Original: In Progress [ 3 ] New: Resolved [ 5 ]
          heckes Frank Heckes (Inactive) made changes -
          Attachment New: ost-failover-setttings-20160726_0702 [ 22357 ]
          yong.fan nasf (Inactive) made changes -
          Status Original: Open [ 1 ] New: In Progress [ 3 ]
          heckes Frank Heckes (Inactive) made changes -
          jgmitter Joseph Gmitter (Inactive) made changes -
          Assignee Original: WC Triage [ wc-triage ] New: nasf [ yong.fan ]
          adilger Andreas Dilger made changes -
          Link New: This issue is related to LDEV-398 [ LDEV-398 ]
          pjones Peter Jones made changes -
          Fix Version/s New: Lustre 2.9.0 [ 11891 ]
          heckes Frank Heckes (Inactive) made changes -
          Attachment New: lola-4-lustre-log-20160415-0420.bz2 [ 21152 ]

          People

            yong.fan nasf (Inactive)
            heckes Frank Heckes (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: