Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-15123

sanity-quota: test_7a Error: 'reintegration failed'

Details

    • Bug
    • Resolution: Fixed
    • Minor
    • Lustre 2.16.0, Lustre 2.15.4
    • Lustre 2.16.0, Lustre 2.15.3
    • None
    • 3
    • 9223372036854775807

    Description

      This issue was created by maloo for paf <pfarrell@whamcloud.com>

      This issue relates to the following test suite run: https://testing.whamcloud.com/test_sets/ded8a9a9-b77c-41ca-bb53-105290b2709e

      Attachments

        Issue Links

          Activity

            [LU-15123] sanity-quota: test_7a Error: 'reintegration failed'

            "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/51233/
            Subject: LU-15123 tests: check quota reintegration after recovery
            Project: fs/lustre-release
            Branch: b2_15
            Current Patch Set:
            Commit: 13805e3a2d4f520e297bc408d94b9971a6094f9a

            gerrit Gerrit Updater added a comment - "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/51233/ Subject: LU-15123 tests: check quota reintegration after recovery Project: fs/lustre-release Branch: b2_15 Current Patch Set: Commit: 13805e3a2d4f520e297bc408d94b9971a6094f9a

            "Xing Huang <hxing@ddn.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/51233
            Subject: LU-15123 tests: check quota reintegration after recovery
            Project: fs/lustre-release
            Branch: b2_15
            Current Patch Set: 1
            Commit: 8002f7d111f817032ffe8ee3485abf5e4472a148

            gerrit Gerrit Updater added a comment - "Xing Huang <hxing@ddn.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/51233 Subject: LU-15123 tests: check quota reintegration after recovery Project: fs/lustre-release Branch: b2_15 Current Patch Set: 1 Commit: 8002f7d111f817032ffe8ee3485abf5e4472a148
            pjones Peter Jones added a comment -

            Landed for 2.16

            pjones Peter Jones added a comment - Landed for 2.16

            "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/50688/
            Subject: LU-15123 tests: check quota reintegration after recovery
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: 4432b6e2824775e292f96e202d6fc0db231bc749

            gerrit Gerrit Updater added a comment - "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/50688/ Subject: LU-15123 tests: check quota reintegration after recovery Project: fs/lustre-release Branch: master Current Patch Set: Commit: 4432b6e2824775e292f96e202d6fc0db231bc749

            "Alex Zhuravlev <bzzz@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/50688
            Subject: LU-15123 tests: quota reintegration starts after recovery
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: e6a2cb8c1aa0a96ec1d6e4603132635459ac0615

            gerrit Gerrit Updater added a comment - "Alex Zhuravlev <bzzz@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/50688 Subject: LU-15123 tests: quota reintegration starts after recovery Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: e6a2cb8c1aa0a96ec1d6e4603132635459ac0615

            [13964.128411] Lustre: lustre-OST0000: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180
            [13965.655492] Lustre: lustre-OST0000: Will be in recovery for at least 1:00, or until 5 clients reconnect
            ...
            [14061.885567] Lustre: DEBUG MARKER: /usr/sbin/lctl mark sanity-quota test_7a: @@@@@@ FAIL: reintegration failed
            [14067.469119] Lustre: lustre-OST0000: recovery is timed out, evict stale exports
            [14067.470635] Lustre: lustre-OST0000: disconnecting 1 stale clients
            [14067.787175] Lustre: lustre-OST0000: Recovery over after 1:42, of 5 clients 4 recovered and 1 was evicted.

            
            

            reintegration starts only when recovery is over. in this case the recovery process was stuck due to a missing client (to be evicted in the end) and the recovery process took 102 seconds while test 7a waits 90s at most.

            bzzz Alex Zhuravlev added a comment - [13964.128411] Lustre: lustre-OST0000: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [13965.655492] Lustre: lustre-OST0000: Will be in recovery for at least 1:00, or until 5 clients reconnect ... [14061.885567] Lustre: DEBUG MARKER: /usr/sbin/lctl mark sanity-quota test_7a: @@@@@@ FAIL: reintegration failed [14067.469119] Lustre: lustre-OST0000: recovery is timed out, evict stale exports [14067.470635] Lustre: lustre-OST0000: disconnecting 1 stale clients [14067.787175] Lustre: lustre-OST0000: Recovery over after 1:42, of 5 clients 4 recovered and 1 was evicted. reintegration starts only when recovery is over. in this case the recovery process was stuck due to a missing client (to be evicted in the end) and the recovery process took 102 seconds while test 7a waits 90s at most.
            yujian Jian Yu added a comment - +1 on b2_15 branch: https://testing.whamcloud.com/test_sets/150ed79f-6d70-4048-b875-56a9bccc54cf
            nangelinas Nikitas Angelinas added a comment - +1 on master: https://testing.whamcloud.com/test_sets/3d837ba0-73c7-4737-9894-5bbca2c9b479
            nangelinas Nikitas Angelinas added a comment - +1 on master: https://testing.whamcloud.com/test_sets/dc29fa4d-ef8c-4838-a182-7a544385f4cc
            zam Alexander Zarochentsev added a comment - +1 on master: https://testing.whamcloud.com/test_sets/db2d7940-4287-4ff0-9435-81c6520360b7

            People

              bzzz Alex Zhuravlev
              maloo Maloo
              Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: