Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-6724

Downgrading from 2.8 with DNE2 patches to 2.5 servers fails: unsupported read-only filesystem feature(s) 2

Details

    • Bug
    • Resolution: Fixed
    • Blocker
    • Lustre 2.8.0
    • Lustre 2.8.0
    • Lustre 2.5.3 servers plus lastest Lustre 2.8 with DNE2 patches.
    • 3
    • 9223372036854775807

    Description

      This morning I updated to the latest vanilla master and ended up in a state where I could not mount the file system. So I tried migrating back to lustre 2.5 and when I attempted to mount the file system I got these errors:

      [ 1025.018232] Lustre: Lustre: Build Version: 2.5.4--CHANGED-2.6.32-431.29.2.el6.atlas.x86_64
      [ 1062.767104] LDISKFS-fs (dm-7): recovery complete
      [ 1062.791599] LDISKFS-fs (dm-7): mounted filesystem with ordered data mode. quota=on. Opts:
      [ 1063.389770] LustreError: 24686:0:(ofd_fs.c:594:ofd_server_data_init()) sultan-OST0000: unsupported read-only filesystem feature(s) 2
      [ 1063.412582] LustreError: 24686:0:(obd_config.c:572:class_setup()) setup sultan-OST0000 failed (-22)
      [ 1063.421818] LustreError: 24686:0:(obd_config.c:1629:class_config_llog_handler()) MGC10.37.248.67@o2ib1: cfg command failed: rc = -22
      [ 1063.433944] Lustre: cmd=cf003 0:sultan-OST0000 1:dev 2:0 3:f
      [ 1063.440506] LustreError: 15b-f: MGC10.37.248.67@o2ib1: The configuration from log 'sultan-OST0000'failed from the MGS (-22). Make sure this client and the MGS are running compatible versions of Lustre.
      [ 1063.458690] LustreError: 15c-8: MGC10.37.248.67@o2ib1: The configuration from log 'sultan-OST0000' failed (-22). This may be the result of communication errors between this node and the MGS, a bad configuration, or other errors. See the syslog for more information.
      [ 1063.482398] LustreError: 24600:0:(obd_mount_server.c:1254:server_start_targets()) failed to start server sultan-OST0000: -22
      [ 1063.493822] LustreError: 24600:0:(obd_mount_server.c:1737:server_fill_super()) Unable to start targets: -22
      [ 1063.503768] LustreError: 24600:0:(obd_mount_server.c:847:lustre_disconnect_lwp()) sultan-MDT0000-lwp-OST0000: Can't end config log sultan-client.
      [ 1063.516947] LustreError: 24600:0:(obd_mount_server.c:1422:server_put_super()) sultan-OST0000: failed to disconnect lwp. (rc=-2)
      [ 1063.528574] LustreError: 24600:0:(obd_config.c:619:class_cleanup()) Device 3 not setup
      [ 1063.539611] Lustre: server umount sultan-OST0000 complete
      [ 1063.545093] LustreError: 24600:0:(obd_mount.c:1330:lustre_fill_super()) Unable to mount /dev/mapper/sultan-ddn-l0 (-22)
      [ 1070.949382] LDISKFS-fs (dm-6): recovery complete
      [ 1070.956045] LDISKFS-fs (dm-6): mounted filesystem with ordered data mode. quota=on. Opts:
      [ 1071.472962] LustreError: 24982:0:(ofd_fs.c:594:ofd_server_data_init()) sultan-OST0004: unsupported read-only filesystem feature(s) 2
      [ 1071.495949] LustreError: 24982:0:(obd_config.c:572:class_setup()) setup sultan-OST0004 failed (-22)
      [ 1071.505140] LustreError: 24982:0:(obd_config.c:1629:class_config_llog_handler()) MGC10.37.248.67@o2ib1: cfg command failed: rc = -22
      

      Attachments

        Issue Links

          Activity

            [LU-6724] Downgrading from 2.8 with DNE2 patches to 2.5 servers fails: unsupported read-only filesystem feature(s) 2
            simmonsja James A Simmons made changes -
            Fix Version/s New: Lustre 2.8.0 [ 11113 ]
            Resolution New: Fixed [ 1 ]
            Status Original: In Progress [ 3 ] New: Resolved [ 5 ]

            Oh this is a old ticket. This problem doesn't exist anymore. Well you do see the issue with mult-slot but their exist a work around. For those that don't know you umount your 2.8 file system then remount with recovery abort. Then unmount again. After that you can reboot into your 2.5 image and remount with no problem. This clears the multi-slot support from your config logs. Other than that we haven't see migration issues.

            simmonsja James A Simmons added a comment - Oh this is a old ticket. This problem doesn't exist anymore. Well you do see the issue with mult-slot but their exist a work around. For those that don't know you umount your 2.8 file system then remount with recovery abort. Then unmount again. After that you can reboot into your 2.5 image and remount with no problem. This clears the multi-slot support from your config logs. Other than that we haven't see migration issues.
            adilger Andreas Dilger made changes -
            Link Original: This issue is related to LDEV-418 [ LDEV-418 ]
            adilger Andreas Dilger made changes -
            Link New: This issue is duplicated by LDEV-418 [ LDEV-418 ]
            adilger Andreas Dilger made changes -
            Summary Original: Downgrading from 2.8 with DNE2 patches to 2.5 servers can corrupt the file system. New: Downgrading from 2.8 with DNE2 patches to 2.5 servers fails: unsupported read-only filesystem feature(s) 2
            pjones Peter Jones made changes -
            Link New: This issue is related to LDEV-418 [ LDEV-418 ]
            adilger Andreas Dilger made changes -
            Description Original: This morning I updated to the latest vanilla master and ended up in a state where I could not mount the file system. So I tried migrating back to lustre 2.5 and when I attempted to mount the file system I got these errors:

            [ 1025.018232] Lustre: Lustre: Build Version: 2.5.4--CHANGED-2.6.32-431.29.2.el6.atlas.x86_64
            [ 1062.767104] LDISKFS-fs (dm-7): recovery complete
            [ 1062.791599] LDISKFS-fs (dm-7): mounted filesystem with ordered data mode. quota=on. Opts:
            [ 1063.389770] LustreError: 24686:0:(ofd_fs.c:594:ofd_server_data_init()) sultan-OST0000: unsupported read-only filesystem feature(s) 2
            [ 1063.412582] LustreError: 24686:0:(obd_config.c:572:class_setup()) setup sultan-OST0000 failed (-22)
            [ 1063.421818] LustreError: 24686:0:(obd_config.c:1629:class_config_llog_handler()) MGC10.37.248.67@o2ib1: cfg command failed: rc = -22
            [ 1063.433944] Lustre: cmd=cf003 0:sultan-OST0000 1:dev 2:0 3:f
            [ 1063.440506] LustreError: 15b-f: MGC10.37.248.67@o2ib1: The configuration from log 'sultan-OST0000'failed from the MGS (-22). Make sure this client and the MGS are running compatible versions of Lustre.
            [ 1063.458690] LustreError: 15c-8: MGC10.37.248.67@o2ib1: The configuration from log 'sultan-OST0000' failed (-22). This may be the result of communication errors between this node and the MGS, a bad configuration, or other errors. See the syslog for more information.
            [ 1063.482398] LustreError: 24600:0:(obd_mount_server.c:1254:server_start_targets()) failed to start server sultan-OST0000: -22
            [ 1063.493822] LustreError: 24600:0:(obd_mount_server.c:1737:server_fill_super()) Unable to start targets: -22
            [ 1063.503768] LustreError: 24600:0:(obd_mount_server.c:847:lustre_disconnect_lwp()) sultan-MDT0000-lwp-OST0000: Can't end config log sultan-client.
            [ 1063.516947] LustreError: 24600:0:(obd_mount_server.c:1422:server_put_super()) sultan-OST0000: failed to disconnect lwp. (rc=-2)
            [ 1063.528574] LustreError: 24600:0:(obd_config.c:619:class_cleanup()) Device 3 not setup
            [ 1063.539611] Lustre: server umount sultan-OST0000 complete
            [ 1063.545093] LustreError: 24600:0:(obd_mount.c:1330:lustre_fill_super()) Unable to mount /dev/mapper/sultan-ddn-l0 (-22)
            [ 1070.949382] LDISKFS-fs (dm-6): recovery complete
            [ 1070.956045] LDISKFS-fs (dm-6): mounted filesystem with ordered data mode. quota=on. Opts:
            [ 1071.472962] LustreError: 24982:0:(ofd_fs.c:594:ofd_server_data_init()) sultan-OST0004: unsupported read-only filesystem feature(s) 2
            [ 1071.495949] LustreError: 24982:0:(obd_config.c:572:class_setup()) setup sultan-OST0004 failed (-22)
            [ 1071.505140] LustreError: 24982:0:(obd_config.c:1629:class_config_llog_handler()) MGC10.37.248.67@o2ib1: cfg command failed: rc = -22


            New: This morning I updated to the latest vanilla master and ended up in a state where I could not mount the file system. So I tried migrating back to lustre 2.5 and when I attempted to mount the file system I got these errors:
            {noformat}
            [ 1025.018232] Lustre: Lustre: Build Version: 2.5.4--CHANGED-2.6.32-431.29.2.el6.atlas.x86_64
            [ 1062.767104] LDISKFS-fs (dm-7): recovery complete
            [ 1062.791599] LDISKFS-fs (dm-7): mounted filesystem with ordered data mode. quota=on. Opts:
            [ 1063.389770] LustreError: 24686:0:(ofd_fs.c:594:ofd_server_data_init()) sultan-OST0000: unsupported read-only filesystem feature(s) 2
            [ 1063.412582] LustreError: 24686:0:(obd_config.c:572:class_setup()) setup sultan-OST0000 failed (-22)
            [ 1063.421818] LustreError: 24686:0:(obd_config.c:1629:class_config_llog_handler()) MGC10.37.248.67@o2ib1: cfg command failed: rc = -22
            [ 1063.433944] Lustre: cmd=cf003 0:sultan-OST0000 1:dev 2:0 3:f
            [ 1063.440506] LustreError: 15b-f: MGC10.37.248.67@o2ib1: The configuration from log 'sultan-OST0000'failed from the MGS (-22). Make sure this client and the MGS are running compatible versions of Lustre.
            [ 1063.458690] LustreError: 15c-8: MGC10.37.248.67@o2ib1: The configuration from log 'sultan-OST0000' failed (-22). This may be the result of communication errors between this node and the MGS, a bad configuration, or other errors. See the syslog for more information.
            [ 1063.482398] LustreError: 24600:0:(obd_mount_server.c:1254:server_start_targets()) failed to start server sultan-OST0000: -22
            [ 1063.493822] LustreError: 24600:0:(obd_mount_server.c:1737:server_fill_super()) Unable to start targets: -22
            [ 1063.503768] LustreError: 24600:0:(obd_mount_server.c:847:lustre_disconnect_lwp()) sultan-MDT0000-lwp-OST0000: Can't end config log sultan-client.
            [ 1063.516947] LustreError: 24600:0:(obd_mount_server.c:1422:server_put_super()) sultan-OST0000: failed to disconnect lwp. (rc=-2)
            [ 1063.528574] LustreError: 24600:0:(obd_config.c:619:class_cleanup()) Device 3 not setup
            [ 1063.539611] Lustre: server umount sultan-OST0000 complete
            [ 1063.545093] LustreError: 24600:0:(obd_mount.c:1330:lustre_fill_super()) Unable to mount /dev/mapper/sultan-ddn-l0 (-22)
            [ 1070.949382] LDISKFS-fs (dm-6): recovery complete
            [ 1070.956045] LDISKFS-fs (dm-6): mounted filesystem with ordered data mode. quota=on. Opts:
            [ 1071.472962] LustreError: 24982:0:(ofd_fs.c:594:ofd_server_data_init()) sultan-OST0004: unsupported read-only filesystem feature(s) 2
            [ 1071.495949] LustreError: 24982:0:(obd_config.c:572:class_setup()) setup sultan-OST0004 failed (-22)
            [ 1071.505140] LustreError: 24982:0:(obd_config.c:1629:class_config_llog_handler()) MGC10.37.248.67@o2ib1: cfg command failed: rc = -22
            {noformat}
            standan Saurabh Tandan (Inactive) made changes -
            Comment [ A comment with security level 'Employeees' was removed. ]
            pjones Peter Jones made changes -
            End date New: 18/Jun/15
            Start date New: 15/Jun/15

            A detailed description of the corruption and the patch to make it required intervention are in LU-6050.
            https://jira.hpdd.intel.com/browse/LU-6050
            http://review.whamcloud.com/#/c/13516/

            paf Patrick Farrell (Inactive) added a comment - A detailed description of the corruption and the patch to make it required intervention are in LU-6050 . https://jira.hpdd.intel.com/browse/LU-6050 http://review.whamcloud.com/#/c/13516/

            People

              laisiyao Lai Siyao
              simmonsja James A Simmons
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: