Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-4593

Lustre OSTs on one of the login nodes are evicted but never reconnect.

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • None
    • Lustre 2.4.1
    • 2
    • 12546

      AWE has reported the following problem about Lustre.
      Occasionally some of the Lustre OSTs on sprig2 (one of the login nodes) are evicted but never reconnect and they see the following error in the syslog:

      Jan 14 08:54:13 sprig2 kernel: [680476.616831] LustreError: 809:0:(cl_lock.c:1420:cl_unuse_try()) result = -108, this is unlikely!

      extract of syslog
      Jan 14 08:55:19 sprig2 kernel: [680478.577953] LustreError: 829:0:(cl_lock.c:1435:cl_unuse_locked()) } lock@ffff882d00691eb8
      Jan 14 08:55:19 sprig2 kernel: [680478.577958] LustreError: 829:0:(cl_lock.c:1435:cl_unuse_locked()) 3 0: —
      Jan 14 08:55:19 sprig2 kernel: [680478.577963] LustreError: 829:0:(cl_lock.c:1435:cl_unuse_locked()) 4 0: —
      Jan 14 08:55:19 sprig2 kernel: [680478.577972] LustreError: 829:0:(cl_lock.c:1435:cl_unuse_locked()) 5 0: lock@ffff8838357b34d8[0 5 0 0 0 00000000] R(1):[0, 18446744073709551615]@[0x100070000:0x41dd7e:0x0] {
      Jan 14 08:55:19 sprig2 kernel: [680478.577982] LustreError: 829:0:(cl_lock.c:1435:cl_unuse_locked()) lovsub@ffff882cb4efc760: [5 ffff883bbf9a88e8 P(0):[0, 1844674
      4073709551615]@[0x5000013f2:0x2872:0x0]]
      Jan 14 08:55:19 sprig2 kernel: [680478.577992] LustreError: 829:0:(cl_lock.c:1435:cl_unuse_locked()) osc@ffff883cb5239d80: ffff883c470b7b40 0x20000041001 0x54d9d8518dc4ea31

      Looks similar to issue reported on https://jira.hpdd.intel.com/browse/LU-3889.

        1. sprig-oss2-messages.txt
          5 kB
          Oz Rentas
        2. sprig-oss1-messages.txt
          4 kB
          Oz Rentas
        3. sprig-mds2-messages.txt
          0.4 kB
          Oz Rentas
        4. sprig-mds1-messages.txt
          0.7 kB
          Oz Rentas
        5. sprig5-messages-20140226.txt
          20 kB
          Oz Rentas
        6. sprig5-lustre-log.1393411844.12537
          270 kB
          Oz Rentas
        7. sprig5-dmesg-20140226.txt
          20 kB
          Oz Rentas
        8. sprig2-messages-20140130.txt
          27 kB
          Oz Rentas
        9. sprig2-lfs.txt
          2 kB
          Oz Rentas
        10. oss2-messages
          5 kB
          Oz Rentas
        11. oss1-messages
          6 kB
          Oz Rentas
        12. messages-sprig3
          40 kB
          Oz Rentas
        13. messages-sprig2
          28 kB
          Oz Rentas
        14. messages-oss2
          7 kB
          Oz Rentas
        15. messages-oss1
          7 kB
          Oz Rentas
        16. messages-mds2
          5 kB
          Oz Rentas
        17. messages-mds1
          4 kB
          Oz Rentas
        18. messages-20140304
          118 kB
          Rajeshwaran Ganesan
        19. mds2-messages
          3 kB
          Oz Rentas
        20. mds1-messages
          0.9 kB
          Oz Rentas
        21. lustre-log.1393924031.9592
          185 kB
          Rajeshwaran Ganesan
        22. lustre-evict-7mar-client.tar
          1.91 MB
          Rajeshwaran Ganesan
        23. lustre-evict-7mar14-mds1.tar
          150 kB
          Rajeshwaran Ganesan
        24. LU-4593-lustre-log.1393924031.9587.gz
          234 kB
          Rajeshwaran Ganesan
        25. lctl_ping.txt
          13 kB
          Oz Rentas
        26. dmesg-20140304
          212 kB
          Rajeshwaran Ganesan

            bobijam Zhenyu Xu
            orentas Oz Rentas (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            12 Start watching this issue

              Created:
              Updated:
              Resolved: