Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-6538

Client s getting "operation ost_write failed with -3."

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Blocker
    • None
    • Lustre 2.5.3
    • Linux meerkat-mds-10-1.local 2.6.32-431.29.2.el6_lustre.gb8d9077.x86_64 #1 SMP Mon Apr 27 12:55:47 PDT 2015 x86_64 x86_64 x86_64 GNU/Linux
    • 4
    • 9223372036854775807

    Description

      Some clients can't write to some OSTs.

      user@gordon-ln3 LTEST]$ cd /oasis/projects/nsf/use300/mahidhar/LTEST
      [user@gordon-ln3 LTEST]$ lfs setstripe -c 1 -i 54 .
      [user@gordon-ln3 LTEST]$ ls
      IOR-2.10.3.tar
      [user@gordon-ln3 LTEST]$ rm IOR-2.10.3.tar
      [user@gordon-ln3 LTEST]$ cp ../IOR-2.10.3.tar .
      cp: writing `./IOR-2.10.3.tar': No such process
      [user@gordon-ln3 LTEST]$

      Client dmesg got:

      LustreError: 11-0: meerkat-OST0015-osc-ffff88102f998c00: Communicating with xx.xx.xx.xx244@tcp, operation ost_write failed with -3.
      LustreError: 11-0: meerkat-OST0036-osc-ffff88102f998c00: Communicating with xx.xx.xx.xx@tcp, operation ost_write failed with -3.
      LustreError: 11-0: meerkat-OST000e-osc-ffff88102f998c00: Communicating with xx.xx.xx.xx@tcp, operation ost_write failed with -3.
      LustreError: 11-0: meerkat-OST000d-osc-ffff88102f998c00: Communicating with xx.xx.xx.xx@tcp, operation ost_write failed with -3.
      LustreError: 11-0: meerkat-OST0024-osc-ffff88102f998c00: Communicating with xx.xx.xx.xx@tcp, operation ost_write failed with -3.
      LustreError: 11-0: meerkat-OST002a-osc-ffff88102f998c00: Communicating with xx.xx.xx.xx@tcp, operation ost_write failed with -3.
      LustreError: 11-0: meerkat-OST0009-osc-ffff88102f998c00: Communicating with xx.xx.xx.xx@tcp, operation ost_write failed with -3.
      LustreError: 11-0: meerkat-OST0036-osc-ffff88102f998c00: Communicating with xx.xx.xx.xx@tcp, operation ost_write failed with -3.
      LustreError: 11-0: meerkat-OST000e-osc-ffff88102f998c00: Communicating with xx.xx.xx.xx@tcp, operation ost_write failed with -3.
      LustreError: 11-0: meerkat-OST000d-osc-ffff88102f998c00: Communicating with xx.xx.xx.xx@tcp, operation ost_write failed with -3.
      LustreError: 11-0: meerkat-OST0024-osc-ffff88102f998c00: Communicating with xx.xx.xx.xx@tcp, operation ost_write failed with -3.
      LustreError: 11-0: meerkat-OST002a-osc-ffff88102f998c00: Communicating with xx.xx.xx.xx@tcp, operation ost_write failed with -3.
      LustreError: 11-0: meerkat-OST0009-osc-ffff88102f998c00: Communicating with xx.xx.xx.xx@tcp, operation ost_write failed with -3.
      LustreError: 11-0: meerkat-OST0036-osc-ffff88102f998c00: Communicating with xx.xx.xx.xx@tcp, operation ost_write failed with -3.
      LustreError: 11-0: meerkat-OST0036-osc-ffff88102f998c00: Communicating with xx.xx.xx.xx@tcp, operation ost_write failed with -3.
      LustreError: 11-0: meerkat-OST0026-osc-ffff88102f998c00: Communicating with xx.xx.xx.xx@tcp, operation ost_write failed with -3.
      LustreError: 11-0: meerkat-OST0026-osc-ffff88102f998c00: Communicating with xx.xx.xx.xx@tcp, operation ost_write failed with -3.
      LustreError: 11-0: meerkat-OST0025-osc-ffff88102f998c00: Communicating with xx.xx.xx.xx@tcp, operation ost_write failed with -3.
      LustreError: 11-0: meerkat-OST0036-osc-ffff88102f998c00: Communicating with xx.xx.xx.xx@tcp, operation ost_write failed with -3.
      LustreError: 11-0: meerkat-OST0017-osc-ffff88102f998c00: Communicating with xx.xx.xx.xx@tcp, operation ost_write failed with -3.
      LustreError: 11-0: meerkat-OST0036-osc-ffff88102f998c00: Communicating with xx.xx.xx.xx@tcp, operation ost_write failed with -3.
      LustreError: 11-0: meerkat-OST0036-osc-ffff88102f998c00: Communicating with xx.xx.xx.xx@tcp, operation ost_write failed with -3.
      LustreError: 11-0: meerkat-OST0031-osc-ffff88102f998c00: Communicating with xx.xx.xx.xx@tcp, operation ost_write failed with -3.
      LustreError: 11-0: meerkat-OST0025-osc-ffff88102f998c00: Communicating with xx.xx.xx.xx@tcp, operation ost_write failed with -3.
      LustreError: 11-0: meerkat-OST0036-osc-ffff88102f998c00: Communicating with xx.xx.xx.xx@tcp, operation ost_write failed with -3.
      LustreError: 11-0: meerkat-OST0036-osc-ffff88102f998c00: Communicating with xx.xx.xx.xx@tcp, operation ost_write failed with -3.
      LustreError: 11-0: meerkat-OST0036-osc-ffff88102f998c00: Communicating with xx.xx.xx.xx@tcp, operation ost_write failed with -3.
      LustreError: 11-0: meerkat-OST0036-osc-ffff88102f998c00: Communicating with xx.xx.xx.xx@tcp, operation ost_write failed with -3.
      LustreError: 11-0: meerkat-OST0031-osc-ffff88102f998c00: Communicating with xx.xx.xx.xx@tcp, operation ost_write failed with -3.
      LustreError: 11-0: meerkat-OST000c-osc-ffff88102f998c00: Communicating with xx.xx.xx.xx@tcp, operation ost_write failed with -3.

      Server log see attachment

      Attachments

        1. aggregated_messages_4-27.gz
          2.82 MB
        2. debugfs_out
          13 kB
        3. lustre_messages_today
          166 kB
        4. proc_out
          13 kB

        Activity

          People

            niu Niu Yawei (Inactive)
            haisong Haisong Cai (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: