Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-10048

osd-ldiskfs to truncate outside of main transaction

Details

    • Bug
    • Resolution: Fixed
    • Major
    • Lustre 2.13.0
    • Lustre 2.12.0, Lustre 2.10.4, Lustre 2.10.5
    • None
    • 9223372036854775807

    Description

      this is needed to implement (transaction first; locking next) order to unify locking among MDT/OST/OUT

      Attachments

        Issue Links

          is blocked by

          Bug - A problem which impairs or prevents the functions of the product. LU-11764 osd_declare_qid() should not rely on constant id

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open
          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. LU-11613 MDS and OSS locked up wait_transaction_locked+0x85/0xd0 [jbd2]

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. LU-8786 Terrible i/o performance of a test application doing repeatable writes and truncates

          • Major - Major loss of function.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. LU-11685 removing file names and freeing inodes are not atomic on the OST

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. LU-11465 OSS/MDS deadlock in 2.10.5

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. LU-5994 DT transaction start and object lock ordering

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. LU-13234 ldiskfs/namei.c:3310 ldiskfs_orphan_add+0x11e/0x2a0 [ldiskfs]

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. LU-12977 fix i_mutex for ldiskfs_truncate() in osd_execute_truncate()

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. LU-5152 Can't enforce block quota when unprivileged user change group

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. LU-8806 LFSCK hangs on MDT - osp_precreate_cleanup_orphans - cannot cleanup orphans: rc = -107

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Activity

            [LU-10048] osd-ldiskfs to truncate outside of main transaction
            eaujames Etienne Aujames added a comment -

            Hello,

            We have applied the https://review.whamcloud.com/43277 ("LU-10048 ofd: take local locks within transaction") + https://review.whamcloud.com/43278 ("LU-13093 osd: fix osd_attr_set race") on the problematic filesystem. The issue never occurred after.

            We were able to reproduce this issue in 5/10 min with many creations of small files and several threads doing file migrations ("lfs migrate" between OSTs).

            eaujames Etienne Aujames added a comment - Hello, We have applied the https://review.whamcloud.com/43277 (" LU-10048 ofd: take local locks within transaction") + https://review.whamcloud.com/43278 (" LU-13093 osd: fix osd_attr_set race") on the problematic filesystem. The issue never occurred after. We were able to reproduce this issue in 5/10 min with many creations of small files and several threads doing file migrations ("lfs migrate" between OSTs).
            eaujames Etienne Aujames added a comment -

            Hello Alex,

            Did you have the time to look to our backtrace?

            If you need more data from the crashdump I can get you some.

            eaujames Etienne Aujames added a comment - Hello Alex, Did you have the time to look to our backtrace? If you need more data from the crashdump I can get you some.
            eaujames Etienne Aujames added a comment -

            I have added the our backtrace to this tickets: crash_bt_jbd2_locked_20210412.log

            eaujames Etienne Aujames added a comment - I have added the our backtrace to this tickets: crash_bt_jbd2_locked_20210412.log
            eaujames Etienne Aujames added a comment -

            The issue seems to occurs relatively often with a lot of migrate threads (today 4 times on different OSTs).

            We will test the backports quickly on a production environment (maybe this week).

            I will try to get backtrace from the crashdump (manually triggered) tomorrow.

            eaujames Etienne Aujames added a comment - The issue seems to occurs relatively often with a lot of migrate threads (today 4 times on different OSTs). We will test the backports quickly on a production environment (maybe this week). I will try to get backtrace from the crashdump (manually triggered) tomorrow.
            bzzz Alex Zhuravlev added a comment -

            please, generate full backtrace (

            echo t >/proc/sysrq-trigger

            ) and attach to the ticket

            bzzz Alex Zhuravlev added a comment - please, generate full backtrace ( echo t >/proc/sysrq-trigger ) and attach to the ticket
            eaujames Etienne Aujames added a comment -

            The "LU-10048 osd: async truncate" has already landed on b2_12.

            eaujames Etienne Aujames added a comment - The " LU-10048 osd: async truncate" has already landed on b2_12.
            bzzz Alex Zhuravlev added a comment -

            as for fast journal - we've got number of nodes running all the tests on RAM-backed devices 24h a day.

            bzzz Alex Zhuravlev added a comment - as for fast journal - we've got number of nodes running all the tests on RAM-backed devices 24h a day.

            People

              bzzz Alex Zhuravlev
              bzzz Alex Zhuravlev
              Votes:
              0 Vote for this issue
              Watchers:
              19 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: