Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-17575

Require specific enable flag for lfs migrate -m (directory migration)

Details

    • Bug
    • Resolution: Unresolved
    • Minor
    • None
    • None
    • None
    • 3
    • 9223372036854775807

    Description

      We know that there are still major corruption possibilities with lfs migrate -m (directory migration).

      Let's add a warning and require a flag to enable this functionality

      Attachments

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. LU-17810 lfs migrate -m leaves 1-stripe dir in a two-level "striped" form

          • Major - Major loss of function.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. LU-17565 migrate racing with another ops can corrupt filesystem

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. LU-17620 Lustre client gets stuck in lov_layout_wait() awaiting for IO completion

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Activity

            [LU-17575] Require specific enable flag for lfs migrate -m (directory migration)
            cfaber Colin Faber added a comment -

            hi spitzcor,

            The two current known issues are now linked, we'll continue to link additional issues as they're discovered.

            cfaber Colin Faber added a comment - hi spitzcor , The two current known issues are now linked, we'll continue to link additional issues as they're discovered.
            spitzcor Cory Spitz added a comment -

            Is there a list of known defects that we can enumerate and link here? Then, it would be easier to track when the (known) defects are fixed and the warning could be removed.

            spitzcor Cory Spitz added a comment - Is there a list of known defects that we can enumerate and link here? Then, it would be easier to track when the (known) defects are fixed and the warning could be removed.
            timday Tim Day added a comment -

            From Andreas in Slack:

             

            Generally, `lfs migrate -m` is working OK, but there are still race conditions when the directory or its files are actively being modified by other processes during migration, or errors are hit during migration (e.g. target full or goes into recovery).  we've been incrementally fixing issues that we have found, but discourage users from doing mass migrations of directories between MDTs, and instead recommend to use "passive" space balancing - `lfs setdirstripe -D -c 1 -i -1 --max-inherit-rr=N DIR` or specific `lfs setdirstripe -D -c 1 -i N DIR` to steer new subdirectories to less-full MDTs.

            timday Tim Day added a comment - From Andreas in Slack:   Generally, `lfs migrate -m` is working OK, but there are still race conditions when the directory or its files are actively being modified by other processes during migration, or errors are hit during migration (e.g. target full or goes into recovery).  we've been incrementally fixing issues that we have found, but discourage users from doing mass migrations of directories between MDTs, and instead recommend to use "passive" space balancing - `lfs setdirstripe -D -c 1 -i -1 --max-inherit-rr=N DIR` or specific `lfs setdirstripe -D -c 1 -i N DIR` to steer new subdirectories to less-full MDTs.
            timday Tim Day added a comment -

            A warning makes sense, it seems. But I'm curious what kind of corruption could happen? The only major ticket I see is related to interrupted migrates: https://jira.whamcloud.com/browse/LU-14211.

            timday Tim Day added a comment - A warning makes sense, it seems. But I'm curious what kind of corruption could happen? The only major ticket I see is related to interrupted migrates: https://jira.whamcloud.com/browse/LU-14211 .

            People

              wc-triage WC Triage
              cfaber Colin Faber
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated: