Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-12998

DNE3: tunable to disable directory creation on MDT

Details

    • Improvement
    • Resolution: Fixed
    • Major
    • Lustre 2.16.0
    • Lustre 2.14.0, Lustre 2.12.4

    Description

      In order to allow draining an MDT for removal from the filesystem, or to disable it temporarily, it makes sense to add an "mdt.*.no_create" parameter on the MDT similar to "obdfilter.*.no_precreate" on the OST, and a matching mount option "-o no_create". The OS_STATE_NOPRECREATE" flag should be set in obd_statfs from the MDT to let clients/MDS know that it should be skipped.

      This should result in new stripes directories skipping the MDT during selection for "lfs mkdir -i -1", auto striping, etc.

      Attachments

        Issue Links

          Activity

            [LU-12998] DNE3: tunable to disable directory creation on MDT

            bzzz, probably better to open a new ticket linked to this one and re-attach the logs there. We shouldn't be tracking problems on a ticket that is closed.

            adilger Andreas Dilger added a comment - bzzz , probably better to open a new ticket linked to this one and re-attach the logs there. We shouldn't be tracking problems on a ticket that is closed.

            Re-closing. Will remind Alex about opening a new ticket once CSDC work is complete

            find the logs attached.

            bzzz Alex Zhuravlev added a comment - Re-closing. Will remind Alex about opening a new ticket once CSDC work is complete find the logs attached.

            bzzz, I don't see any failures of this subtest in Maloo, so it isn't possible to check the debug logs to see where the error is coming from.

            It probably makes sense to file this as a separate LU ticket, and attach debug logs there, or see if a test patch with the right Test-Parameters: can reproduce the issue in autotest to collect the logs.

            adilger Andreas Dilger added a comment - bzzz , I don't see any failures of this subtest in Maloo, so it isn't possible to check the debug logs to see where the error is coming from. It probably makes sense to file this as a separate LU ticket, and attach debug logs there, or see if a test patch with the right Test-Parameters: can reproduce the issue in autotest to collect the logs.
            bzzz Alex Zhuravlev added a comment - - edited
            # OSTCOUNT=2 MDSCOUNT=2 ONLY=300l bash sanity.sh
            ....
            == sanity test 300l: non-root user to create dir under striped dir with stale layout ========================================================== 14:46:41 (1718894801)
            striped dir -i0 -c2 -H fnv_1a_64 /mnt/lustre/d300l.sanity/striped_dir
            fail_loc=0x80000158
            running as uid/gid/euid/egid 500/500/500/500, groups:
             [mkdir] [/mnt/lustre/d300l.sanity/striped_dir/test_dir]
            mkdir: cannot create directory '/mnt/lustre/d300l.sanity/striped_dir/test_dir': Operation not permitted
             sanity test_300l: @@@@@@ FAIL: create dir fails 
              Trace dump:
              = ./../tests/test-framework.sh:7090:error()
              = sanity.sh:26901:test_300l()
              = ./../tests/test-framework.sh:7435:run_one()
              = ./../tests/test-framework.sh:7498:run_one_logged()
              = ./../tests/test-framework.sh:7321:run_test()
              = sanity.sh:26907:main()
            Dumping lctl log to /tmp/ltest-logs/sanity.test_300l.*.1718894801.log
            Dumping logs only on local client.
            FAIL 300l (0s)
            

            my bisection points to this commit: LU-12998 lod: statfs upon nocreate check

            bzzz Alex Zhuravlev added a comment - - edited # OSTCOUNT=2 MDSCOUNT=2 ONLY=300l bash sanity.sh .... == sanity test 300l: non-root user to create dir under striped dir with stale layout ========================================================== 14:46:41 (1718894801) striped dir -i0 -c2 -H fnv_1a_64 /mnt/lustre/d300l.sanity/striped_dir fail_loc=0x80000158 running as uid/gid/euid/egid 500/500/500/500, groups: [mkdir] [/mnt/lustre/d300l.sanity/striped_dir/test_dir] mkdir: cannot create directory '/mnt/lustre/d300l.sanity/striped_dir/test_dir' : Operation not permitted sanity test_300l: @@@@@@ FAIL: create dir fails Trace dump: = ./../tests/test-framework.sh:7090:error() = sanity.sh:26901:test_300l() = ./../tests/test-framework.sh:7435:run_one() = ./../tests/test-framework.sh:7498:run_one_logged() = ./../tests/test-framework.sh:7321:run_test() = sanity.sh:26907:main() Dumping lctl log to /tmp/ltest-logs/sanity.test_300l.*.1718894801.log Dumping logs only on local client. FAIL 300l (0s) my bisection points to this commit: LU-12998 lod: statfs upon nocreate check
            pjones Peter Jones added a comment -

            Merged for 2.16

            pjones Peter Jones added a comment - Merged for 2.16

            "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/53437/
            Subject: LU-12998 lod: statfs upon nocreate check
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: 6dbb4c6c826133452c8c2da3937711c2030f99eb

            gerrit Gerrit Updater added a comment - "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/53437/ Subject: LU-12998 lod: statfs upon nocreate check Project: fs/lustre-release Branch: master Current Patch Set: Commit: 6dbb4c6c826133452c8c2da3937711c2030f99eb

            mdilger can you please update the lustre/doc/mount.lustre.8 man page to describe the no_create mount option for the server.

            adilger Andreas Dilger added a comment - mdilger can you please update the lustre/doc/mount.lustre.8 man page to describe the no_create mount option for the server.
            pjones Peter Jones added a comment - laisiyao can you please rebase https://review.whamcloud.com/c/fs/lustre-release/+/53437 ?

            Nathaniel, it looks like the patch didn't update the lustre/doc/mount.lustre.8 man page (my bad). I usually catch that on other people's patches, but didn't for my own. The parsing of the no_create option is done in the kernel in lmd_parse() since 2.14.0-ddn126 (EXA6.3.0+27 patches), so it should work but it should be documented properly it as well. The old "no_precreate" mount option is still there as well, mainly for backward compatibility. If there is a chance of new EMF being used to configure new 6.3.0-vanilla (or older) OSTs in a filesystem then it might be prudent to use "-o no_precreate" for now, and switch to no_create at some point in the future. Both of them would fail equally for older MDTs, but no_precreate would also work for older OSTs.

            adilger Andreas Dilger added a comment - Nathaniel, it looks like the patch didn't update the lustre/doc/mount.lustre.8 man page (my bad). I usually catch that on other people's patches, but didn't for my own. The parsing of the no_create option is done in the kernel in lmd_parse() since 2.14.0-ddn126 (EXA6.3.0+27 patches), so it should work but it should be documented properly it as well. The old " no_precreate " mount option is still there as well, mainly for backward compatibility. If there is a chance of new EMF being used to configure new 6.3.0-vanilla (or older) OSTs in a filesystem then it might be prudent to use " -o no_precreate " for now, and switch to no_create at some point in the future. Both of them would fail equally for older MDTs, but no_precreate would also work for older OSTs.

            laisiyao , adilger 

            Is there a patch for the mount.lustre(8) man page I'm not seeing?

            "no_precreate" is listed in the man page, but I believe this patch series changes the name and how it behaves on MDTs.

            utopiabound Nathaniel Clark added a comment - laisiyao , adilger   Is there a patch for the mount.lustre(8) man page I'm not seeing? "no_precreate" is listed in the man page, but I believe this patch series changes the name and how it behaves on MDTs.

            People

              laisiyao Lai Siyao
              adilger Andreas Dilger
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: