Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-2182

Add llapi_file_get_layout() function in liblustreapi

Details

    • Bug
    • Resolution: Fixed
    • Minor
    • Lustre 2.7.0
    • Lustre 2.4.0
    • 3
    • 5220

    Description

      Add llapi_file_get_layout() function in liblustreapi, and deprecate llapi_file_get_stripe(), as it does not take a 'size' parameter to pass in the buffer size information.

      Attachments

        Issue Links

          Activity

            [LU-2182] Add llapi_file_get_layout() function in liblustreapi

            It can be closed. Thanks

            nedbass Ned Bass (Inactive) added a comment - It can be closed. Thanks

            Is any additional work begin tracked under this ticket, or can it be closed?

            jlevi Jodi Levi (Inactive) added a comment - Is any additional work begin tracked under this ticket, or can it be closed?

            Works fine for me.

            $ sudo FSNAME=ertsul OSTCOUNT=2 FSTYPE=zfs ./lustre/tests/llmount.sh
            [...]
            Starting client: t222:  -o user_xattr,flock t222@tcp:/ertsul /mnt/ertsul
            Using TIMEOUT=20
            seting jobstats to procname_uid
            Setting ertsul.sys.jobid_var from disable to procname_uid
            Waiting 90 secs for update
            Updated after 6s: wanted 'procname_uid' got 'procname_uid'
            disable quota as required
            $ sudo ./lustre/utils/lctl pool_new ertsul.testpool
            Pool ertsul.testpool created
            $ sudo ./lustre/utils/lctl pool_add ertsul.testpool OST[0-1]
            OST ertsul-OST0000_UUID added to pool ertsul.testpool
            OST ertsul-OST0001_UUID added to pool ertsul.testpool
            $ sudo LFS=/home/bass6/lustre-release/lustre/utils/lfs ./lustre/tests/llapi_layout_test  -d /mnt/ertsul
             test  0: Read/write layout attributes then create a file ... pass
             test  1: Read test0 file by path and verify attributes ..... pass
             test  2: Read test0 file by FD and verify attributes ....... pass
             test  3: Read test0 file by FID and verify attributes ...... pass
             test  4: Verify compatibility with 'lfs setstripe' ......... pass
             test  5: llapi_layout_get_by_path ENOENT handling .......... pass
             test  6: llapi_layout_get_by_fd EBADF handling ............. pass
             test  7: llapi_layout_get_by_path EACCES handling .......... pass
             test  8: llapi_layout_get_by_path ENODATA handling ......... pass
             test  9: llapi_layout_pattern_set() EOPNOTSUPP handling .... pass
             test 10: stripe_count error handling ....................... pass
             test 11: stripe_size error handling ........................ pass
             test 12: pool_name error handling .......................... pass
             test 13: ost_index error handling .......................... pass
             test 14: llapi_layout_file_create error handling ........... pass
             test 15: Can't change striping attributes of existing file . pass
             test 16: Default stripe attributes are applied as expected . pass
             test 17: LLAPI_LAYOUT_WIDE is honored ...................... pass
             test 18: Setting pool with fsname.pool notation ............ pass
             test 19: Maximum length pool name is NULL-terminated ....... pass
             test 20: LLAPI_LAYOUT_DEFAULT is honored ................... pass
             test 21: llapi_layout_file_create fails for non-Lustre file  pass
             test 22: llapi_layout_file_create applied mode correctly ... pass
             test 23: llapi_layout_get_by_path fails for non-Lustre file  pass
             test 24: LAYOUT_GET_EXPECTED works with existing file ...... pass
             test 25: LAYOUT_GET_EXPECTED works with directory .......... pass
             test 26: LAYOUT_GET_EXPECTED partially specified parent .... pass
             test 27: LAYOUT_GET_EXPECTED with non existing file ........ pass
             test 28: LLAPI_LAYOUT_WIDE returned as expected ............ pass
            
            nedbass Ned Bass (Inactive) added a comment - Works fine for me. $ sudo FSNAME=ertsul OSTCOUNT=2 FSTYPE=zfs ./lustre/tests/llmount.sh [...] Starting client: t222: -o user_xattr,flock t222@tcp:/ertsul /mnt/ertsul Using TIMEOUT=20 seting jobstats to procname_uid Setting ertsul.sys.jobid_var from disable to procname_uid Waiting 90 secs for update Updated after 6s: wanted 'procname_uid' got 'procname_uid' disable quota as required $ sudo ./lustre/utils/lctl pool_new ertsul.testpool Pool ertsul.testpool created $ sudo ./lustre/utils/lctl pool_add ertsul.testpool OST[0-1] OST ertsul-OST0000_UUID added to pool ertsul.testpool OST ertsul-OST0001_UUID added to pool ertsul.testpool $ sudo LFS=/home/bass6/lustre-release/lustre/utils/lfs ./lustre/tests/llapi_layout_test -d /mnt/ertsul test 0: Read/write layout attributes then create a file ... pass test 1: Read test0 file by path and verify attributes ..... pass test 2: Read test0 file by FD and verify attributes ....... pass test 3: Read test0 file by FID and verify attributes ...... pass test 4: Verify compatibility with 'lfs setstripe' ......... pass test 5: llapi_layout_get_by_path ENOENT handling .......... pass test 6: llapi_layout_get_by_fd EBADF handling ............. pass test 7: llapi_layout_get_by_path EACCES handling .......... pass test 8: llapi_layout_get_by_path ENODATA handling ......... pass test 9: llapi_layout_pattern_set() EOPNOTSUPP handling .... pass test 10: stripe_count error handling ....................... pass test 11: stripe_size error handling ........................ pass test 12: pool_name error handling .......................... pass test 13: ost_index error handling .......................... pass test 14: llapi_layout_file_create error handling ........... pass test 15: Can't change striping attributes of existing file . pass test 16: Default stripe attributes are applied as expected . pass test 17: LLAPI_LAYOUT_WIDE is honored ...................... pass test 18: Setting pool with fsname.pool notation ............ pass test 19: Maximum length pool name is NULL-terminated ....... pass test 20: LLAPI_LAYOUT_DEFAULT is honored ................... pass test 21: llapi_layout_file_create fails for non-Lustre file pass test 22: llapi_layout_file_create applied mode correctly ... pass test 23: llapi_layout_get_by_path fails for non-Lustre file pass test 24: LAYOUT_GET_EXPECTED works with existing file ...... pass test 25: LAYOUT_GET_EXPECTED works with directory .......... pass test 26: LAYOUT_GET_EXPECTED partially specified parent .... pass test 27: LAYOUT_GET_EXPECTED with non existing file ........ pass test 28: LLAPI_LAYOUT_WIDE returned as expected ............ pass

            There is a bug in test 18 for llapi_layout_test.c which causes it to fail in my test setup. The error is due to the pool being set to lustre.* which is only valid if the file system name is lustre. It is possible to use a file system with a different name. Best way to handle this is make fsname in main() a global variable.

            simmonsja James A Simmons added a comment - There is a bug in test 18 for llapi_layout_test.c which causes it to fail in my test setup. The error is due to the pool being set to lustre.* which is only valid if the file system name is lustre. It is possible to use a file system with a different name. Best way to handle this is make fsname in main() a global variable.
            rread Robert Read added a comment -

            I agree with Andy, and would also like the new library be LGPL, if possible. Unfortunately, liblustreapi is a poor name for any library, not to mention It is confusingly similar to liblustre, and it isn't the Lustre API. If everything else is changing, then it makes sense to also choose a more suitable name for the new library as well, and then maybe we will be able to forget the old api, someday.

            rread Robert Read added a comment - I agree with Andy, and would also like the new library be LGPL, if possible. Unfortunately, liblustreapi is a poor name for any library, not to mention It is confusingly similar to liblustre, and it isn't the Lustre API. If everything else is changing, then it makes sense to also choose a more suitable name for the new library as well, and then maybe we will be able to forget the old api, someday.

            Andy I brought up the discussion on hpdd-discuss about liblustreapi2.0. Feel free to voice your opinion.

            simmonsja James A Simmons added a comment - Andy I brought up the discussion on hpdd-discuss about liblustreapi2.0. Feel free to voice your opinion.
            afn Andy Nelson added a comment -

            I guess I would have though the best reason for a new library not mixed with the old is that the old one contains

            1) redundant functionality that
            2) is implemented in a much less user friendly way than the new api and
            3) had/has problems with maintenance and bitrot over the years

            From my end, as a user of the api, I'd rather only have to maintain one set of code
            calling one set of functionality. The transition over to the new api will have its own pains
            and such, and the old one is already causing me problems between the requirements that
            my code compile/run on lustre 1.8x (crays) and on lustre 2.x (most everything else) and sill
            accomodate the fact that none of these versions have a lustre_idl.h which compiles
            in userspace at all. That means I have to hack the system various system ones for each version
            and carry them around with me until I can just forget about that old api altogether.

            The sooner the old api and the libraries implementing it die completely, the better.

            afn Andy Nelson added a comment - I guess I would have though the best reason for a new library not mixed with the old is that the old one contains 1) redundant functionality that 2) is implemented in a much less user friendly way than the new api and 3) had/has problems with maintenance and bitrot over the years From my end, as a user of the api, I'd rather only have to maintain one set of code calling one set of functionality. The transition over to the new api will have its own pains and such, and the old one is already causing me problems between the requirements that my code compile/run on lustre 1.8x (crays) and on lustre 2.x (most everything else) and sill accomodate the fact that none of these versions have a lustre_idl.h which compiles in userspace at all. That means I have to hack the system various system ones for each version and carry them around with me until I can just forget about that old api altogether. The sooner the old api and the libraries implementing it die completely, the better.

            Is this so the new libraries can be LGPL? Understand I'm not against moving to LGPL. The reason I ask is because I'm involved in the LU-5030 which is a rework of handling the parameters code. This would impact me as well.

            Not to confuse users we should try to stick to the a similar library name. Can you create 2.0 version of the library of the same name and change the license? This way we can end up with a liblustreapi2.so with a LGPL license.

            simmonsja James A Simmons added a comment - Is this so the new libraries can be LGPL? Understand I'm not against moving to LGPL. The reason I ask is because I'm involved in the LU-5030 which is a rework of handling the parameters code. This would impact me as well. Not to confuse users we should try to stick to the a similar library name. Can you create 2.0 version of the library of the same name and change the license? This way we can end up with a liblustreapi2.so with a LGPL license.
            rread Robert Read added a comment -

            This new API looks great. I suggest we consider putting this into a new library, though, instead of adding it to the existing liblustreapi. Ideally, we would also move hsm functions into the new library as well, and update the posix copytool (and other users of the hsm api) to use the new library. On first glance it looks like only a few functions such as fd2fid, path2fid will need to be reimplemented for the copytool to stop using old liblustreapi completely, but those pretty trivial.

            rread Robert Read added a comment - This new API looks great. I suggest we consider putting this into a new library, though, instead of adding it to the existing liblustreapi. Ideally, we would also move hsm functions into the new library as well, and update the posix copytool (and other users of the hsm api) to use the new library. On first glance it looks like only a few functions such as fd2fid, path2fid will need to be reimplemented for the copytool to stop using old liblustreapi completely, but those pretty trivial.

            Could you please file a bug for tracking that issue separately

            See LU-2786.

            nedbass Ned Bass (Inactive) added a comment - Could you please file a bug for tracking that issue separately See LU-2786 .

            People

              bogl Bob Glossman (Inactive)
              sebastien.buisson Sebastien Buisson (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              19 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: