Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-16077

Cannot use tbf to filter brw request per effective uid/gid, inode attr ids is used instead

Details

    • Bug
    • Resolution: Fixed
    • Major
    • Lustre 2.16.0
    • None
    • None
    • 3
    • 9223372036854775807

    Description

      Recently, we had to limit a user on an OST of products (lot of libs in it). But we were not able to do that with TBF. After some tries, we were able to set a limit, but with the GID's owner.
      Setting this limit, globally decrease the bandwidth for everybody (all files in the products have the same uid/gid).

      The TBF rules on an OST get the uid/gid values from:

      static int ost_tbf_id_cli_set(struct ptlrpc_request *req,
                                    struct tbf_id *id)         
              struct ost_body *body; 
      ...
                       id->ti_uid = body->oa.o_uid;
                       id->ti_gid = body->oa.o_gid;
      

      These values are filled by the osc here:

      int osc_build_rpc(const struct lu_env *env, struct client_obd *cli,
                        struct list_head *ext_list, int cmd)             
      ....
               crattr->cra_oa = oa;                      
               cl_req_attr_set(env, osc2cl(obj), crattr);
      
      
      static int                                                            
      osc_brw_prep_request(int cmd, struct client_obd *cli, struct obdo *oa,
                           u32 page_count, struct brw_page **pga,           
                           struct ptlrpc_request **reqp, int resend)        
      .....
              lustre_set_wire_obdo(&req->rq_import->imp_connect_data, &body->oa, oa); 
                                                                                      
              /* For READ and WRITE, we can't fill o_uid and o_gid using from_kuid()  
               * and from_kgid(), because they are asynchronous. Fortunately, variable
               * oa contains valid o_uid and o_gid in these two operations.           
               * Besides, filling o_uid and o_gid is enough for nrs-tbf, see LU-9658. 
               * OBD_MD_FLUID and OBD_MD_FLUID is not set in order to avoid breaking  
               * other process logic */                                               
              body->oa.o_uid = oa->o_uid;                                             
              body->oa.o_gid = oa->o_gid;                                             
      

      So the uid/gid use to filter the request came directly from the inode attr.

      Attachments

        Issue Links

          Activity

            [LU-16077] Cannot use tbf to filter brw request per effective uid/gid, inode attr ids is used instead

            "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/51122/
            Subject: LU-16077 ptlrpc: Fix ptlrpc_body_v2 with pb_uid/pb_gid
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: afe5813a482478642acc0aacfe195d465e0f1e2c

            gerrit Gerrit Updater added a comment - "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/51122/ Subject: LU-16077 ptlrpc: Fix ptlrpc_body_v2 with pb_uid/pb_gid Project: fs/lustre-release Branch: master Current Patch Set: Commit: afe5813a482478642acc0aacfe195d465e0f1e2c

            "Etienne AUJAMES <eaujames@ddn.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/51537
            Subject: LU-16077 ptlrpc: Fix ptlrpc_body_v2 with pb_uid/pb_gid
            Project: fs/lustre-release
            Branch: b2_15
            Current Patch Set: 1
            Commit: f2ea4a0f9d29f463b53e92bdb6bd07d886bb5387

            gerrit Gerrit Updater added a comment - "Etienne AUJAMES <eaujames@ddn.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/51537 Subject: LU-16077 ptlrpc: Fix ptlrpc_body_v2 with pb_uid/pb_gid Project: fs/lustre-release Branch: b2_15 Current Patch Set: 1 Commit: f2ea4a0f9d29f463b53e92bdb6bd07d886bb5387

            "Etienne AUJAMES <eaujames@ddn.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/51536
            Subject: LU-16077 tbf: pb_uid/pb_gid ptlrpc_body fields for TBF rules
            Project: fs/lustre-release
            Branch: b2_15
            Current Patch Set: 1
            Commit: 160408f1e5947c3fe2748e4eeff2340e18b293ca

            gerrit Gerrit Updater added a comment - "Etienne AUJAMES <eaujames@ddn.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/51536 Subject: LU-16077 tbf: pb_uid/pb_gid ptlrpc_body fields for TBF rules Project: fs/lustre-release Branch: b2_15 Current Patch Set: 1 Commit: 160408f1e5947c3fe2748e4eeff2340e18b293ca

            "Etienne AUJAMES <eaujames@ddn.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/51122
            Subject: LU-16077 ptlrpc: Fix ptlrpc_body_v2 with pb_uid/pb_gid
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: d45e48db07c8820e86ad60faae153ed0f6affcb6

            gerrit Gerrit Updater added a comment - "Etienne AUJAMES <eaujames@ddn.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/51122 Subject: LU-16077 ptlrpc: Fix ptlrpc_body_v2 with pb_uid/pb_gid Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: d45e48db07c8820e86ad60faae153ed0f6affcb6

            I noticed a couple of issues with the patch after it landed that need to be fixed before 2.16 is released. Comments on the patch.

            adilger Andreas Dilger added a comment - I noticed a couple of issues with the patch after it landed that need to be fixed before 2.16 is released. Comments on the patch.
            pjones Peter Jones added a comment -

            Landed for 2.16

            pjones Peter Jones added a comment - Landed for 2.16

            "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/48235/
            Subject: LU-16077 tbf: pb_uid/pb_gid ptlrpc_body fields for TBF rules
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: 0544c108c12c87a43562b4ef31120448bf0018c8

            gerrit Gerrit Updater added a comment - "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/48235/ Subject: LU-16077 tbf: pb_uid/pb_gid ptlrpc_body fields for TBF rules Project: fs/lustre-release Branch: master Current Patch Set: Commit: 0544c108c12c87a43562b4ef31120448bf0018c8

            "Etienne AUJAMES <eaujames@ddn.com>" uploaded a new patch: https://review.whamcloud.com/48235
            Subject: LU-16077 tbf: pb_uid/pb_gid ptlrpc_body fields for tbf rules
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: a53baca225a9b7640d022b3ae28b6bbe478e03c0

            gerrit Gerrit Updater added a comment - "Etienne AUJAMES <eaujames@ddn.com>" uploaded a new patch: https://review.whamcloud.com/48235 Subject: LU-16077 tbf: pb_uid/pb_gid ptlrpc_body fields for tbf rules Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: a53baca225a9b7640d022b3ae28b6bbe478e03c0

            "Etienne AUJAMES <eaujames@ddn.com>" uploaded a new patch: https://review.whamcloud.com/48213
            Subject: LU-16077 osc: restore uid/gid from inode info for brw request
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: 790029bed88b1acb0624333df7b75ccb7d945698

            gerrit Gerrit Updater added a comment - "Etienne AUJAMES <eaujames@ddn.com>" uploaded a new patch: https://review.whamcloud.com/48213 Subject: LU-16077 osc: restore uid/gid from inode info for brw request Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 790029bed88b1acb0624333df7b75ccb7d945698

            I think this issue can be fixed the same way that for jobid:

            int vvp_io_init(const struct lu_env *env, struct cl_object *obj,
                            struct cl_io *io)                               
            ....
                    if (io->ci_type == CIT_READ || io->ci_type == CIT_WRITE) {       
                            size_t count;                                            
                            struct ll_inode_info *lli = ll_i2info(inode);            
                                                                                     
                            count = io->u.ci_rw.crw_count;                           
                            /* "If nbyte is 0, read() will return 0 and have no other
                             *  results."  -- Single Unix Spec */                    
                            if (count == 0)                                          
                                    result = 1;                                      
                            else                                                     
                                    vio->vui_tot_count = count;                      
                                                                                     
                            /* for read/write, we store the jobid in the inode, and  
                             * it'll be fetched by osc when building RPC.            
                             *                                                       
                             * it's not accurate if the file is shared by different  
                             * jobs.                                                 
                             */                                                      
                            lustre_get_jobid(lli->lli_jobid, sizeof(lli->lli_jobid));
            
            static void vvp_req_attr_set(const struct lu_env *env, struct cl_object *obj,
                                         struct cl_req_attr *attr)                       
            ....
                     memcpy(attr->cra_jobid, ll_i2info(inode)->lli_jobid,
                            sizeof(attr->cra_jobid));                    
            
            int osc_build_rpc(const struct lu_env *env, struct client_obd *cli,
                              struct list_head *ext_list, int cmd)             
            ....
                    lustre_msg_set_jobid(req->rq_reqmsg, crattr->cra_jobid);
            
            eaujames Etienne Aujames added a comment - I think this issue can be fixed the same way that for jobid: int vvp_io_init( const struct lu_env *env, struct cl_object *obj, struct cl_io *io) .... if (io->ci_type == CIT_READ || io->ci_type == CIT_WRITE) { size_t count; struct ll_inode_info *lli = ll_i2info(inode); count = io->u.ci_rw.crw_count; /* "If nbyte is 0, read() will return 0 and have no other * results." -- Single Unix Spec */ if (count == 0) result = 1; else vio->vui_tot_count = count; /* for read/write, we store the jobid in the inode, and * it'll be fetched by osc when building RPC. * * it's not accurate if the file is shared by different * jobs. */ lustre_get_jobid(lli->lli_jobid, sizeof (lli->lli_jobid)); static void vvp_req_attr_set( const struct lu_env *env, struct cl_object *obj, struct cl_req_attr *attr) .... memcpy(attr->cra_jobid, ll_i2info(inode)->lli_jobid, sizeof (attr->cra_jobid)); int osc_build_rpc( const struct lu_env *env, struct client_obd *cli, struct list_head *ext_list, int cmd) .... lustre_msg_set_jobid(req->rq_reqmsg, crattr->cra_jobid);

            People

              eaujames Etienne Aujames
              eaujames Etienne Aujames
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: