[LU-16077] Cannot use tbf to filter brw request per effective uid/gid, inode attr ids is used instead

• Status: Resolved
• Project: Lustre
• Fix Version/s: Lustre 2.16.0
• Type: Bug
• Priority: Major
• Reporter: Etienne Aujames
• Assignee: Etienne Aujames
• Resolution: Fixed
• Epic/Theme: OSC, QoS-TBF
• Severity: 3

Description

Recently, we had to limit a user on an OST of products (lot of libs in it). But we were not able to do that with TBF. After some tries, we were able to set a limit, but with the GID's owner.
Setting this limit, globally decrease the bandwidth for everybody (all files in the products have the same uid/gid).

The TBF rules on an OST get the uid/gid values from:

static int ost_tbf_id_cli_set(struct ptlrpc_request *req,
                              struct tbf_id *id)         
        struct ost_body *body; 
...
                 id->ti_uid = body->oa.o_uid;
                 id->ti_gid = body->oa.o_gid;

These values are filled by the osc here:

int osc_build_rpc(const struct lu_env *env, struct client_obd *cli,
                  struct list_head *ext_list, int cmd)             
....
         crattr->cra_oa = oa;                      
         cl_req_attr_set(env, osc2cl(obj), crattr);


static int                                                            
osc_brw_prep_request(int cmd, struct client_obd *cli, struct obdo *oa,
                     u32 page_count, struct brw_page **pga,           
                     struct ptlrpc_request **reqp, int resend)        
.....
        lustre_set_wire_obdo(&req->rq_import->imp_connect_data, &body->oa, oa); 
                                                                                
        /* For READ and WRITE, we can't fill o_uid and o_gid using from_kuid()  
         * and from_kgid(), because they are asynchronous. Fortunately, variable
         * oa contains valid o_uid and o_gid in these two operations.           
         * Besides, filling o_uid and o_gid is enough for nrs-tbf, see LU-9658. 
         * OBD_MD_FLUID and OBD_MD_FLUID is not set in order to avoid breaking  
         * other process logic */                                               
        body->oa.o_uid = oa->o_uid;                                             
        body->oa.o_gid = oa->o_gid;                                             

So the uid/gid use to filter the request came directly from the inode attr.

Issue Links

• is related to: LU-17149 TBF: req_capsule_extend() ASSERTION( ...

Comments

[Etienne Aujames]

I think this issue can be fixed the same way that for jobid:

int vvp_io_init(const struct lu_env *env, struct cl_object *obj,
                struct cl_io *io)                               
....
        if (io->ci_type == CIT_READ || io->ci_type == CIT_WRITE) {       
                size_t count;                                            
                struct ll_inode_info *lli = ll_i2info(inode);            
                                                                         
                count = io->u.ci_rw.crw_count;                           
                /* "If nbyte is 0, read() will return 0 and have no other
                 *  results."  -- Single Unix Spec */                    
                if (count == 0)                                          
                        result = 1;                                      
                else                                                     
                        vio->vui_tot_count = count;                      
                                                                         
                /* for read/write, we store the jobid in the inode, and  
                 * it'll be fetched by osc when building RPC.            
                 *                                                       
                 * it's not accurate if the file is shared by different  
                 * jobs.                                                 
                 */                                                      
                lustre_get_jobid(lli->lli_jobid, sizeof(lli->lli_jobid));

static void vvp_req_attr_set(const struct lu_env *env, struct cl_object *obj,
                             struct cl_req_attr *attr)                       
....
         memcpy(attr->cra_jobid, ll_i2info(inode)->lli_jobid,
                sizeof(attr->cra_jobid));                    

int osc_build_rpc(const struct lu_env *env, struct client_obd *cli,
                  struct list_head *ext_list, int cmd)             
....
        lustre_msg_set_jobid(req->rq_reqmsg, crattr->cra_jobid);

[Gerrit Updater]

"Etienne AUJAMES <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/48213
Subject: LU-16077 osc: restore uid/gid from inode info for brw request
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 790029bed88b1acb0624333df7b75ccb7d945698

[Gerrit Updater]

"Etienne AUJAMES <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/48235
Subject: LU-16077 tbf: pb_uid/pb_gid ptlrpc_body fields for tbf rules
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: a53baca225a9b7640d022b3ae28b6bbe478e03c0

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/48235/
Subject: LU-16077 tbf: pb_uid/pb_gid ptlrpc_body fields for TBF rules
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 0544c108c12c87a43562b4ef31120448bf0018c8

[Peter Jones]

Landed for 2.16

[Andreas Dilger]

I noticed a couple of issues with the patch after it landed that need to be fixed before 2.16 is released. Comments on the patch.

[Gerrit Updater]

"Etienne AUJAMES <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/51122
Subject: LU-16077 ptlrpc: Fix ptlrpc_body_v2 with pb_uid/pb_gid
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: d45e48db07c8820e86ad60faae153ed0f6affcb6

[Gerrit Updater]

"Etienne AUJAMES <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/51536
Subject: LU-16077 tbf: pb_uid/pb_gid ptlrpc_body fields for TBF rules
Project: fs/lustre-release
Branch: b2_15
Current Patch Set: 1
Commit: 160408f1e5947c3fe2748e4eeff2340e18b293ca

[Gerrit Updater]

"Etienne AUJAMES <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/51537
Subject: LU-16077 ptlrpc: Fix ptlrpc_body_v2 with pb_uid/pb_gid
Project: fs/lustre-release
Branch: b2_15
Current Patch Set: 1
Commit: f2ea4a0f9d29f463b53e92bdb6bd07d886bb5387

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/51122/
Subject: LU-16077 ptlrpc: Fix ptlrpc_body_v2 with pb_uid/pb_gid
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: afe5813a482478642acc0aacfe195d465e0f1e2c