[LU-16077] Cannot use tbf to filter brw request per effective uid/gid, inode attr ids is used instead - Whamcloud Community JIRA

Details

Type: Bug
Resolution: Fixed
Priority: Major
Fix Version/s: Lustre 2.16.0
Affects Version/s: None
Labels:
None

Epic/Theme:
- OSC
- QoS-TBF
Severity:
3
Rank (Obsolete):
9223372036854775807

Description

Recently, we had to limit a user on an OST of products (lot of libs in it). But we were not able to do that with TBF. After some tries, we were able to set a limit, but with the GID's owner.
Setting this limit, globally decrease the bandwidth for everybody (all files in the products have the same uid/gid).

The TBF rules on an OST get the uid/gid values from:

static int ost_tbf_id_cli_set(struct ptlrpc_request *req,
                              struct tbf_id *id)         
        struct ost_body *body; 
...
                 id->ti_uid = body->oa.o_uid;
                 id->ti_gid = body->oa.o_gid;

These values are filled by the osc here:

int osc_build_rpc(const struct lu_env *env, struct client_obd *cli,
                  struct list_head *ext_list, int cmd)             
....
         crattr->cra_oa = oa;                      
         cl_req_attr_set(env, osc2cl(obj), crattr);


static int                                                            
osc_brw_prep_request(int cmd, struct client_obd *cli, struct obdo *oa,
                     u32 page_count, struct brw_page **pga,           
                     struct ptlrpc_request **reqp, int resend)        
.....
        lustre_set_wire_obdo(&req->rq_import->imp_connect_data, &body->oa, oa); 
                                                                                
        /* For READ and WRITE, we can't fill o_uid and o_gid using from_kuid()  
         * and from_kgid(), because they are asynchronous. Fortunately, variable
         * oa contains valid o_uid and o_gid in these two operations.           
         * Besides, filling o_uid and o_gid is enough for nrs-tbf, see LU-9658. 
         * OBD_MD_FLUID and OBD_MD_FLUID is not set in order to avoid breaking  
         * other process logic */                                               
        body->oa.o_uid = oa->o_uid;                                             
        body->oa.o_gid = oa->o_gid;

So the uid/gid use to filter the request came directly from the inode attr.

Attachments

Issue Links

is related to

LU-17149 TBF: req_capsule_extend() ASSERTION( fmt->rf_fields[i].nr >= old->rf_fields[i].nr )

Resolved

LU-18180 UID in req_buffer_history

Open

LU-3468 Add UID/GID into RPC request

Resolved

Activity

[LU-16077] Cannot use tbf to filter brw request per effective uid/gid, inode attr ids is used instead

Andreas Dilger made changes - 29/Aug/24 7:48 PM

Link

New: This issue is related to ~~LU-3468~~ [ ~~LU-3468~~ ]

Andreas Dilger made changes - 29/Aug/24 7:47 PM

Link

New: This issue is related to LU-18180 [ LU-18180 ]

Etienne Aujames made changes - 27/Sep/23 11:18 AM

Link

New: This issue is related to ~~LU-17149~~ [ ~~LU-17149~~ ]

Peter Jones made changes - 31/Aug/23 2:49 PM

Resolution		New: Fixed [ 1 ]
Status	Original: Reopened [ 4 ]	New: Resolved [ 5 ]

Andreas Dilger made changes - 24/Apr/23 6:13 PM

Priority

Original: Minor [ 4 ]

New: Major [ 3 ]

Andreas Dilger made changes - 24/Apr/23 6:13 PM

Resolution	Original: Fixed [ 1 ]
Status	Original: Resolved [ 5 ]	New: Reopened [ 4 ]

Peter Jones made changes - 18/Apr/23 12:11 PM

Fix Version/s		New: Lustre 2.16.0 [ 15190 ]
Resolution		New: Fixed [ 1 ]
Status	Original: Open [ 1 ]	New: Resolved [ 5 ]

Peter Jones made changes - 05/Aug/22 1:43 PM

Assignee

Original: WC Triage [ wc-triage ]

New: Etienne Aujames [ eaujames ]

Etienne Aujames made changes - 05/Aug/22 10:51 AM

Description

Original: Recently, we had to limit a user on an OST of products (lot of libs in it). But we were not able to do that with TBF. After some tries, we were able to set a limit, but with the GID's owner.
Setting this limit, globally decrease the bandwidth for everybody (all files in the products have the same uid/gid).

The TBF rules on an OST get the uid/gid values from:
{code:C}
static int ost_tbf_id_cli_set(struct ptlrpc_request *req,
                              struct tbf_id *id)
        struct ost_body *body;
...
                 id->ti_uid = body->oa.o_uid;
                 id->ti_gid = body->oa.o_gid;
{code}

These values are filled by the osc here:
{code: C}
int osc_build_rpc(const struct lu_env *env, struct client_obd *cli,
                  struct list_head *ext_list, int cmd)
....
         crattr->cra_oa = oa;
         cl_req_attr_set(env, osc2cl(obj), crattr);

static int
osc_brw_prep_request(int cmd, struct client_obd *cli, struct obdo *oa,
                     u32 page_count, struct brw_page **pga,
                     struct ptlrpc_request **reqp, int resend)
.....
        lustre_set_wire_obdo(&req->rq_import->imp_connect_data, &body->oa, oa);

        /* For READ and WRITE, we can't fill o_uid and o_gid using from_kuid()
         * and from_kgid(), because they are asynchronous. Fortunately, variable
         * oa contains valid o_uid and o_gid in these two operations.
         * Besides, filling o_uid and o_gid is enough for nrs-tbf, see ~~LU-9658~~.
         * OBD_MD_FLUID and OBD_MD_FLUID is not set in order to avoid breaking
         * other process logic */
        body->oa.o_uid = oa->o_uid;
        body->oa.o_gid = oa->o_gid;
{code}

So the uid/gid use to filter the request came directly from the inode attr.

New: Recently, we had to limit a user on an OST of products (lot of libs in it). But we were not able to do that with TBF. After some tries, we were able to set a limit, but with the GID's owner.
Setting this limit, globally decrease the bandwidth for everybody (all files in the products have the same uid/gid).

The TBF rules on an OST get the uid/gid values from:
{code:C}
static int ost_tbf_id_cli_set(struct ptlrpc_request *req,
                              struct tbf_id *id)
        struct ost_body *body;
...
                 id->ti_uid = body->oa.o_uid;
                 id->ti_gid = body->oa.o_gid;
{code}

These values are filled by the osc here:
{code:C}
int osc_build_rpc(const struct lu_env *env, struct client_obd *cli,
                  struct list_head *ext_list, int cmd)
....
         crattr->cra_oa = oa;
         cl_req_attr_set(env, osc2cl(obj), crattr);

static int
osc_brw_prep_request(int cmd, struct client_obd *cli, struct obdo *oa,
                     u32 page_count, struct brw_page **pga,
                     struct ptlrpc_request **reqp, int resend)
.....
        lustre_set_wire_obdo(&req->rq_import->imp_connect_data, &body->oa, oa);

        /* For READ and WRITE, we can't fill o_uid and o_gid using from_kuid()
         * and from_kgid(), because they are asynchronous. Fortunately, variable
         * oa contains valid o_uid and o_gid in these two operations.
         * Besides, filling o_uid and o_gid is enough for nrs-tbf, see ~~LU-9658~~.
         * OBD_MD_FLUID and OBD_MD_FLUID is not set in order to avoid breaking
         * other process logic */
        body->oa.o_uid = oa->o_uid;
        body->oa.o_gid = oa->o_gid;
{code}

So the uid/gid use to filter the request came directly from the inode attr.

Etienne Aujames made changes - 05/Aug/22 10:50 AM

Description

Original: Recently, we had to limit a user on an OST of products (lot of libs in it). But we were not able to do that with TBF. After some tries, we were able to set a limit, but with the GID's owner.
Setting this limit, globally decrease the bandwidth for everybody (all files in the products have the same uid/gid).

The TBF rules on an OST get the uid/gid values from:
{noformat}
static int ost_tbf_id_cli_set(struct ptlrpc_request *req,
                              struct tbf_id *id)
        struct ost_body *body;
...
                 id->ti_uid = body->oa.o_uid;
                 id->ti_gid = body->oa.o_gid;
{noformat}

These values are filled by the osc here:
{noformat}
int osc_build_rpc(const struct lu_env *env, struct client_obd *cli,
                  struct list_head *ext_list, int cmd)
....
         crattr->cra_oa = oa;
         cl_req_attr_set(env, osc2cl(obj), crattr);

static int
osc_brw_prep_request(int cmd, struct client_obd *cli, struct obdo *oa,
                     u32 page_count, struct brw_page **pga,
                     struct ptlrpc_request **reqp, int resend)
.....
        lustre_set_wire_obdo(&req->rq_import->imp_connect_data, &body->oa, oa);

        /* For READ and WRITE, we can't fill o_uid and o_gid using from_kuid()
         * and from_kgid(), because they are asynchronous. Fortunately, variable
         * oa contains valid o_uid and o_gid in these two operations.
         * Besides, filling o_uid and o_gid is enough for nrs-tbf, see ~~LU-9658~~.
         * OBD_MD_FLUID and OBD_MD_FLUID is not set in order to avoid breaking
         * other process logic */
        body->oa.o_uid = oa->o_uid;
        body->oa.o_gid = oa->o_gid;
{noformat}

So the uid/gid use to filter the request came directly from the inode attr.

New: Recently, we had to limit a user on an OST of products (lot of libs in it). But we were not able to do that with TBF. After some tries, we were able to set a limit, but with the GID's owner.
Setting this limit, globally decrease the bandwidth for everybody (all files in the products have the same uid/gid).

The TBF rules on an OST get the uid/gid values from:
{code:C}
static int ost_tbf_id_cli_set(struct ptlrpc_request *req,
                              struct tbf_id *id)
        struct ost_body *body;
...
                 id->ti_uid = body->oa.o_uid;
                 id->ti_gid = body->oa.o_gid;
{code}

These values are filled by the osc here:
{code: C}
int osc_build_rpc(const struct lu_env *env, struct client_obd *cli,
                  struct list_head *ext_list, int cmd)
....
         crattr->cra_oa = oa;
         cl_req_attr_set(env, osc2cl(obj), crattr);

static int
osc_brw_prep_request(int cmd, struct client_obd *cli, struct obdo *oa,
                     u32 page_count, struct brw_page **pga,
                     struct ptlrpc_request **reqp, int resend)
.....
        lustre_set_wire_obdo(&req->rq_import->imp_connect_data, &body->oa, oa);

        /* For READ and WRITE, we can't fill o_uid and o_gid using from_kuid()
         * and from_kgid(), because they are asynchronous. Fortunately, variable
         * oa contains valid o_uid and o_gid in these two operations.
         * Besides, filling o_uid and o_gid is enough for nrs-tbf, see ~~LU-9658~~.
         * OBD_MD_FLUID and OBD_MD_FLUID is not set in order to avoid breaking
         * other process logic */
        body->oa.o_uid = oa->o_uid;
        body->oa.o_gid = oa->o_gid;
{code}

So the uid/gid use to filter the request came directly from the inode attr.

Etienne Aujames created issue - 05/Aug/22 8:52 AM

People

Assignee:: Etienne Aujames

Reporter:: Etienne Aujames

Votes:: 0 Vote for this issue

Watchers:: 9 Start watching this issue

Dates

Created:: 05/Aug/22 8:52 AM

Updated:: 29/Aug/24 7:48 PM

Resolved:: 31/Aug/23 2:49 PM