[LU-651] Strange error message "is it ok to have flags 0x... and 0x... in the same brw?"

• Status: Resolved
• Project: Lustre
• Affects Version/s: Lustre 2.0.0, Lustre 2.1.0
• Fix Version/s: Lustre 2.2.0, Lustre 2.1.2
• Type: Bug
• Priority: Minor
• Reporter: Aurelien Degremont (Inactive)
• Assignee: Zhenyu Xu
• Resolution: Fixed
• Severity: 3

Description

We often face the very weird message "is it ok to have flags 0x... and 0x... in the same brw?".
It seems this message is not a real issue, the system seems stable. So, this is not a CERROR().

This message has no meaning for any Lustre administrators, only for Lustre developpers.
As Lustre has a long history of having un-understandable error messages, it could be nice to do something better here.
(lustre/osc/osc_request.c:can_merge_pages())

Comments

[Johann Lombardi (Inactive)]

There are some flags which should not be mixed in the same BRW. Could you please give us some samples of the flag mismatch you see in the logs?

[Aurelien Degremont (Inactive)]

We always have "0x520" and "0x420".

[Peter Jones]

Bobijam

Could you please comment on this one?

Thanks

Peter

[Zhenyu Xu]

I think we need lower this message level.

[Zhenyu Xu]

patch tracking at http://review.whamcloud.com/1328

[Peter Jones]

Question asked by Oleg on gerrit "Well, are we sure we only want to print this warning once. I mean then the visibility of it is greatly reduced if it happpens, and it should not happen even once, so if it does happen we want to know and people to report it."

[Oleg Drokin]

Bobi, can you please go and see if these particular flags could in fact be mixed and if so, just update the safe mask in can_merge_pages?

[Zhenyu Xu]

0x100 is OBD_BRW_NOQUOTA, and from filter_commitrw_write() comment

                /* if one page is a write-back page from client cache and
                 * not from direct_io, or it's written by root, then mark
                 * the whole io request as ignore quota request, remote
                 * client can not break through quota. */
                if (exp_connect_rmtclient(exp))
                        flags &= ~OBD_BRW_NOQUOTA;
                if ((flags & OBD_BRW_NOQUOTA) ||
                    (flags & (OBD_BRW_FROM_GRANT | OBD_BRW_SYNC)) ==
                     OBD_BRW_FROM_GRANT)
                        iobuf->dr_ignore_quota = 1;

so adjacent pages disaccording with OBD_BRW_NOQUOTA flag can not be merged.

[Oleg Drokin]

Then this needs to be fixed properly so that such pages are not attempted to be merged

[Christopher Morrone]

As I commented in the gerrit ticket, even if we want a message of some kind, the current message is not acceptable. Asking a question "is this ok?" on the console is NOT ok. We need to make a clear statement that an admin will be able to handle. Perhaps something like:

Saw flags X and Y in the same brw: please report this to the Lustre developers.

[Christopher Morrone]

How bad is the quota accounting going to get with the http://review.whamcloud.com/1328 patch?

Is it roughly comparable? Is it going to be noticeably worse?

[Niu Yawei (Inactive)]

How bad is the quota accounting going to get with the http://review.whamcloud.com/1328 patch?

Is it roughly comparable? Is it going to be noticeably worse?

The OBD_BRW_NOQUOTA is applied to the pages written by superuser who can ignore quota limits, but this patch only changes error messages, so it doesn't affect the quota accounting.

[Oleg Drokin]

Right, the patch does not change the accounting.

I wonder if current code leads to incorrect quota accounting anyway?

For the case of both root and some user writing to the same file, depending on who got the first offset, the entire BRW will be quota-enforced (based on the first page flags) and potentially root's write might be lost OR not quota-enforced and then user's write that should fail will not fail.

Is this really ok?

[Niu Yawei (Inactive)]

I wonder if current code leads to incorrect quota accounting anyway?

For the case of both root and some user writing to the same file, depending on who got the first offset, the entire BRW will be quota-enforced (based on the first page flags) and potentially root's write might be lost OR not quota-enforced and then user's write that should fail will not fail.

Is this really ok?

Our currenty implementaion is that the whole BRW request will ignore quota if it has one ignore quota page, so if there is a page is from root or from cached write (cached data has to be written back not matter if it'll exceed quota), the whole BRW request will bypass quota enforcement, so root's write will not be lost but normal user's write could exceed quota limit in such case.

Whenever server realize some uid/gid has exceeded quota limit, it'll notify the client to enter sync write mode for normal users, that'll decrease the probability of the case you mentioned.

[Build Master (Inactive)]

Integrated in lustre-master » x86_64,server,el5,ofa #375
LU-651 osc: suppress message in can_merge_pages() (Revision 6f8e1b7c5a10862dc12265cb500803d1c19b5019)

Result = SUCCESS
Oleg Drokin : 6f8e1b7c5a10862dc12265cb500803d1c19b5019
Files :

• lustre/osc/osc_request.c

[Peter Jones]

Landed for 2.2. Please reopen if any more work is still required for this issue

[Build Master (Inactive)]

Integrated in lustre-master » x86_64,client,el5,inkernel #375
LU-651 osc: suppress message in can_merge_pages() (Revision 6f8e1b7c5a10862dc12265cb500803d1c19b5019)

Result = SUCCESS
Oleg Drokin : 6f8e1b7c5a10862dc12265cb500803d1c19b5019
Files :

• lustre/osc/osc_request.c

[Build Master (Inactive)]

Integrated in lustre-master » i686,client,el6,inkernel #375
LU-651 osc: suppress message in can_merge_pages() (Revision 6f8e1b7c5a10862dc12265cb500803d1c19b5019)

Result = SUCCESS
Oleg Drokin : 6f8e1b7c5a10862dc12265cb500803d1c19b5019
Files :

• lustre/osc/osc_request.c

[Build Master (Inactive)]

Integrated in lustre-master » x86_64,client,el6,inkernel #375
LU-651 osc: suppress message in can_merge_pages() (Revision 6f8e1b7c5a10862dc12265cb500803d1c19b5019)

Result = SUCCESS
Oleg Drokin : 6f8e1b7c5a10862dc12265cb500803d1c19b5019
Files :

• lustre/osc/osc_request.c

[Build Master (Inactive)]

Integrated in lustre-master » x86_64,server,el5,inkernel #375
LU-651 osc: suppress message in can_merge_pages() (Revision 6f8e1b7c5a10862dc12265cb500803d1c19b5019)

Result = SUCCESS
Oleg Drokin : 6f8e1b7c5a10862dc12265cb500803d1c19b5019
Files :

• lustre/osc/osc_request.c

[Build Master (Inactive)]

Integrated in lustre-master » x86_64,client,el5,ofa #375
LU-651 osc: suppress message in can_merge_pages() (Revision 6f8e1b7c5a10862dc12265cb500803d1c19b5019)

Result = SUCCESS
Oleg Drokin : 6f8e1b7c5a10862dc12265cb500803d1c19b5019
Files :

• lustre/osc/osc_request.c

[Build Master (Inactive)]

Integrated in lustre-master » x86_64,client,sles11,inkernel #375
LU-651 osc: suppress message in can_merge_pages() (Revision 6f8e1b7c5a10862dc12265cb500803d1c19b5019)

Result = SUCCESS
Oleg Drokin : 6f8e1b7c5a10862dc12265cb500803d1c19b5019
Files :

• lustre/osc/osc_request.c

[Build Master (Inactive)]

Integrated in lustre-master » i686,server,el6,inkernel #375
LU-651 osc: suppress message in can_merge_pages() (Revision 6f8e1b7c5a10862dc12265cb500803d1c19b5019)

Result = SUCCESS
Oleg Drokin : 6f8e1b7c5a10862dc12265cb500803d1c19b5019
Files :

• lustre/osc/osc_request.c

[Build Master (Inactive)]

Integrated in lustre-master » x86_64,client,ubuntu1004,inkernel #375
LU-651 osc: suppress message in can_merge_pages() (Revision 6f8e1b7c5a10862dc12265cb500803d1c19b5019)

Result = SUCCESS
Oleg Drokin : 6f8e1b7c5a10862dc12265cb500803d1c19b5019
Files :

• lustre/osc/osc_request.c

[Build Master (Inactive)]

Integrated in lustre-master » x86_64,server,el6,inkernel #375
LU-651 osc: suppress message in can_merge_pages() (Revision 6f8e1b7c5a10862dc12265cb500803d1c19b5019)

Result = SUCCESS
Oleg Drokin : 6f8e1b7c5a10862dc12265cb500803d1c19b5019
Files :

• lustre/osc/osc_request.c

[Build Master (Inactive)]

Integrated in lustre-master » i686,client,el5,ofa #375
LU-651 osc: suppress message in can_merge_pages() (Revision 6f8e1b7c5a10862dc12265cb500803d1c19b5019)

Result = SUCCESS
Oleg Drokin : 6f8e1b7c5a10862dc12265cb500803d1c19b5019
Files :

• lustre/osc/osc_request.c

[Build Master (Inactive)]

Integrated in lustre-master » i686,client,el5,inkernel #375
LU-651 osc: suppress message in can_merge_pages() (Revision 6f8e1b7c5a10862dc12265cb500803d1c19b5019)

Result = SUCCESS
Oleg Drokin : 6f8e1b7c5a10862dc12265cb500803d1c19b5019
Files :

• lustre/osc/osc_request.c

[Build Master (Inactive)]

Integrated in lustre-master » i686,server,el5,inkernel #375
LU-651 osc: suppress message in can_merge_pages() (Revision 6f8e1b7c5a10862dc12265cb500803d1c19b5019)

Result = SUCCESS
Oleg Drokin : 6f8e1b7c5a10862dc12265cb500803d1c19b5019
Files :

• lustre/osc/osc_request.c

[Build Master (Inactive)]

Integrated in lustre-master » i686,server,el5,ofa #375
LU-651 osc: suppress message in can_merge_pages() (Revision 6f8e1b7c5a10862dc12265cb500803d1c19b5019)

Result = SUCCESS
Oleg Drokin : 6f8e1b7c5a10862dc12265cb500803d1c19b5019
Files :

• lustre/osc/osc_request.c

[Build Master (Inactive)]

Integrated in lustre-b2_1 » x86_64,client,sles11,inkernel #41
LU-651 osc: suppress message in can_merge_pages() (Revision 2ba2f5371ca95732ac44c521e7e699beaed645ac)

Result = SUCCESS
Oleg Drokin : 2ba2f5371ca95732ac44c521e7e699beaed645ac
Files :

• lustre/osc/osc_request.c

[Build Master (Inactive)]

Integrated in lustre-b2_1 » i686,client,el6,inkernel #41
LU-651 osc: suppress message in can_merge_pages() (Revision 2ba2f5371ca95732ac44c521e7e699beaed645ac)

Result = SUCCESS
Oleg Drokin : 2ba2f5371ca95732ac44c521e7e699beaed645ac
Files :

• lustre/osc/osc_request.c

[Build Master (Inactive)]

Integrated in lustre-b2_1 » x86_64,server,el6,inkernel #41
LU-651 osc: suppress message in can_merge_pages() (Revision 2ba2f5371ca95732ac44c521e7e699beaed645ac)

Result = SUCCESS
Oleg Drokin : 2ba2f5371ca95732ac44c521e7e699beaed645ac
Files :

• lustre/osc/osc_request.c

[Build Master (Inactive)]

Integrated in lustre-b2_1 » i686,client,el5,ofa #41
LU-651 osc: suppress message in can_merge_pages() (Revision 2ba2f5371ca95732ac44c521e7e699beaed645ac)

Result = SUCCESS
Oleg Drokin : 2ba2f5371ca95732ac44c521e7e699beaed645ac
Files :

• lustre/osc/osc_request.c

[Build Master (Inactive)]

Integrated in lustre-b2_1 » x86_64,server,el5,ofa #41
LU-651 osc: suppress message in can_merge_pages() (Revision 2ba2f5371ca95732ac44c521e7e699beaed645ac)

Result = SUCCESS
Oleg Drokin : 2ba2f5371ca95732ac44c521e7e699beaed645ac
Files :

• lustre/osc/osc_request.c

[Build Master (Inactive)]

Integrated in lustre-b2_1 » x86_64,client,el6,inkernel #41
LU-651 osc: suppress message in can_merge_pages() (Revision 2ba2f5371ca95732ac44c521e7e699beaed645ac)

Result = SUCCESS
Oleg Drokin : 2ba2f5371ca95732ac44c521e7e699beaed645ac
Files :

• lustre/osc/osc_request.c

[Build Master (Inactive)]

Integrated in lustre-b2_1 » i686,server,el6,inkernel #41
LU-651 osc: suppress message in can_merge_pages() (Revision 2ba2f5371ca95732ac44c521e7e699beaed645ac)

Result = SUCCESS
Oleg Drokin : 2ba2f5371ca95732ac44c521e7e699beaed645ac
Files :

• lustre/osc/osc_request.c

[Build Master (Inactive)]

Integrated in lustre-b2_1 » x86_64,client,el5,inkernel #41
LU-651 osc: suppress message in can_merge_pages() (Revision 2ba2f5371ca95732ac44c521e7e699beaed645ac)

Result = SUCCESS
Oleg Drokin : 2ba2f5371ca95732ac44c521e7e699beaed645ac
Files :

• lustre/osc/osc_request.c

[Build Master (Inactive)]

Integrated in lustre-b2_1 » i686,server,el5,inkernel #41
LU-651 osc: suppress message in can_merge_pages() (Revision 2ba2f5371ca95732ac44c521e7e699beaed645ac)

Result = SUCCESS
Oleg Drokin : 2ba2f5371ca95732ac44c521e7e699beaed645ac
Files :

• lustre/osc/osc_request.c

[Build Master (Inactive)]

Integrated in lustre-b2_1 » x86_64,server,el5,inkernel #41
LU-651 osc: suppress message in can_merge_pages() (Revision 2ba2f5371ca95732ac44c521e7e699beaed645ac)

Result = SUCCESS
Oleg Drokin : 2ba2f5371ca95732ac44c521e7e699beaed645ac
Files :

• lustre/osc/osc_request.c

[Build Master (Inactive)]

Integrated in lustre-b2_1 » i686,server,el5,ofa #41
LU-651 osc: suppress message in can_merge_pages() (Revision 2ba2f5371ca95732ac44c521e7e699beaed645ac)

Result = SUCCESS
Oleg Drokin : 2ba2f5371ca95732ac44c521e7e699beaed645ac
Files :

• lustre/osc/osc_request.c

[Build Master (Inactive)]

Integrated in lustre-b2_1 » x86_64,client,el5,ofa #41
LU-651 osc: suppress message in can_merge_pages() (Revision 2ba2f5371ca95732ac44c521e7e699beaed645ac)

Result = SUCCESS
Oleg Drokin : 2ba2f5371ca95732ac44c521e7e699beaed645ac
Files :

• lustre/osc/osc_request.c

[Build Master (Inactive)]

Integrated in lustre-b2_1 » i686,client,el5,inkernel #41
LU-651 osc: suppress message in can_merge_pages() (Revision 2ba2f5371ca95732ac44c521e7e699beaed645ac)

Result = SUCCESS
Oleg Drokin : 2ba2f5371ca95732ac44c521e7e699beaed645ac
Files :

• lustre/osc/osc_request.c