[LU-979] Barriers should be disabled by default on RHEL6

• Status: Resolved
• Project: Lustre
• Affects Version/s: Lustre 2.1.0
• Type: Bug
• Priority: Minor
• Reporter: Diego Moreno (Inactive)
• Assignee: Yang Sheng
• Resolution: Duplicate
• Labels: ldiskfs, performance, rhel

Environment

RHEL6.1 with Lustre2.0/2.1

• Severity: 3

Description

Formatting a lustre filesystem in RHEL6 barriers are enabled by default with our Lustre 2.0 (same for Lustre 2.1). The impact on small IO performance is very important if we don't specify nobarrier as an MDT mount option. A single fileop bench running with '-f 15' will take more than 2m30sec while the one with nobarrier as mount option on MDT will take less than a minute. Compiling a kernel in Lustre is also a good way to see the problem due to jbd2 barriers.

This problem had already been addressed in SLES with ldiskfs patch ext4-disable-write-bar-by-default.patch. I adapted a similar patch to RHEL6.1. I'll try to submit it as soon as possible.

Comments

[Johann Lombardi (Inactive)]

I think the latest patchset of LU-751 (available here http://review.whamcloud.com/#change,1892) already takes care of this.

[Peter Jones]

YangSheng

Please consider this when working on the RHEL6.2 support

Thanks

Peter

[Lustre Bull]

Right Johann,

thanks!

[Peter Jones]

Ah good - thanks YangSheng\Johann!

[Diego Moreno (Inactive)]

Hi,

Coming back to this issue, I couldn't see in the patch where are you disabling barriers for rhel6.2.

[Diego Moreno (Inactive)]

Hi,

Coming back to this issue, could you please specify where are barriers disabled on lustre 2.1 code? I cannot see anywhere any code disabling barriers by default. This question also stands for lustre 2.2 but in this case the patch is there but it's not applied.

Thanks,

[Yang Sheng]

Hi, Diego, As Andreas point out in gerrit:

I think we should leave barriers enabled by default, and if performance is down for specific types of hardware then the administrator can choose to mount with "-o nobarrier" by themselves. I'm not sure this will happen, because the kernel barrier implementation has improved significantly since SLES10 when it was first needed, and it is enabled by default for both ext3 and ext4 in all modern kernels.

[Shuichi Ihara (Inactive)]

I think nobarrier option for OSTs really helps to improve direct IO perforamnce. I saw dramatically performance improvements with nobarrier option.
In the direct IO mode, journal transaction starts for each RPC to commit to disk and jbd issues barrier to storage for each journal commit. If there are battery backup raid cache and cache mirroring feature between storage controllers, nobarrier can be enabled and help a lot to reduce the latency for journal commeit. thgouht?