[#LU-264] MMP update

[LU-264] MMP update Created: 03/May/11 Updated: 16/Jun/11 Resolved: 06/Jun/11
Status:	Resolved
Project:	Lustre
Component/s:	None
Affects Version/s:	Lustre 1.8.6
Fix Version/s:	None

Type:

Bug

Priority:

Minor

Reporter:

Johann Lombardi (Inactive)

Assignee:

Niu Yawei (Inactive)

Resolution:

Fixed

Votes:

Labels:

None

Issue Links:

Duplicate
is duplicated by	~~LU-268~~	Possible bug in userspace block handl...	Resolved

Severity:

Rank (Obsolete):

5001

Description

I made some changes to MMP to push it upstream:
http://thread.gmane.org/gmane.comp.file-systems.ext4/24675

e2fsprogs patch:

use blk64_t instead of blk_t
bump update interval to 5s

kernel patch:

add more comments to struct mmp_struct
remove unused EXT4_MMP_UPDATE_INTERVAL
change check interval multiplier from 5 to 2
use ext4_fsblk_t instead of unsigned long for the MMP block

We should integrate those modifications into our own kernel & e2fsprogs patches.

Here is the initial thread for reference:
http://thread.gmane.org/gmane.comp.file-systems.ext4/24301/focus=24306

Comments

Comment by Peter Jones [ 03/May/11 ]

Niu

Could you please look into this one?

Thanks

Peter

Comment by Johann Lombardi (Inactive) [ 03/May/11 ]

BTW, once landed, it would be great if YuJian can check that the >2TB bug is fixed.

Comment by Andreas Dilger [ 09/May/11 ]

Johann,
I spoke with Ted about landing the MMP patch. He said he would be interested to land this for the next kernel merge window (in the next 2 weeks or so). However, he would like the mmp code moved to a separate mmp.c file instead of being added to super.c because that file is getting too large. Also, the patch should include the required fix to the MD RAID-5 code (raid5-mmp-unplug-dev-rhel6.patch).

Comment by Niu Yawei (Inactive) [ 16/May/11 ]

Hi, Johann

Should I update the MMP patch now or after you moving the code to a separate mmp.c file? Thanks.

Comment by Johann Lombardi (Inactive) [ 17/May/11 ]

Hi Niu, i already submitted the updated patch to the linux-ext4 mailing list last week.
See http://thread.gmane.org/gmane.comp.file-systems.ext4/25256.

It would be great if you could integrate those changes in our MMP patches.
Thanks in advance.

Comment by Niu Yawei (Inactive) [ 18/May/11 ]

Hi, Johann

I'm not sure why the e2fsprogs patch from http://thread.gmane.org/gmane.comp.file-systems.ext4/24675 displayed in wrong format in my brower (@ was replace with <at>), however, the kernel patch is fine. For safety, could you send me the e2fsprogs patch directly?

btw, I saw there are several branches in our e2fsprogs git repository, should I just apply the patch to master? Thanks.

Comment by Johann Lombardi (Inactive) [ 18/May/11 ]

The kernel patch:
http://patchwork.ozlabs.org/patch/95539/raw

The e2fsprogs patch:
http://patchwork.ozlabs.org/patch/93605/raw/

As for the e2fsprogs branch, i think you only need to care about master, but i will let Andreas confirm.

Comment by Niu Yawei (Inactive) [ 18/May/11 ]

Looks there is a minor defect in the e2fsprogs patch, the PR_0_MMP_INVALID_BLK and PR_0_MMP_INVALID_MAGIC are inserted after PR_1B_ADJ_EA_REFCOUNT (see problem_table[] of e2fsck/problem.c), which caused verify_problem_table() failure. (http://newbuild.whamcloud.com/job/e2fsprogs-reviews/22/arch=x86_64,distro=el5/console)

Hi, Johann
Should I just move the PR_0_MMP_INVALID_BLK/PR_0_MMP_INVALID_MAGIC to the end of pass 0 errors(just behind PR_0_GDT_CSUM_LATCH)? Thanks.

Comment by Andreas Dilger [ 19/May/11 ]

Yes, please move the PR_0_* error definitions to the appropriate location in problem.c. This is by numerical order, so they should go at the end of the other PR_0_* errors if that is how the error numbers were #defined.

I've landed a fix for the f_messy_inode test failure that was waiting for inspection, so you should be able to pass this test now. Please see other inspection comments on the patch.

Comment by Niu Yawei (Inactive) [ 19/May/11 ]

Thank you, Andreas. Will update the patch according to your comment soon.

Comment by Build Master (Inactive) [ 20/May/11 ]

Integrated in lustre-b1_8 » i686,client,el5,inkernel #59
~~LU-264~~ ext4 MMP update

Johann Lombardi : 386b23c44c9d69361183eea2ff0ad3b741980522
Files :

ldiskfs/kernel_patches/patches/ext4-mmp-rhel5.patch
ldiskfs/kernel_patches/patches/ext4-dynlocks-2.6-rhel5.patch
ldiskfs/kernel_patches/patches/ext4-dynlocks-2.6.patch
ldiskfs/kernel_patches/patches/ext4-misc-rhel5.patch
ldiskfs/ldiskfs/Makefile.in
ldiskfs/kernel_patches/patches/ext4-mmp-sles11.patch

Comment by Build Master (Inactive) [ 20/May/11 ]

Integrated in lustre-b1_8 » x86_64,client,el5,ofa #59
~~LU-264~~ ext4 MMP update

Johann Lombardi : 386b23c44c9d69361183eea2ff0ad3b741980522
Files :

ldiskfs/kernel_patches/patches/ext4-dynlocks-2.6.patch
ldiskfs/kernel_patches/patches/ext4-dynlocks-2.6-rhel5.patch
ldiskfs/kernel_patches/patches/ext4-misc-rhel5.patch
ldiskfs/ldiskfs/Makefile.in
ldiskfs/kernel_patches/patches/ext4-mmp-rhel5.patch
ldiskfs/kernel_patches/patches/ext4-mmp-sles11.patch

Comment by Build Master (Inactive) [ 20/May/11 ]

Integrated in lustre-b1_8 » x86_64,server,el5,ofa #59
~~LU-264~~ ext4 MMP update

Johann Lombardi : 386b23c44c9d69361183eea2ff0ad3b741980522
Files :

ldiskfs/kernel_patches/patches/ext4-dynlocks-2.6.patch
ldiskfs/ldiskfs/Makefile.in
ldiskfs/kernel_patches/patches/ext4-mmp-rhel5.patch
ldiskfs/kernel_patches/patches/ext4-dynlocks-2.6-rhel5.patch
ldiskfs/kernel_patches/patches/ext4-mmp-sles11.patch
ldiskfs/kernel_patches/patches/ext4-misc-rhel5.patch

Comment by Build Master (Inactive) [ 20/May/11 ]

Integrated in lustre-b1_8 » x86_64,client,ubuntu1004,inkernel #59
~~LU-264~~ ext4 MMP update

Johann Lombardi : 386b23c44c9d69361183eea2ff0ad3b741980522
Files :

ldiskfs/kernel_patches/patches/ext4-mmp-rhel5.patch
ldiskfs/kernel_patches/patches/ext4-misc-rhel5.patch
ldiskfs/kernel_patches/patches/ext4-dynlocks-2.6-rhel5.patch
ldiskfs/ldiskfs/Makefile.in
ldiskfs/kernel_patches/patches/ext4-dynlocks-2.6.patch
ldiskfs/kernel_patches/patches/ext4-mmp-sles11.patch

Comment by Build Master (Inactive) [ 20/May/11 ]

Integrated in lustre-b1_8 » i686,client,el5,ofa #59
~~LU-264~~ ext4 MMP update

Johann Lombardi : 386b23c44c9d69361183eea2ff0ad3b741980522
Files :

ldiskfs/kernel_patches/patches/ext4-dynlocks-2.6-rhel5.patch
ldiskfs/kernel_patches/patches/ext4-mmp-rhel5.patch
ldiskfs/kernel_patches/patches/ext4-dynlocks-2.6.patch
ldiskfs/ldiskfs/Makefile.in
ldiskfs/kernel_patches/patches/ext4-mmp-sles11.patch
ldiskfs/kernel_patches/patches/ext4-misc-rhel5.patch

Comment by Build Master (Inactive) [ 20/May/11 ]

Integrated in lustre-b1_8 » x86_64,client,el5,inkernel #59
~~LU-264~~ ext4 MMP update

Johann Lombardi : 386b23c44c9d69361183eea2ff0ad3b741980522
Files :

ldiskfs/kernel_patches/patches/ext4-mmp-sles11.patch
ldiskfs/kernel_patches/patches/ext4-misc-rhel5.patch
ldiskfs/kernel_patches/patches/ext4-dynlocks-2.6-rhel5.patch
ldiskfs/kernel_patches/patches/ext4-mmp-rhel5.patch
ldiskfs/ldiskfs/Makefile.in
ldiskfs/kernel_patches/patches/ext4-dynlocks-2.6.patch

Comment by Build Master (Inactive) [ 20/May/11 ]

Integrated in lustre-b1_8 » x86_64,client,el6,inkernel #59
~~LU-264~~ ext4 MMP update

Johann Lombardi : 386b23c44c9d69361183eea2ff0ad3b741980522
Files :

ldiskfs/kernel_patches/patches/ext4-mmp-rhel5.patch
ldiskfs/ldiskfs/Makefile.in
ldiskfs/kernel_patches/patches/ext4-misc-rhel5.patch
ldiskfs/kernel_patches/patches/ext4-dynlocks-2.6.patch
ldiskfs/kernel_patches/patches/ext4-mmp-sles11.patch
ldiskfs/kernel_patches/patches/ext4-dynlocks-2.6-rhel5.patch

Comment by Build Master (Inactive) [ 20/May/11 ]

Integrated in lustre-b1_8 » i686,server,el5,inkernel #59
~~LU-264~~ ext4 MMP update

Johann Lombardi : 386b23c44c9d69361183eea2ff0ad3b741980522
Files :

ldiskfs/kernel_patches/patches/ext4-mmp-rhel5.patch
ldiskfs/kernel_patches/patches/ext4-misc-rhel5.patch
ldiskfs/kernel_patches/patches/ext4-dynlocks-2.6.patch
ldiskfs/kernel_patches/patches/ext4-dynlocks-2.6-rhel5.patch
ldiskfs/ldiskfs/Makefile.in
ldiskfs/kernel_patches/patches/ext4-mmp-sles11.patch

Comment by Build Master (Inactive) [ 20/May/11 ]

Integrated in lustre-b1_8 » i686,server,el5,ofa #59
~~LU-264~~ ext4 MMP update

Johann Lombardi : 386b23c44c9d69361183eea2ff0ad3b741980522
Files :

ldiskfs/kernel_patches/patches/ext4-mmp-rhel5.patch
ldiskfs/kernel_patches/patches/ext4-dynlocks-2.6-rhel5.patch
ldiskfs/kernel_patches/patches/ext4-mmp-sles11.patch
ldiskfs/kernel_patches/patches/ext4-misc-rhel5.patch
ldiskfs/kernel_patches/patches/ext4-dynlocks-2.6.patch
ldiskfs/ldiskfs/Makefile.in

Comment by Build Master (Inactive) [ 20/May/11 ]

Integrated in lustre-b1_8 » x86_64,server,el5,inkernel #59
~~LU-264~~ ext4 MMP update

Johann Lombardi : 386b23c44c9d69361183eea2ff0ad3b741980522
Files :

ldiskfs/kernel_patches/patches/ext4-misc-rhel5.patch
ldiskfs/kernel_patches/patches/ext4-dynlocks-2.6.patch
ldiskfs/kernel_patches/patches/ext4-dynlocks-2.6-rhel5.patch
ldiskfs/kernel_patches/patches/ext4-mmp-rhel5.patch
ldiskfs/ldiskfs/Makefile.in
ldiskfs/kernel_patches/patches/ext4-mmp-sles11.patch

Comment by Build Master (Inactive) [ 20/May/11 ]

Integrated in lustre-b1_8 » i686,client,el6,inkernel #59
~~LU-264~~ ext4 MMP update

Johann Lombardi : 386b23c44c9d69361183eea2ff0ad3b741980522
Files :

ldiskfs/kernel_patches/patches/ext4-dynlocks-2.6.patch
ldiskfs/kernel_patches/patches/ext4-mmp-rhel5.patch
ldiskfs/kernel_patches/patches/ext4-misc-rhel5.patch
ldiskfs/ldiskfs/Makefile.in
ldiskfs/kernel_patches/patches/ext4-dynlocks-2.6-rhel5.patch
ldiskfs/kernel_patches/patches/ext4-mmp-sles11.patch

Comment by Peter Jones [ 20/May/11 ]

will this patch need porting to master?

Comment by Niu Yawei (Inactive) [ 22/May/11 ]

I think it should be ported to master when the master server support kernerl upgraded to 2.6.18-238 (current is 2.6.18-194, someone else is working on the upgrading I think).

Comment by Peter Jones [ 23/May/11 ]

Niu

Thanks for clarifying. I think that this kernel update just landed as http://git.whamcloud.com/?p=fs/lustre-release.git;a=commit;h=1d95408cfbacf7462314325996d3cdf087764266 tracked under LU228

Regards

Peter

Comment by Build Master (Inactive) [ 25/May/11 ]

Integrated in e2fsprogs-master » x86_64,el6 #27
~~LU-264~~ MMP update

Niu Yawei : 06506568215ef1b7e7c612e585969fd00aac6b9d
Files :

patches/e2fsprogs-large-ea.patch
patches/e2fsprogs-mmp.patch

Comment by Build Master (Inactive) [ 25/May/11 ]

Integrated in e2fsprogs-master » i686,el6 #27
~~LU-264~~ MMP update

Niu Yawei : 06506568215ef1b7e7c612e585969fd00aac6b9d
Files :

patches/e2fsprogs-large-ea.patch
patches/e2fsprogs-mmp.patch

Comment by Build Master (Inactive) [ 25/May/11 ]

Integrated in e2fsprogs-master » i686,el5 #27
~~LU-264~~ MMP update

Niu Yawei : 06506568215ef1b7e7c612e585969fd00aac6b9d
Files :

patches/e2fsprogs-large-ea.patch
patches/e2fsprogs-mmp.patch

Comment by Build Master (Inactive) [ 25/May/11 ]

Integrated in e2fsprogs-master » x86_64,el5 #27
~~LU-264~~ MMP update

Niu Yawei : 06506568215ef1b7e7c612e585969fd00aac6b9d
Files :

patches/e2fsprogs-mmp.patch
patches/e2fsprogs-large-ea.patch