[LU-419] Running 1.8.6WC the mkfs.lustre failed to format a 22.94TB LUN, Size of device too big to be expressed in 32 bits using a block size of 4096

• Type: Bug
• Resolution: Fixed
• Priority: Major
• Fix Version/s: Lustre 2.1.0, Lustre 1.8.7
• Affects Version/s: Lustre 1.8.6
• Environment: Centos/Rhel 5.5 Linux servers running 1.8.6WC
• Story Points: 1
• Severity: 2
• Assignee: Andreas Dilger
• Reporter: John Salinas (Inactive)

Description

1. rpm -qa |grep -i lustre
   kernel-headers-2.6.18-238.12.1.el5_lustre
   lustre-1.8.6-wc1_2.6.18_238.12.1.el5_lustre
   ddn-sfa-ofed-1.5.3.1-2.6.18_238.12.1.el5_lustre
   lustre-modules-1.8.6-wc1_2.6.18_238.12.1.el5_lustre
   kernel-2.6.18-238.12.1.el5_lustre
   kernel-devel-2.6.18-238.12.1.el5_lustre
   kernel-ib-1.5.3.1-2.6.18_238.12.1.el5_lustre
   kernel-ib-devel-1.5.3.1-2.6.18_238.12.1.el5_lustre
   ddn-lustre-tools-0.4-2011.06.08.123659
   lustre-ldiskfs-3.1.50-wc1_2.6.18_238.12.1.el5_lustre

1. uname -a
   Linux oss0 2.6.18-238.12.1.el5_lustre #1 SMP Fri Jun 10 19:09:49 MDT 2011 x86_64 x86_64 x86_64 GNU/Linux

1. fdisk -l /dev/ddn/ost_testfs_54

Disk /dev/ddn/ost_testfs_54: 23493.4 GB, 23493471109120 bytes
255 heads, 63 sectors/track, 2856251 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/ddn/ost_testfs_54 doesn't contain a valid partition table

1. mkfs.lustre --mgsnode=127.0.0.2@tcp --failnode=127.0.0.2@tcp --fs
   name=testfs --ost --mkfsoptions=-m1 --index=9 /dev/ddn/ost_testfs_54

Permanent disk data:
Target: testfs-OST0009
Index: 9
Lustre FS: testfs
Mount type: ldiskfs
Flags: 0x62
(OST first_time update )
Persistent mount opts: errors=remount-ro,extents,mballoc
Parameters: mgsnode=127.0.0.2@tcp failover.node=127.0.0.2@tcp

checking for existing Lustre data: not found
device size = 22405120MB
2 6 18
formatting backing filesystem ldiskfs on /dev/ddn/ost_testfs_54
target name testfs-OST0009
4k blocks 5735710720
options -m1 -J size=400 -i 16384 -I 256 -q -O dir_index,extents,uninit_groups,mmp -F
mkfs_cmd = mke2fs -j -b 4096 -L testfs-OST0009 -m1 -J size=400 -i 16384 -I 256 -q -O dir_index,extents,uninit_groups,mmp -F /dev/ddn/ost_testfs_54 5735710720

Warning: the fs_type huge is not defined in mke2fs.conf

mke2fs: Size of device (0x155e00000 blocks) /dev/ddn/ost_testfs_54 too big to be expressed
in 32 bits using a blocksize of 4096.

mkfs.lustre FATAL: Unable to build fs /dev/ddn/ost_testfs_54 (256)

mkfs.lustre FATAL: mkfs failed 256

This also failed yesterday when we built out of b1_8:

1. rpm -qa |grep -i lustre
   lustre-debuginfo-1.8.5.56-2.6.18_238.9.1.el5_lustre.20110528080747
   kernel-ib-devel-1.5.3.1-2.6.18_238.9.1.el5_lustre.20110525141730
   kernel-headers-2.6.18-238.9.1.el5_lustre.20110525141730
   lustre-ldiskfs-debuginfo-3.1.50-2.6.18_238.9.1.el5_lustre.20110528080747
   lustre-modules-1.8.5.56-2.6.18_238.9.1.el5_lustre.20110528080747
   lustre-1.8.5.56-2.6.18_238.9.1.el5_lustre.20110528080747
   kernel-devel-2.6.18-238.9.1.el5_lustre.20110525141730
   kernel-2.6.18-238.9.1.el5_lustre.20110525141730
   kernel-debuginfo-common-2.6.18-238.9.1.el5_lustre.20110525141730
   kernel-debuginfo-2.6.18-238.9.1.el5_lustre.20110525141730
   ddn-lustre-tools-0.4-2011.04.01.153313
   lustre-ldiskfs-3.1.50-2.6.18_238.9.1.el5_lustre.20110528080747
   ddn-sfa-ofed-1.5.3.1-2.6.18_238.9.1.el5_lustre.20110525141730
   kernel-ib-1.5.3.1-2.6.18_238.9.1.el5_lustre.20110525141730

1. mkfs.lustre --mgsnode=127.0.0.2@tcp --failnode=127.0.0.2@tcp --fsname=testfs --ost --mkfsoptions=-m1 -i 131072 --index=54 /dev/ddn/ost_testfs_54

Permanent disk data:
Target: testfs-OST0036
Index: 54
Lustre FS: testfs
Mount type: ldiskfs
Flags: 0x62
(OST first_time update )
Persistent mount opts: errors=remount-ro,extents,mballoc
Parameters: mgsnode=127.0.0.2@tcp failover.node=127.0.0.2@tcp

checking for existing Lustre data: not found
device size = 22405120MB
2 6 18
formatting backing filesystem ldiskfs on /dev/ddn/ost_testfs_54
target name testfs-OST0036
4k blocks 5735710720
options -m1 -J size=400 -i 16384 -I 256 -q -O dir_index,extents,uninit_groups,mmp -F
mkfs_cmd = mke2fs -j -b 4096 -L testfs-OST0036 -m1 -J size=400 -i 16384 -I 256 -q -O dir_index,extents,uninit_groups,mmp -F /dev/ddn/ost_testfs_54 5735710720

Warning: the fs_type huge is not defined in mke2fs.conf

mke2fs: Size of device (0x155e00000 blocks) /dev/ddn/ost_testfs_54 too big to be expressed
in 32 bits using a blocksize of 4096.

mkfs.lustre FATAL: Unable to build fs /dev/ddn/ost_testfs_54 (256)

mkfs.lustre FATAL: mkfs failed 256

Our simple attempt to increase the block size to 8192 not work either:

Warning: blocksize 8192 not usable on most systems.
Warning: 8192-byte blocks too big for system (max 4096), forced to continue

Warning: the fs_type huge is not defined in mke2fs.conf

mke2fs: Size of device (0x155e00000 blocks) /dev/ddn/ost_testfs_54 too big to be expressed
in 32 bits using a blocksize of 8192.

Issue Links

• Trackbacks: Lustre 1.8.x known issues tracker While testing against Lustre b18 branch, we would hit known bugs which were already reported in Lustre Bugzilla https://bugzilla.lustre.org/. In order to move away from relying on Bugzilla, we would create a JIRA
• Trackbacks: Changelog 1.8 {}version 1.8.7wc1{} {}Support for networks: socklnd \any kernel supported by Lustre, qswlnd Qsnet kernel modules 5.20 and later, openiblnd IbGold 1.8.2, o2iblnd OFED 1.3, 1.4.1, 1.4.2, 1.5.1, 1.5.2, 1.5.3.1 and 1.5.3.2 gmlnd GM 2.1....

Comments

[Andreas Dilger]

Patch to enable 64bit feature for filesystems over 16TB was landed for 1.8.7 and 2.1.0.

[Build Master (Inactive)]

Integrated in lustre-b1_8 » x86_64,server,el5,inkernel #103
LU-419 enable ext4 features by default
LU-419 increase default inode ratio for OST/MDT

Johann Lombardi : 78dde2206d49901e7315d07edf9eefdc8383d96c
Files :

• lustre/utils/mkfs_lustre.c
• lustre/lvfs/fsfilt_ext3.c

Johann Lombardi : 438c81791e3ce8582d12776d93a27e111d438cdb
Files :

• lustre/utils/mkfs_lustre.c

[Build Master (Inactive)]

Integrated in lustre-b1_8 » i686,server,el5,ofa #103
LU-419 enable ext4 features by default
LU-419 increase default inode ratio for OST/MDT

Johann Lombardi : 78dde2206d49901e7315d07edf9eefdc8383d96c
Files :

• lustre/lvfs/fsfilt_ext3.c
• lustre/utils/mkfs_lustre.c

Johann Lombardi : 438c81791e3ce8582d12776d93a27e111d438cdb
Files :

• lustre/utils/mkfs_lustre.c

[Build Master (Inactive)]

Integrated in lustre-b1_8 » i686,client,el5,inkernel #103
LU-419 enable ext4 features by default
LU-419 increase default inode ratio for OST/MDT

Johann Lombardi : 78dde2206d49901e7315d07edf9eefdc8383d96c
Files :

• lustre/utils/mkfs_lustre.c
• lustre/lvfs/fsfilt_ext3.c

Johann Lombardi : 438c81791e3ce8582d12776d93a27e111d438cdb
Files :

• lustre/utils/mkfs_lustre.c

[Build Master (Inactive)]

Integrated in lustre-b1_8 » x86_64,server,el5,ofa #103
LU-419 enable ext4 features by default
LU-419 increase default inode ratio for OST/MDT

Johann Lombardi : 78dde2206d49901e7315d07edf9eefdc8383d96c
Files :

• lustre/lvfs/fsfilt_ext3.c
• lustre/utils/mkfs_lustre.c

Johann Lombardi : 438c81791e3ce8582d12776d93a27e111d438cdb
Files :

• lustre/utils/mkfs_lustre.c

[Build Master (Inactive)]

Integrated in lustre-b1_8 » i686,server,el5,inkernel #103
LU-419 enable ext4 features by default
LU-419 increase default inode ratio for OST/MDT

Johann Lombardi : 78dde2206d49901e7315d07edf9eefdc8383d96c
Files :

• lustre/lvfs/fsfilt_ext3.c
• lustre/utils/mkfs_lustre.c

Johann Lombardi : 438c81791e3ce8582d12776d93a27e111d438cdb
Files :

• lustre/utils/mkfs_lustre.c

[Build Master (Inactive)]

Integrated in lustre-b1_8 » x86_64,client,ubuntu1004,inkernel #103
LU-419 enable ext4 features by default
LU-419 increase default inode ratio for OST/MDT

Johann Lombardi : 78dde2206d49901e7315d07edf9eefdc8383d96c
Files :

• lustre/utils/mkfs_lustre.c
• lustre/lvfs/fsfilt_ext3.c

Johann Lombardi : 438c81791e3ce8582d12776d93a27e111d438cdb
Files :

• lustre/utils/mkfs_lustre.c

[Build Master (Inactive)]

Integrated in lustre-b1_8 » i686,client,el5,ofa #103
LU-419 enable ext4 features by default
LU-419 increase default inode ratio for OST/MDT

Johann Lombardi : 78dde2206d49901e7315d07edf9eefdc8383d96c
Files :

• lustre/utils/mkfs_lustre.c
• lustre/lvfs/fsfilt_ext3.c

Johann Lombardi : 438c81791e3ce8582d12776d93a27e111d438cdb
Files :

• lustre/utils/mkfs_lustre.c

[Build Master (Inactive)]

Integrated in lustre-b1_8 » x86_64,client,el5,ofa #103
LU-419 enable ext4 features by default
LU-419 increase default inode ratio for OST/MDT

Johann Lombardi : 78dde2206d49901e7315d07edf9eefdc8383d96c
Files :

• lustre/lvfs/fsfilt_ext3.c
• lustre/utils/mkfs_lustre.c

Johann Lombardi : 438c81791e3ce8582d12776d93a27e111d438cdb
Files :

• lustre/utils/mkfs_lustre.c

[Build Master (Inactive)]

Integrated in lustre-b1_8 » i686,client,el6,inkernel #103
LU-419 enable ext4 features by default
LU-419 increase default inode ratio for OST/MDT

Johann Lombardi : 78dde2206d49901e7315d07edf9eefdc8383d96c
Files :

• lustre/utils/mkfs_lustre.c
• lustre/lvfs/fsfilt_ext3.c

Johann Lombardi : 438c81791e3ce8582d12776d93a27e111d438cdb
Files :

• lustre/utils/mkfs_lustre.c