[LU-13217] What mkfsoptions are necessary for huge OSTs Created: 07/Feb/20 Updated: 09/Feb/20 Resolved: 07/Feb/20 |
|
| Status: | Resolved |
| Project: | Lustre |
| Component/s: | None |
| Affects Version/s: | Lustre 2.12.3 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Minor |
| Reporter: | Joe Mervini | Assignee: | WC Triage |
| Resolution: | Duplicate | Votes: | 0 |
| Labels: | None | ||
| Environment: |
Dell 740 servers IB connected to DDN SFA18K storage systems |
||
| Issue Links: |
|
||||
| Severity: | 3 | ||||
| Rank (Obsolete): | 9223372036854775807 | ||||
| Description |
|
We have some new SFA18K systems that are configured as de-clustered raid pools and the VDs that are being presented are 655TB on one system and 695TB on the other 2. What I'd like to know is the specific mkfs.lustre command options I should be using for these large OST sizes. |
| Comments |
| Comment by Peter Jones [ 07/Feb/20 ] |
|
Joe I have seen this question come in from you via DDN support channels so we'll answer that way Peter |
| Comment by Joe Mervini [ 07/Feb/20 ] |
|
Peter, The DDN ticket is regarding performance and I think that I've proven it's some kind of hardware issue. In this ticket I'm asking specific information on mkfs.lustre options for OST greater than 512TB. To add some detail we are running the 1.45.2 version of e2fsprogs. |
| Comment by Joe Mervini [ 09/Feb/20 ] |
|
To follow on: The hardware issue has been resolved. However, I am unable to mount the newly formatted OSTs because the OST size is greater than 512TB. I don't like the log message that says that I could encounter data corruption.
Feb 9 12:09:11 qoss4 kernel: [ 6800.934088] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro Feb 9 12:09:51 qoss4 kernel: [ 6840.520185] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 Feb 9 12:09:54 qoss4 kernel: [ 6843.538123] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc Feb 9 12:09:54 qoss4 kernel: [ 6843.538134] LustreError: 17863:0:(osd_handler.c:7656:osd_mount()) qscratch-OST0003-osd: device /dev/mapper/360001ff0c02e50000000000489ad0003 LDISKFS does not support filesystems greater than 512TB and can cause data corruption. Use "force_over_512tb" mount option to override.
Feb 9 12:09:54 qoss4 kernel: [ 6843.565186] LustreError: 17863:0:(obd_config.c:559:class_setup()) setup qscratch-OST0003-osd failed (-22) Feb 9 12:09:54 qoss4 kernel: [ 6843.575571] LustreError: 17863:0:(obd_mount.c:202:lustre_start_simple()) qscratch-OST0003-osd setup error -22 Feb 9 12:09:54 qoss4 kernel: [ 6843.586269] LustreError: 17863:0:(obd_mount_server.c:1947:server_fill_super()) Unable to start osd on /dev/mapper/360001ff0c02e50000000000489ad0003: -22 Feb 9 12:09:54 qoss4 kernel: [ 6843.601447] LustreError: 17863:0:(obd_mount.c:1608:lustre_fill_super()) Unable to mount (-22) Feb 9 12:09:54 qoss4 ldev[17856]: qscratch-OST0003: mount.lustre: mount /dev/mapper/360001ff0c02e50000000000489ad0003 at /mnt/lustre/local/qscratch-OST0003 failed: Invalid argument Feb 9 12:09:54 qoss4 ldev[17856]: qscratch-OST0003: This may have multiple causes. Feb 9 12:09:54 qoss4 ldev[17856]: qscratch-OST0003: Are the mount options correct? Feb 9 12:09:54 qoss4 ldev[17856]: qscratch-OST0003: Check the syslog for more info. Feb 9 12:09:54 qoss4 systemd[1]: lustre.service: main process exited, code=exited, status=1/FAILURE |