https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-13217 Lustre <p>We have some new SFA18K systems that are configured as de-clustered raid pools and the VDs that are being presented are 655TB on one system and 695TB on the other 2. </p> <p>What I'd like to know is the specific mkfs.lustre command options I should be using for these large OST sizes. </p> Dell 740 servers IB connected to DDN SFA18K storage systems LU-13217

What mkfsoptions are necessary for huge OSTs

Bug Minor Resolved Duplicate WC Triage Joe Mervini Fri, 7 Feb 2020 17:16:19 +0000 Sun, 9 Feb 2020 19:26:56 +0000 Fri, 7 Feb 2020 19:00:47 +0000 Lustre 2.12.3 0 1 <p>Joe</p> <p>I have seen this question come in from you via DDN support channels so we'll answer that way</p> <p>Peter</p> <p>Peter,</p> <p>The DDN ticket is regarding performance and I think that I've proven it's some kind of hardware issue. </p> <p>In this ticket I'm asking specific information on mkfs.lustre options for OST greater than 512TB. To add some detail we are running the 1.45.2 version of e2fsprogs.</p> <p>To follow on: The hardware issue has been resolved. However, I am unable to mount the newly formatted OSTs because the OST size is greater than 512TB. I don't like the log message that says that I could encounter data corruption.</p> <p> </p> <p>Feb  9 12:09:11 qoss4 kernel: [ 6800.934088] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro</p> <p>Feb  9 12:09:51 qoss4 kernel: [ 6840.520185] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5</p> <p>Feb  9 12:09:54 qoss4 kernel: [ 6843.538123] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc</p> <p>Feb  9 12:09:54 qoss4 kernel: [ 6843.538134] LustreError: 17863:0:(osd_handler.c:7656:osd_mount()) qscratch-OST0003-osd: device /dev/mapper/360001ff0c02e50000000000489ad0003 LDISKFS does not support filesystems greater than 512TB and can cause data corruption. Use "force_over_512tb" mount option to override.</p> <p> </p> <p>Feb  9 12:09:54 qoss4 kernel: [ 6843.565186] LustreError: 17863:0:(obd_config.c:559:class_setup()) setup qscratch-OST0003-osd failed (-22)</p> <p>Feb  9 12:09:54 qoss4 kernel: [ 6843.575571] LustreError: 17863:0:(obd_mount.c:202:lustre_start_simple()) qscratch-OST0003-osd setup error -22</p> <p>Feb  9 12:09:54 qoss4 kernel: [ 6843.586269] LustreError: 17863:0:(obd_mount_server.c:1947:server_fill_super()) Unable to start osd on /dev/mapper/360001ff0c02e50000000000489ad0003: -22</p> <p>Feb  9 12:09:54 qoss4 kernel: [ 6843.601447] LustreError: 17863:0:(obd_mount.c:1608:lustre_fill_super()) Unable to mount  (-22)</p> <p>Feb  9 12:09:54 qoss4 ldev<span class="error">&#91;17856&#93;</span>: qscratch-OST0003: mount.lustre: mount /dev/mapper/360001ff0c02e50000000000489ad0003 at /mnt/lustre/local/qscratch-OST0003 failed: Invalid argument</p> <p>Feb  9 12:09:54 qoss4 ldev<span class="error">&#91;17856&#93;</span>: qscratch-OST0003: This may have multiple causes.</p> <p>Feb  9 12:09:54 qoss4 ldev<span class="error">&#91;17856&#93;</span>: qscratch-OST0003: Are the mount options correct?</p> <p>Feb  9 12:09:54 qoss4 ldev<span class="error">&#91;17856&#93;</span>: qscratch-OST0003: Check the syslog for more info.</p> <p>Feb  9 12:09:54 qoss4 systemd<span class="error">&#91;1&#93;</span>: lustre.service: main process exited, code=exited, status=1/FAILURE</p> Related Development Rank 1|i00tdz: Rank (Obsolete) 9223372036854775807 Severity