[LU-8342] ZFS dnodesize and recordsize should be set at file system creation

• Type: Improvement
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.11.0
• Labels: llnl
• Assignee: Giuseppe Di Natale (Inactive)
• Reporter: Giuseppe Di Natale (Inactive)

Description

ZFS dnodesize and recordsize should be set to appropriate defaults upon dataset creation at filesystem creation time. We can set dnodesize=auto and recordsize=1M by default if the installed version of zfs supports it.

Issue Links

• is related to: LU-8042 mkfs.lustre should set ashift=12 recordsize=1024k compression=lz4" by default for new ZFS OSTs

Comments

[Peter Jones]

Landed for 2.11

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/21055/
Subject: LU-8342 utils: Set dnodesize/recordsize at zfs dataset create
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 1617b8f6b6cdd0f5b74d7bfb8166d74b63cfed81

[Giuseppe Di Natale (Inactive)]

Currently no updates to report. I will try to revisit this soon.

[Gu Zheng (Inactive)]

Hi Giuseppe,

Any update about the patch http://review.whamcloud.com/21055?

[Andreas Dilger]

Olaf, I'm not against that, but it would definitely be more work than the current patch, and likely push the change out into 2.10.

[Olaf Faaland]

Andreas,
I understand. I'm suggesting that if those good defaults are encoded in a config file instead of in code, they (a) are visible to the user and (b) require trival code review to change. Also, the existing options do not distinguish between pool properties and dataset properties.

[Andreas Dilger]

There are already -mkfsoptions and -mountfsoptions that can be used to pass extra options to mkfs.lustre and to the internal mount command for the back-end filesystem. They should be able to override the default options specified internally by mkfs.lustre. My goal in specifying these options internally is that the majority of users should get the best performance out of the box if possible, rather than having to specify extra options.

[Olaf Faaland]

Perhaps lustre/conf/lustre is not the right place for such settings; I see there is an /etc/mke2fs.conf with ini-style contents, and some userspace apps seem to use /etc/default/foo. Anyway, basic proposal that these settings be put in a config file, instead of in the code, still stands.

[Olaf Faaland]

I see that lustre/conf/lustre already has ZPOOL_IMPORT_DIR and ZPOOL_IMPORT_ARGS. So perhaps ZPOOL_CREATE_ARGS and ZFS_CREATE_ARGS?

[Olaf Faaland]

Even if these pool and dataset settings are correct for all cases for now, they may become incorrect with future changes in either lustre or zfs. Furthermore, there are likely unusual cases (e.g. testing, see Jinshan's concern about test data often being highly compressible) where one or more of these settings are undesirable.

Both http://review.whamcloud.com/#/c/19892/2 and http://review.whamcloud.com/#/c/21055/3 set the desired settings by hard-coding specific values into mkfs.lustre. How about putting the settings themselves into a configuration file, e.g. /etc/sysconfig/lustre or distro-specific equivalent which is parsed by mkfs.lustre? Then they are visible to the user, can easily be changed when appropriate, and the defaults can be changed with a trivial patch that is easy to review.

[Andreas Dilger]

Based on past postings on the ZFS mailing lists, users have reported terrible performance when ZFS doesn't auto-detect the 4KB sector size correctly (usually because drives are advertising 512-byte sectors for "maximum compatibility" even when they have 4KB sectors internally). Not only does this hurt performance, it can potentially lead to data reliability problems if sectors are being modified that do not belong to the current block.

Also, even if drives are correctly reporting 512-byte sectors, there is a long-term maintenance problem if those drives need to be replaced by newer drives, because all newer/larger drives have 4KB sectors and it isn't possible to replace any drives in a 512-byte sector VDEV with 4096-byte sector drives without a full backup/restore. That makes maintenance more problematic, as well as prevents VDEV "autoexpand" to work if existing drives are replaced with larger drives.

While I understand Brian's concern that changing the OpenZFS default to ashift=12 since it would increase space usage for some workloads (despite repeated requests to change it), this is less of a concern for Lustre OSTs. From a support and "best performance out of the box" point of view I'd prefer setting ashift=12 on OSTs by default.