[LU-8342] ZFS dnodesize and recordsize should be set at file system creation

• Type: Improvement
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.11.0
• Labels: llnl
• Assignee: Giuseppe Di Natale (Inactive)
• Reporter: Giuseppe Di Natale (Inactive)

Description

ZFS dnodesize and recordsize should be set to appropriate defaults upon dataset creation at filesystem creation time. We can set dnodesize=auto and recordsize=1M by default if the installed version of zfs supports it.

Issue Links

• is related to: LU-8042 mkfs.lustre should set ashift=12 recordsize=1024k compression=lz4" by default for new ZFS OSTs

Comments

[Peter Jones]

Landed for 2.11

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/21055/
Subject: LU-8342 utils: Set dnodesize/recordsize at zfs dataset create
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 1617b8f6b6cdd0f5b74d7bfb8166d74b63cfed81

[Giuseppe Di Natale (Inactive)]

Currently no updates to report. I will try to revisit this soon.

[Gu Zheng (Inactive)]

Hi Giuseppe,

Any update about the patch http://review.whamcloud.com/21055?

[Andreas Dilger]

Olaf, I'm not against that, but it would definitely be more work than the current patch, and likely push the change out into 2.10.

[Olaf Faaland]

Andreas,
I understand. I'm suggesting that if those good defaults are encoded in a config file instead of in code, they (a) are visible to the user and (b) require trival code review to change. Also, the existing options do not distinguish between pool properties and dataset properties.

[Andreas Dilger]

There are already -mkfsoptions and -mountfsoptions that can be used to pass extra options to mkfs.lustre and to the internal mount command for the back-end filesystem. They should be able to override the default options specified internally by mkfs.lustre. My goal in specifying these options internally is that the majority of users should get the best performance out of the box if possible, rather than having to specify extra options.

[Olaf Faaland]

Perhaps lustre/conf/lustre is not the right place for such settings; I see there is an /etc/mke2fs.conf with ini-style contents, and some userspace apps seem to use /etc/default/foo. Anyway, basic proposal that these settings be put in a config file, instead of in the code, still stands.

[Olaf Faaland]

I see that lustre/conf/lustre already has ZPOOL_IMPORT_DIR and ZPOOL_IMPORT_ARGS. So perhaps ZPOOL_CREATE_ARGS and ZFS_CREATE_ARGS?

[Olaf Faaland]

Even if these pool and dataset settings are correct for all cases for now, they may become incorrect with future changes in either lustre or zfs. Furthermore, there are likely unusual cases (e.g. testing, see Jinshan's concern about test data often being highly compressible) where one or more of these settings are undesirable.

Both http://review.whamcloud.com/#/c/19892/2 and http://review.whamcloud.com/#/c/21055/3 set the desired settings by hard-coding specific values into mkfs.lustre. How about putting the settings themselves into a configuration file, e.g. /etc/sysconfig/lustre or distro-specific equivalent which is parsed by mkfs.lustre? Then they are visible to the user, can easily be changed when appropriate, and the defaults can be changed with a trivial patch that is easy to review.