[LU-11868] ZFS ea size limited to 32K

• Type: Bug
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.13.0
• Severity: 3
• Assignee: Patrick Farrell (Inactive)
• Reporter: Patrick Farrell (Inactive)

Description

The ZFS OSD limits the ea size to DXATTR_MAX_ENTRY_SIZE, which defaults to 32K.

This is done when ddp_max_ea_size is set:

param->ddp_max_ea_size = DXATTR_MAX_ENTRY_SIZE;

 

Per Alex Z., this is probably incorrect, since ZFS can use dedicated objects for EAs.

This was discovered and confirmed in testing overstriping (LU-9846):
https://testing.whamcloud.com/test_sets/ea834356-1552-11e9-9ed8-52540065bddc

Specifically, test 27ci:
This is a test for overstriping, which is > 1 stripe per OST. This tries to create 2000 stripes.
It works on ldiskfs (with ea_inode) enabled, but on ZFS, we only get 1363 total stripes

32768/24 bytes per stripe = 1365

So, minus a little for the rest of the layout EA, this matches.

 

So 32K is too small, especially if we increase the stripe limit to 10K, as the overstriping patch series does in a later patch.  The question is what should the limit be.

 

I would suggest as a possible value:
/* Maximum EA size is limited by LNET_MTU for remote objects */
#define OSD_MAX_EA_SIZE 1048364

Which is currently in the ldiskfs OSD, but is clearly not ldiskfs specific.

I'm curious to get feedback here.

Issue Links

• is related to: LU-12481 conf-sanity test_61: setfattr: /mnt/lustre/f61.conf-sanity: Argument list too long
• is related to: LU-11910 Improve repbuf/easize/mdsize handling on client
• is related to: LU-9846 Overstriping - more than stripe per OST per component

Comments

[Peter Jones]

Landed for 2.13

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/34058/
Subject: LU-11868 osd: Set max ea size to XATTR_SIZE_MAX
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 3ec712bd183a859a7bb09280b8a5a1776ec5e2c2

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/34059/
Subject: LU-11868 mdc: Improve xattr buffer allocations
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 4f78164f8748cf8013331637ba33388e83fbd627

[Gerrit Updater]

Patrick Farrell ([e-mail]) uploaded a new patch: https://review.whamcloud.com/34059
Subject: LU-11868 mdc: Improve xattr buffer allocations
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 055702a13537c99d7f09364a350d8027359c694e

[Gerrit Updater]

Patrick Farrell ([e-mail]) uploaded a new patch: https://review.whamcloud.com/34058
Subject: LU-11868 osd: Set max ea size to XATTR_SIZE_MAX
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: b730ac71c6461b34b196a90066a38d94d3baf87d

[Patrick Farrell (Inactive)]

Yes, trying to access an ea_size greater than 64K causes E2BIG from getfattr (specifically, the getxattr syscall).

Here's an example with 2,730 stripes, trusted.lov is 65792 bytes:
getxattr("2730file", "trusted.lov", 0x23e2f70, 65792) = -1 E2BIG (Argument list too long)

2720 stripes is fine:

getxattr("2720file", "trusted.lov", [...], 65536) = 65312

I'm going to assume this limit applies to at least some of the other tools as well, so we'll have to respect it.

Just to confirm: This matters for manual editing and backups of MDTs, right?