[LU-9341] PFL: append should not instantiate full layout

• Type: Bug
• Resolution: Fixed
• Priority: Major
• Fix Version/s: Lustre 2.13.0, Lustre 2.12.4
• Affects Version/s: Lustre 2.10.0
• Labels: DoM2 pfl
• Severity: 3
• Assignee: Patrick Farrell (Inactive)
• Reporter: Andreas Dilger

Description

Appending to a PFL file will cause all layout components to be instantiated because it isn't possible to know what the ending offset is at the time the write is started.

It would be better to avoid this, potentially by locking/instantiating some large(r), but not gigantic range beyond current EOF, and if that fails retry the layout intent? The client must currently be in charge of locking the file during append, so it should know at write time how much of the file to instantiate, and it could retry.

Issue Links

• is duplicated by: LU-10665 DoM: append to file causes OST component initialization
• is related to: LU-9479 sanity test 184d 244: don't instantiate PFL component when taking group lock
• is related to: LU-10176 Data-on-MDT phase II
• is related to: LU-13420 append to PFL-file without 'eof' component fails
• is related to: LU-17694 sanity-compr test_184d: last component index number got assigned even if it was not used after layout swap
• is related to: LU-15727 lod_get_default_lov_striping() misinterprets composite striping for append
• is related to: LU-12738 PFL: append of PFL file should not instantiate full layout
• is related to: LU-10782 Enable tiny write append for singly striped non-composite file
• is related to: LU-8998 Progressive File Layout (PFL)

Comments

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/49935/
Subject: LU-9341 tests: skip interop sanity test_27U
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 1ee5d4021a7123d43465b0db8cf2114944b4a980

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/37007/
Subject: LU-9341 lod: Add special O_APPEND striping
Project: fs/lustre-release
Branch: b2_12
Current Patch Set:
Commit: d0767ae660e7662215a07df83dbf784bb5fa6eb6

[Gerrit Updater]

Minh Diep ([e-mail]) uploaded a new patch: https://review.whamcloud.com/37007
Subject: LU-9341 lod: Add special O_APPEND striping
Project: fs/lustre-release
Branch: b2_12
Current Patch Set: 1
Commit: f377c3a34e73acfe3f74ee262f58086e0c4a2992

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/36935/
Subject: LU-9341 utils: fix lfs find for composite files
Project: fs/lustre-release
Branch: b2_12
Current Patch Set:
Commit: 6341522cc8088a367cd156a6f284823c69b92f7b

[Gerrit Updater]

Minh Diep ([e-mail]) uploaded a new patch: https://review.whamcloud.com/36935
Subject: LU-9341 utils: fix lfs find for composite files
Project: fs/lustre-release
Branch: b2_12
Current Patch Set: 1
Commit: 7939fd638606837b2552eb73cfb49d9ccf0fc7d8

[Nathan Dauchy (Inactive)]

Can this get pulled back into b2_10 and/or b2_12 for LTS?

[Peter Jones]

Landed for 2.13

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/35617/
Subject: LU-9341 lod: Add special O_APPEND striping
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: e2ac6e1eaa108eef3493837e9bd881629582ea1d

[Patrick Farrell (Inactive)]

Opened LU-12738 to track remaining work, this can be closed once https://review.whamcloud.com/#/c/35617/ lands.

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/35611/
Subject: LU-9341 utils: fix lfs find for composite files
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 72479a52be5f77f601d8234d957f5d6176edf6e8