[LU-10157] LNET_MAX_IOV hard coded to 256

• Type: Bug
• Resolution: Fixed
• Priority: Blocker
• Fix Version/s: Lustre 2.12.0, Lustre 2.10.6, Lustre 2.14.0, Lustre 2.12.7
• Labels: patch
• Severity: 3
• Assignee: Alexey Lyashkov
• Reporter: Amir Shehata (Inactive)

Description

LNET_MAX_IOV is hard coded to 256 which works well for architectures with 4096 PAGE_SIZE. However on systems with page sizes bigger or smaller than 4K it will result in long MD count calculations, causing write errors.

Issue Links

• duplicates: LU-10775 (sec.c:2363:sptlrpc_svc_unwrap_bulk()) @@@ truncated bulk GET 1048576(2097152)
• is related to: LU-13181 kiblnd_fmr_pool_map error on the AARCH64 with 64k pages
• is related to: LU-10073 lnet-selftest test_smoke: lst Error found
• is related to: LU-6387 Add Power8 support to Lustre
• is related to: LU-10300 Can the Lustre 2.10.x clients support 64K kernel page?
• is related to: LU-10129 map-on-demand set to 32 doesn't work on OPA
• is related to: LU-7650 ko2iblnd map_on_demand can't negotitate when page sizes are different between nodes.

Comments

[Gerrit Updater]

Serguei Smirnov ([e-mail]) uploaded a new patch: https://review.whamcloud.com/41277
Subject: LU-10157 lnet: restore an maximal fragments count
Project: fs/lustre-release
Branch: b2_12
Current Patch Set: 1
Commit: 385819a69da21e8caf1b4b5ef80beb1b87f2c560

[Gerrit Updater]

Serguei Smirnov ([e-mail]) uploaded a new patch: https://review.whamcloud.com/41276
Subject: LU-10157 ptlrpc: fill md correctly.
Project: fs/lustre-release
Branch: b2_12
Current Patch Set: 1
Commit: 1b3f8e4ad1adeb52505aaa34be2a9246997658bf

[Gerrit Updater]

Serguei Smirnov ([e-mail]) uploaded a new patch: https://review.whamcloud.com/41275
Subject: LU-10157 ptlrpc: separate number MD and refrences for bulk
Project: fs/lustre-release
Branch: b2_12
Current Patch Set: 1
Commit: 0d8b5acaf0cf3e741652e93e9894c43a5bf21f2e

[Peter Jones]

Landed for 2.14

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/37385/
Subject: LU-10157 lnet: restore an maximal fragments count
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 4072d863c240fa5466f0f616f7e9b1cfcdf0aa0e

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/37387/
Subject: LU-10157 ptlrpc: fill md correctly.
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: e1ac9e74844dc75d77ef740b3a44fad2efde30c5

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/37386/
Subject: LU-10157 ptlrpc: separate number MD and refrences for bulk
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 8a7f2d4b11801eae4c91904da9f9750a012a6b11

[Alexey Lyashkov]

I think LNet should check an MD region size before send or attach. Other possibility is it export an fragments / MTU size tunable into ptlrpc instead of defines.

[James A Simmons]

I need to really think about this. I just see potential issues with confusion with fragment count and number of pages. I see it in the patch for lnet selftest user land tool for example. I think a better way can be done. Especially since this change touches all of the LNet layer yet its a IB specific issue.

[Cory Spitz]

Hmm, somehow there are two comments here from Gerrit Updater about https://review.whamcloud.com/#/c/37386, but none about https://review.whamcloud.com/#/c/37387. Please note that three pending patches (37385, 37386, and 37387) are tied to this ticket.

[Alexey Lyashkov]

No. up 256 fragments, depend of overall transfer size.
random io workload with <= 4k size on 64k pages - will fill a 256 entries in single transfer - but overall size is 1Mb or less.