[LU-13485] Enable parallel compile tests during configure Created: 24/Apr/20  Updated: 19/Nov/23  Resolved: 06/Oct/23

Status: Closed
Project: Lustre
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Minor
Reporter: Shaun Tancheff Assignee: Shaun Tancheff
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
Blocker
is blocked by LU-17288 LU-13485 breaks Ubuntu build In Progress
Related
is related to LU-16053 osd-zfs build system update Open
Rank (Obsolete): 9223372036854775807

 Description   

Inspired by OpenZFS adding parallel compile testing
Adapt a similar scheme for lustre builds

Initial work adding parallel support for libcfs (libcfs/autoconf/lustre-libcfs.m4)
helps reduce the configure run from:

	serial		parallel	p: 38361
real	2m24.365s	1m53.446s	0m53.153s
user	1m34.807s	1m20.060s	0m54.937s
sys	1m9.891s	0m57.069s	0m31.493s

The conversion process is straight forward if a bit tedious



 Comments   
Comment by Gerrit Updater [ 24/Apr/20 ]

Shaun Tancheff (shaun.tancheff@hpe.com) uploaded a new patch: https://review.whamcloud.com/38347
Subject: LU-13485 build: Enable 2 stage configure tests
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: cdb4ca11a8513236a7c6171a8bec1045c3b0539f

Comment by Gerrit Updater [ 24/Apr/20 ]

Shaun Tancheff (shaun.tancheff@hpe.com) uploaded a new patch: https://review.whamcloud.com/38349
Subject: LU-13485 libcfs: Parallel configure tests
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 176f7c53447a7b5876c836ac19120ef3207cce74

Comment by Gerrit Updater [ 24/Apr/20 ]

Shaun Tancheff (shaun.tancheff@hpe.com) uploaded a new patch: https://review.whamcloud.com/38351
Subject: LU-13485 ldiskfs: Parallel configure tests
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: ab226d637eab8feb731c17d7ecb6dc7b8bbe922c

Comment by Gerrit Updater [ 24/Apr/20 ]

Shaun Tancheff (shaun.tancheff@hpe.com) uploaded a new patch: https://review.whamcloud.com/38361
Subject: LU-13485 kernel: Parallel configure tests
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 26711cf9db024997795a65b0c893e287b1bda098

Comment by Gerrit Updater [ 25/Apr/20 ]

Shaun Tancheff (shaun.tancheff@hpe.com) uploaded a new patch: https://review.whamcloud.com/38363
Subject: LU-13485 zfs: Parallel configure tests for zfs
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 76119d6ca17fc6b5a34113566742da36d98f4848

Comment by Gerrit Updater [ 25/Apr/20 ]

Shaun Tancheff (shaun.tancheff@hpe.com) uploaded a new patch: https://review.whamcloud.com/38368
Subject: LU-13485 lnet: Parallel configure tests for lnet
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 846c449bc169161d868069d3ac618821a81217b1

Comment by Gerrit Updater [ 10/May/20 ]

Shaun Tancheff (shaun.tancheff@hpe.com) uploaded a new patch: https://review.whamcloud.com/38562
Subject: LU-13485 all: Some kernel configure tests in parallel
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 06eb3ff8c4f5b7ca5d73fa60ee834827603cd54d

Comment by Gerrit Updater [ 27/May/20 ]

Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/38347/
Subject: LU-13485 build: Enable 2 stage configure tests
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: b0209c2d4d771eb1c728549483244a63e11a818e

Comment by Gerrit Updater [ 23/Jun/20 ]

Shaun Tancheff (shaun.tancheff@hpe.com) uploaded a new patch: https://review.whamcloud.com/39161
Subject: LU-13485 build: Make parallel tests names unique
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 9d5a1d5b071ab279e56dd49069b13e2fa1d7d646

Comment by Gerrit Updater [ 17/Jul/20 ]

Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/39161/
Subject: LU-13485 build: Make parallel tests names unique
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 24efa706939752487d8f8c9742400182d513157f

Comment by Gerrit Updater [ 23/Aug/20 ]

Shaun Tancheff (shaun.tancheff@hpe.com) uploaded a new patch: https://review.whamcloud.com/39710
Subject: LU-13485 libcfs: FIELD_SIZEOF macro removed
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 50c3332e43c4519263298013bb12b16c79bed2ce

Comment by Gerrit Updater [ 26/Feb/21 ]

Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/39710/
Subject: LU-13485 libcfs: FIELD_SIZEOF macro removed
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 03b7befcc0a9308cbac91370046f6c00e5cf1005

Comment by Gerrit Updater [ 12/Sep/21 ]

"Shaun Tancheff <shaun.tancheff@hpe.com>" uploaded a new patch: https://review.whamcloud.com/44896
Subject: LU-13485 all: Verbosity and resilience
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: a030121e702866796b7275ac6dd02444db455460

Comment by Gerrit Updater [ 15/Apr/22 ]

"Jian Yu <yujian@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/47077
Subject: LU-13485 build: remove parallel configure dir
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: e0db865abbe8d7fe3a61a649a17f70a88741eb66

Comment by Gerrit Updater [ 11/Jun/22 ]

"Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/47077/
Subject: LU-13485 build: remove parallel configure dir
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 67da76f1a89b20e16da15aafab11ca2bb3779dc7

Comment by Gerrit Updater [ 24/Jun/22 ]

"Jian Yu <yujian@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/47735
Subject: LU-13485 build: remove parallel configure dir
Project: fs/lustre-release
Branch: b2_15
Current Patch Set: 1
Commit: 3eb055e259a74c0c76338f91f2f58556b340bd49

Comment by Gerrit Updater [ 05/Jul/22 ]

"Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/47735/
Subject: LU-13485 build: remove parallel configure dir
Project: fs/lustre-release
Branch: b2_15
Current Patch Set:
Commit: 61e09a8f140dda9f8f1c3ec4a355383f4746e6f7

Comment by Gerrit Updater [ 30/Jul/22 ]

"Shaun Tancheff <shaun.tancheff@hpe.com>" uploaded a new patch: https://review.whamcloud.com/48091
Subject: LU-13485 libcfs: Remove unused iter_type check
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: d18b4c0ad27a6046736ec593549f7d662353ca90

Comment by Gerrit Updater [ 23/Sep/22 ]

"Shaun Tancheff <shaun.tancheff@hpe.com>" uploaded a new patch: https://review.whamcloud.com/48636
Subject: LU-13485 kernel: Remove verbose configure tests results
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 7fb00abc3deaa2b6f8f0a83b18747ed9e6bc594c

Comment by Gerrit Updater [ 10/Nov/22 ]

"Jian Yu <yujian@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/49092
Subject: LU-13485 libcfs: Parallel configure tests for libcfs
Project: fs/lustre-release
Branch: b2_15
Current Patch Set: 1
Commit: ed422ab53ca169196858c6ddbfdf6854627b89b9

Comment by Gerrit Updater [ 10/Nov/22 ]

"Jian Yu <yujian@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/49093
Subject: LU-13485 ldiskfs: Parallel configure tests
Project: fs/lustre-release
Branch: b2_15
Current Patch Set: 1
Commit: b26520b0b08215b6c454ae990712d7e6f13aa973

Comment by Gerrit Updater [ 10/Nov/22 ]

"Jian Yu <yujian@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/49098
Subject: LU-13485 kernel: Parallel core configure tests
Project: fs/lustre-release
Branch: b2_15
Current Patch Set: 1
Commit: 7cb7e52977ca24f064159392027369a36230e183

Comment by Gerrit Updater [ 10/Nov/22 ]

"Jian Yu <yujian@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/49099
Subject: LU-13485 zfs: Parallel configure tests for zfs
Project: fs/lustre-release
Branch: b2_15
Current Patch Set: 1
Commit: 016f6b16cd984052333c120c8c55c676e48cf42a

Comment by Gerrit Updater [ 10/Nov/22 ]

"Jian Yu <yujian@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/49100
Subject: LU-13485 lnet: Parallel configure tests for lnet
Project: fs/lustre-release
Branch: b2_15
Current Patch Set: 1
Commit: 03f2d3732be7d12c9945ec7e9354f8209b605582

Comment by Gerrit Updater [ 10/Nov/22 ]

"Jian Yu <yujian@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/49101
Subject: LU-13485 libcfs: Remove unused iter_type check
Project: fs/lustre-release
Branch: b2_15
Current Patch Set: 1
Commit: 4919b4d531b81fef0155b3476b489e227b39927d

Comment by Gerrit Updater [ 14/Nov/22 ]

"Shaun Tancheff <shaun.tancheff@hpe.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/49149
Subject: LU-13485 build: Parallel build configure cache fixes
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: c7ef9a8d0f59f72e4de4339a1b6f634677f560a3

Comment by Gerrit Updater [ 06/Dec/22 ]

"Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/49149/
Subject: LU-13485 build: Parallel build configure cache fixes
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 24c9f1f4c56d266656572e7bd97197d690d49369

Comment by Gerrit Updater [ 27/Jan/23 ]

"Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/38349/
Subject: LU-13485 libcfs: Parallel configure tests for libcfs
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 182fa9be075f5866aba2f37fbc3434cd0292ac0e

Comment by Gerrit Updater [ 27/Jan/23 ]

"Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/38361/
Subject: LU-13485 kernel: Parallel core configure tests
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: a346cf6cf22c285e05177a5a5e70a7d8e5bd9fa8

Comment by Gerrit Updater [ 27/Jan/23 ]

"Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/38351/
Subject: LU-13485 ldiskfs: Parallel configure tests for ldiskfs
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 3774b6afbe3b67e869bb61c9cb212cc37e8705fa

Comment by Gerrit Updater [ 27/Jan/23 ]

"Jian Yu <yujian@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/49805
Subject: LU-13485 build: Parallel build configure cache fixes
Project: fs/lustre-release
Branch: b2_15
Current Patch Set: 1
Commit: b01eb62089a5e5acf94f99c0b487eac7afe3bfb3

Comment by Gerrit Updater [ 22/Feb/23 ]

"Shaun Tancheff <shaun.tancheff@hpe.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/50110
Subject: LU-13485 build: fscrypt checks can be run in parallel
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 0d187c90229c61869d97d86d9127ff8ae975dec0

Comment by Gerrit Updater [ 28/Mar/23 ]

"Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/49805/
Subject: LU-13485 build: Parallel build configure cache fixes
Project: fs/lustre-release
Branch: b2_15
Current Patch Set:
Commit: 3bf0b2c3afe89a4db763fe33ac92164b477e4e74

Comment by Gerrit Updater [ 28/Mar/23 ]

"Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/49092/
Subject: LU-13485 libcfs: Parallel configure tests for libcfs
Project: fs/lustre-release
Branch: b2_15
Current Patch Set:
Commit: a9a07e4faa061b948a0b0a92cd35f1165b3a2bcb

Comment by Gerrit Updater [ 04/Apr/23 ]

"Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/38368/
Subject: LU-13485 lnet: Parallel configure tests for lnet
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: fc84caa81b7fb9d27e82229d39f046e83b5ebb7e

Comment by Gerrit Updater [ 04/Apr/23 ]

"Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/48091/
Subject: LU-13485 libcfs: Remove unused iter_type check
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: c755373c567090c49589e5aa0d3134847d4b952e

Comment by Gerrit Updater [ 04/Apr/23 ]

"Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/50110/
Subject: LU-13485 build: fscrypt checks can be run in parallel
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: e7900e9fcb67a39dc8323cdf6ac6290664dd066c

Comment by Shaun Tancheff [ 06/Oct/23 ]

Parallel build is infrastructure is landed. Remaining ZFS parallel work will be completed under another ticket.

Comment by Gerrit Updater [ 19/Nov/23 ]

"Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/49098/
Subject: LU-13485 kernel: Parallel core configure tests
Project: fs/lustre-release
Branch: b2_15
Current Patch Set:
Commit: 355da3ba8df599a771281df3f5db024b033946c9

Generated at Sat Feb 10 03:01:40 UTC 2024 using Jira 9.4.14#940014-sha1:734e6822bbf0d45eff9af51f82432957f73aa32c.