[LU-7968] build on master taking too long, causing huge backlogs Created: 28/Mar/16  Updated: 01/Jun/16  Resolved: 31/Mar/16

Status: Resolved
Project: Lustre
Component/s: None
Affects Version/s: None
Fix Version/s: Lustre 2.9.0

Type: Improvement Priority: Blocker
Reporter: Bob Glossman (Inactive) Assignee: Dmitry Eremin (Inactive)
Resolution: Fixed Votes: 0
Labels: None

Rank (Obsolete): 9223372036854775807

 Description   

It sure looks like as of recently all builds are taking > 90 minutes. I think this is due to all builds repeating server kernel builds even if they don't need to.

When no kernel builds get reused, no review build is fast. Backlogs grow quickly and empty out slowly.

wondering if some recent change in master has stopped reuse of server kernel builds causing all builds to rebuild kernels and take too long. need TEI to look at recent builds. I'm suspicious of recent landing for LU-7699. worried that changes in version and rpm names prevent build from recognizing perfectly good reusable kernel builds.

I know for sure backlogs have been very large lately and it's still happening. My idea about the underlying cause is just a theory. Need somebody in TEI who is in a position to know or find out to investigate.



 Comments   
Comment by Minh Diep [ 30/Mar/16 ]

I agree with Bob that LU-7699 remove BUILDID in lbuild might cause the caching of reusable kernel to be different everytime. Hence it rebuild kernel every time. if this is true, this will be a lustre issue, not TEI.

Comment by Bob Glossman (Inactive) [ 31/Mar/16 ]

Minh: if it is a lustre issue the breakage and the fix seems likely to be in lbuild or other build infrastructure, not in lustre code itself. Not sure who is a proper expert to deal with such a breakage.

Comment by Gerrit Updater [ 31/Mar/16 ]

Dmitry Eremin (dmitry.eremin@intel.com) uploaded a new patch: http://review.whamcloud.com/19262
Subject: LU-7968 lbuild: Fix reuse signature calculation
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: c2de7e4557841ff1045cccacafd21546f5419318

Comment by Gerrit Updater [ 31/Mar/16 ]

Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/19262/
Subject: LU-7968 lbuild: Fix reuse signature calculation
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 0def2d51d9665cd83422210ae1359d62bed38521

Comment by Dmitry Eremin (Inactive) [ 31/Mar/16 ]

patch landed to master.

Generated at Sat Feb 10 02:13:28 UTC 2024 using Jira 9.4.14#940014-sha1:734e6822bbf0d45eff9af51f82432957f73aa32c.