[LU-5037] mpss 3.2.1 fails to build

• Status: Resolved
• Project: Lustre
• Fix Version/s: Lustre 2.7.0
• Type: Bug
• Priority: Major
• Reporter: Brian Murrell (Inactive)
• Assignee: Dmitry Eremin (Inactive)
• Resolution: Fixed
• Severity: 4

Description

If one tries to change the release of MPSS that lbuild tries to build to 3.2.1 (for example) it fails to build with:

+ '[' 3.2.1 = last ']'
+ [[ 3.2.1 != [0-9].[0-9].[0-9]*-[0-9]* ]]
+ fatal 1 'Incorrect MPSS version 3.2.1'
+ cleanup
+ true
+ error 'Incorrect MPSS version 3.2.1'
+ local 'msg=Incorrect MPSS version 3.2.1'
+ '[' -n 'Incorrect MPSS version 3.2.1' ']'
+ echo -e '\nlbuild: Incorrect MPSS version 3.2.1'

lbuild: Incorrect MPSS version 3.2.1

This appears to be because there is an expectation that MPSS versions always end with a -$number but they don't. For example, current releases are:

http://registrationcenter.intel.com/irc_nas/4030/mpss_gold_update_3-2.1.6720-23-rhel-6.0.tar
http://registrationcenter.intel.com/irc_nas/4030/mpss_gold_update_3-2.1.6720-23-rhel-6.1.tar
http://registrationcenter.intel.com/irc_nas/4030/mpss_gold_update_3-2.1.6720-23-rhel-6.2.tar
http://registrationcenter.intel.com/irc_nas/4030/mpss_gold_update_3-2.1.6720-23-rhel-6.3.tar
http://registrationcenter.intel.com/irc_nas/4030/mpss_gold_update_3-2.1.6720-23-rhel-6.4.tar
http://registrationcenter.intel.com/irc_nas/4030/mpss_gold_update_3-2.1.6720-23-suse-11.1.tar
http://registrationcenter.intel.com/irc_nas/4030/mpss_gold_update_3-2.1.6720-23-suse-11.2.tar
http://registrationcenter.intel.com/irc_nas/4030/mpss_gold_update_3_src-2.1.6720-23_rhel.tar
http://registrationcenter.intel.com/irc_nas/4030/mpss_gold_update_3_src-2.1.6720-23_suse.tar
http://registrationcenter.intel.com/irc_nas/3988/mpss-3.1.4-rhel-6.0.tar
http://registrationcenter.intel.com/irc_nas/3988/mpss-3.1.4-rhel-6.1.tar
http://registrationcenter.intel.com/irc_nas/3988/mpss-3.1.4-rhel-6.2.tar
http://registrationcenter.intel.com/irc_nas/3988/mpss-3.1.4-rhel-6.3.tar
http://registrationcenter.intel.com/irc_nas/3988/mpss-3.1.4-rhel-6.4.tar
http://registrationcenter.intel.com/irc_nas/3988/mpss-3.1.4-rhel-6.5.tar
http://registrationcenter.intel.com/irc_nas/3988/mpss-3.1.4-suse-11.2.tar
http://registrationcenter.intel.com/irc_nas/3988/mpss-3.1.4-suse-11.3.tar
http://registrationcenter.intel.com/irc_nas/3988/mpss-src-3.1.4.tar
http://registrationcenter.intel.com/irc_nas/3988/mpss-downloadcache-3.1.4.tar
http://registrationcenter.intel.com/irc_nas/3988/mpss-3.1.4-k1om-gdb.tar
http://registrationcenter.intel.com/irc_nas/4110/mpss-3.2.1-rhel-6.0.tar
http://registrationcenter.intel.com/irc_nas/4110/mpss-3.2.1-rhel-6.1.tar
http://registrationcenter.intel.com/irc_nas/4110/mpss-3.2.1-rhel-6.2.tar
http://registrationcenter.intel.com/irc_nas/4110/mpss-3.2.1-rhel-6.3.tar
http://registrationcenter.intel.com/irc_nas/4110/mpss-3.2.1-rhel-6.4.tar
http://registrationcenter.intel.com/irc_nas/4110/mpss-3.2.1-rhel-6.5.tar
http://registrationcenter.intel.com/irc_nas/4110/mpss-3.2.1-suse-11.2.tar
http://registrationcenter.intel.com/irc_nas/4110/mpss-3.2.1-suse-11.3.tar
http://registrationcenter.intel.com/irc_nas/4110/mpss-src-3.2.1.tar
http://registrationcenter.intel.com/irc_nas/4110/mpss-3.2.1-k1om.tar
http://registrationcenter.intel.com/irc_nas/4110/mpss-downloadcache-3.2.1.tar

Fixing this is required to get us off of a private-not-available-publicly release to something which can be downloaded and built automatically by lbuild. Having to deal with non-public releases is very expensive in terms of manpower as it requires TEI to manually download and deploy these builds.

Comments

[Peter Jones]

Dmitry

Can you comment on this?

Thanks

Peter

[Dmitry Eremin (Inactive)]

Actually the format of specifying MPSS version is not directly related to "private-not-available-publicly release". The number can be specified as zero and publicly available build will be used for build. Probably this behavior should be by default is number is not specified.

[Brian Murrell (Inactive)]

The number can be specified as zero and publicly available build will be used for build

dmiter: Can you be more precise? Which number can be 0? The whole version number or some component of it?

[Dmitry Eremin (Inactive)]

If you specify "--mpss-version=3.2.1-0" option for lbuild script it should download and build Lustre for MPSS 3.2.1.

[Brian Murrell (Inactive)]

Ahhh! Yes, this seems to work. One of the pitfalls of "special meaning" bits is that their special meaning is often lost.

Thanks for the info!

[Brian Murrell (Inactive)]

There is a problem with the solution however:

    # force re-download if build number is zero
    [[ $MPSS_VERSION = [0-9].[0-9].[0-9]*-0 ]] && force=true
...
        download_file "$url" "$file" "$force"

So while adding the -0 does make lbuild find the release, it has the side effect of causing a download to happen for each build even if the files already exist locally.

This is particularly ugly in the case of MPSS given that the download is 1.3GB!

[Dmitry Eremin (Inactive)]

It was done intentionally to avoid caching of old packages. If remove this line it will not be re-downloaded after new package become available on WEB page.

[Dmitry Eremin (Inactive)]

As workaround I can propose just rename packages in cache with fake build number after first run and use this number in command line for subsequent runs of lbuild script.

[Brian Murrell (Inactive)]

Yes, I understand that the -0 is a special nomenclature to force re-download but that conflicts with the requirement to add a (fake) -0 in order to pass the "is this a valid version" test that is in the code. The problem is that those are two different things but they are indicated using the same indicator so you can't have one without the other, but we need one without the other. We need to give a version that will pass the version check test (or change/fix the version check test) but not cause a re-download of 1.3GB of data for every build.

As workaround I can propose just rename packages in cache with fake build number after first run and use this number in command line for subsequent runs of lbuild script.

This is an unacceptable solution since it means having to manually do this on every builder of which there are many. It doesn't scale.

[Dmitry Eremin (Inactive)]

Now master will build with any version specification.