[LU-45] building e2fsprogs should not require lustre to be installed or a built source tree

• Type: Improvement
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.0.0
• Severity: 3
• Assignee: Bruce Cassidy (Inactive)
• Reporter: Brian Murrell (Inactive)

Description

In order to complete the build of e2fsprogs, the liblustreapi library needs to be accessible. Currently this is only possible if a built lustre source tree is available on the system building e2fsprogs. It should be available via an installable lustre-devel package.

It is also available in the rpm installed in /usr/lib[64] but I'm not convinced that e2fsprogs' configure will actually find that.

Comments

[Bruce Cassidy (Inactive)]

Andreas checked in code to fix this issue. That code is included in e2fsprogs-1.41.90.wc1. I tested the code for building in a machine that just had lustre client code installed via rpm. The build succeeded.

[Bruce Cassidy (Inactive)]

Test: Set up a cleanToro machine. yum install:
autoconf
make
db4*
gcc
get lustre-client rpm from lustre build-200 and rpm -i.
get lustre-client-modules from lustre build-200 and rpm -i.
get e2fsprogs-1.41.90.wc1 source code from gerrit.
Insource directory:
autoconf
./configure --with-lustre=/usr
make
make check

Output:
149 tests succeeded 0 tests failed

[Bruce Cassidy (Inactive)]

Test: Set up a cleanToro machine. yum install:
autoconf
make
db4*
gcc
get lustre-client rpm from lustre build-200 and rpm -i.
get lustre-client-modules from lustre build-200 and rpm -i.
get e2fsprogs-1.41.90.wc1 source code from gerrit.
Insource directory:
autoconf
./configure --with-lustre=/usr
make
make check

Output:
149 tests succeeded 0 tests failed

[Bruce Cassidy (Inactive)]

Test: Set up a cleanToro machine. yum install:
autoconf
make
db4*
gcc
get lustre-client rpm from lustre build-200 and rpm -i.
get lustre-client-modules from lustre build-200 and rpm -i.
get e2fsprogs-1.41.90.wc1 source code from gerrit.
Insource directory:
autoconf
./configure --with-lustre=/usr
make
make check

Output:
149 tests succeeded 0 tests failed

[Andreas Dilger]

This is addressed by the e2fsprogs-1.42/master-lustre patchset that is in Gerrit right now.

Bruce should close it when that is committed.

[Build Master (Inactive)]

Integrated in reviews-e2fsprogs #26
LU-45 update e2fsprogs for pre-1.42 build

Andreas Dilger :
Files :

• patches/e2fsprogs-tests-f_ibadness_bad_extents.patch
• patches/e2fsprogs-tests-f_uninit_checksum_invalid.patch
• patches/e2fsprogs-bad-ininode-ea.patch
• patches/sles/libcom_err-no-static-buffer.patch
• patches/e2fsprogs-tests-f_extents_eh_magic.patch
• patches/e2fsprogs-tests-f_extents_orphan_blks.patch
• patches/e2fsprogs-tests-f_extents_res_blk.patch
• patches/e2fsprogs-osx.patch
• patches/e2fsprogs-tests-f_extents_unsorted.patch
• patches/e2fsprogs-tests-f_extents_ee_len.patch
• patches/e2fsprogs-tests-f_uninit_restart_fsck.patch
• patches/e2fsprogs-tests-f_extents_eh_entries.patch
• patches/e2fsprogs-ibadness-counter.patch
• patches/e2fsprogs-version.patch
• patches/e2fsprogs-config-before-cmdline.patch
• patches/e2fsprogs-tests-f_extents_ei_block.patch
• patches/e2fsprogs-tests-f_extents_split.patch
• patches/e2fsprogs-fiemap-lustre.patch
• patches/e2fsprogs-lfsck.patch
• patches/e2fsprogs-extents.patch
• patches/e2fsprogs-blkid-zfs.patch
• patches/series
• patches/e2fsprogs-expand-extra-isize.patch
• patches/e2fsprogs-tests-f_jchksum_bblk.patch
• patches/e2fsprogs-tests-f_extents_ee_start.patch
• patches/e2fsprogs-tests-f_uninit_ino-ref_beyond_bg-itable-unused.patch
• patches/e2fsprogs-tests-f_large_ea.patch
• patches/e2fsprogs-tests-f_extents_inrlevel-incons.patch
• patches/e2fsprogs-tests-f_expisize_ea_del.patch
• patches/README
• patches/e2fsprogs-tests-f_extents_imbalanced_tree.patch
• patches/sles/11/system-blkid.patch
• patches/sles/11/rpm/e2fsprogs-1.41.1-splash_support.patch
• patches/e2fsprogs-e2scan.patch
• patches/e2fsprogs-count_wrong_valid.patch
• patches/e2fsprogs-tests-f_jchksum_remount.patch
• patches/e2fsprogs-large-ea.patch
• patches/e2fsprogs-bigjournal.patch
• patches/e2fsprogs-tests-f_extents_eh_max.patch
• patches/e2fsprogs-uninit-fixes.patch
• patches/e2fsprogs-tests-f_uninit_blk-uninit-set_inode-uninit-not-set.patch
• patches/e2fsprogs-size_high.patch
• patches/e2fsprogs-e2fsck_exclusive.patch
• patches/e2fsprogs-rpm_support.patch
• patches/e2fsprogs-tests-f_mmp.patch
• patches/e2fsprogs-tests-f_extents_overlap.patch
• patches/sles/11/rpm/e2fsprogs-1.41.4.de.po
• patches/e2fsprogs-tests-f_ibadness.patch
• patches/e2fsprogs-tests-f_uninit_incorrect_free_inodes_count.patch
• patches/e2fsprogs-tests-f_expisize.patch
• patches/e2fsprogs-tests-f_extents_ee_block.patch
• patches/e2fsprogs-lazy_journal_init.patch
• patches/e2fsprogs-extended_ops.patch
• patches/e2fsprogs-tests-f_uninit_disable.patch
• patches/e2fsprogs-tests-f_random_corruption.patch
• patches/e2fsprogs-tests-f_uninit_blks-used_blk-uninit-not-set.patch
• patches/e2fsprogs-tests-f_extents_bad_blk.patch
• patches/e2fsprogs-tunefs-dirty.patch
• patches/e2fsprogs-add-io_channel_readahead.patch
• patches/e2fsprogs-attr_cleanup.patch
• patches/e2fsprogs-mmp.patch
• patches/e2fsprogs-skip-extent-tests.patch
• patches/e2fsprogs-tests-f_jchksum_blast_trans.patch
• patches/sles/e2fsprogs-SLES10--m-support.patch
• patches/e2fsprogs-i_size-corruption.patch
• patches/e2fsprogs-tests-f_uninit_enable.patch
• patches/e2fsprogs-new-opt-shrdblks.patch
• patches/e2fsprogs-tests-f_extents_eh_depth.patch
• patches/sles/libcom_err-no-init_error_table.patch
• patches/libcom_err-unidiff.patch
• patches/e2fsprogs-data_in_dirent.patch
• patches/e2fsprogs-dblist.patch
• patches/e2fsprogs-tests-f_extents_shrd_blk.patch
• patches/ChangeLog
• patches/e2fsprogs-tests-f_extents_ei_leaf.patch
• patches/e2fsprogs-specdotin.patch
• patches/e2fsprogs-tests-f_bad_ea_value.patch
• patches/sles/11/rpm/README.SUSE
• patches/e2fsprogs-add-trusted-fid.patch
• patches/e2fsprogs-misc.patch
• patches/e2fsprogs-tests-f_data_in_dirent.patch
• patches/sles/11/rpm/libcom_err-compile_et_permissions.patch
• patches/e2fsprogs-tests-s_basic_scan.patch
• patches/e2fsprogs-lustre_ismounted.patch
• patches/e2fsprogs-tests-f_many_subdirs.patch

[Build Master (Inactive)]

Integrated in reviews-e2fsprogs #25
LU-45 update e2fsprogs for pre-1.42 build

Andreas Dilger :
Files :

• patches/e2fsprogs-tests-f_extents_ee_len.patch
• patches/sles/e2fsprogs-SLES10--m-support.patch
• patches/series
• patches/e2fsprogs-tests-f_data_in_dirent.patch
• patches/e2fsprogs-tests-f_uninit_disable.patch
• patches/e2fsprogs-tests-f_extents_res_blk.patch
• patches/e2fsprogs-config-before-cmdline.patch
• patches/e2fsprogs-tests-f_ibadness_bad_extents.patch
• patches/e2fsprogs-tests-f_jchksum_blast_trans.patch
• patches/e2fsprogs-extended_ops.patch
• patches/e2fsprogs-version.patch
• patches/e2fsprogs-tests-f_large_ea.patch
• patches/e2fsprogs-extents.patch
• patches/libcom_err-unidiff.patch
• patches/e2fsprogs-add-io_channel_readahead.patch
• patches/e2fsprogs-dblist.patch
• patches/e2fsprogs-tests-f_jchksum_remount.patch
• patches/README
• patches/e2fsprogs-tests-f_extents_unsorted.patch
• patches/e2fsprogs-large-ea.patch
• patches/e2fsprogs-rpm_support.patch
• patches/sles/11/rpm/e2fsprogs-1.41.4.de.po
• patches/e2fsprogs-add-trusted-fid.patch
• patches/e2fsprogs-tests-f_extents_ei_block.patch
• patches/sles/11/system-blkid.patch
• patches/e2fsprogs-tests-f_ibadness.patch
• patches/e2fsprogs-tests-f_uninit_blk-uninit-set_inode-uninit-not-set.patch
• patches/sles/11/rpm/libcom_err-compile_et_permissions.patch
• patches/e2fsprogs-uninit-fixes.patch
• patches/e2fsprogs-misc.patch
• patches/e2fsprogs-tests-f_many_subdirs.patch
• patches/e2fsprogs-bad-ininode-ea.patch
• patches/e2fsprogs-tests-f_jchksum_bblk.patch
• patches/e2fsprogs-lustre_ismounted.patch
• patches/sles/11/rpm/README.SUSE
• patches/e2fsprogs-tests-f_extents_shrd_blk.patch
• patches/e2fsprogs-mmp.patch
• patches/e2fsprogs-tests-f_extents_eh_entries.patch
• patches/e2fsprogs-tests-f_uninit_enable.patch
• patches/e2fsprogs-expand-extra-isize.patch
• patches/e2fsprogs-attr_cleanup.patch
• patches/e2fsprogs-tests-f_extents_inrlevel-incons.patch
• patches/e2fsprogs-blkid-zfs.patch
• patches/e2fsprogs-tests-f_uninit_incorrect_free_inodes_count.patch
• patches/e2fsprogs-tests-f_random_corruption.patch
• patches/e2fsprogs-new-opt-shrdblks.patch
• patches/e2fsprogs-tests-f_uninit_checksum_invalid.patch
• patches/e2fsprogs-count_wrong_valid.patch
• patches/e2fsprogs-i_size-corruption.patch
• patches/e2fsprogs-osx.patch
• patches/e2fsprogs-tests-f_extents_ee_start.patch
• patches/e2fsprogs-tests-f_bad_ea_value.patch
• patches/e2fsprogs-size_high.patch
• patches/e2fsprogs-lazy_journal_init.patch
• patches/e2fsprogs-ibadness-counter.patch
• patches/ChangeLog
• patches/e2fsprogs-tests-f_extents_eh_depth.patch
• patches/e2fsprogs-data_in_dirent.patch
• patches/e2fsprogs-tests-f_extents_ei_leaf.patch
• patches/e2fsprogs-tests-f_extents_split.patch
• patches/e2fsprogs-e2scan.patch
• patches/sles/libcom_err-no-static-buffer.patch
• patches/e2fsprogs-skip-extent-tests.patch
• patches/e2fsprogs-tests-f_mmp.patch
• patches/e2fsprogs-tunefs-dirty.patch
• patches/e2fsprogs-tests-f_uninit_restart_fsck.patch
• patches/e2fsprogs-e2fsck_exclusive.patch
• patches/e2fsprogs-tests-f_extents_eh_magic.patch
• patches/e2fsprogs-tests-f_extents_overlap.patch
• patches/e2fsprogs-tests-f_expisize.patch
• patches/e2fsprogs-fiemap-lustre.patch
• patches/e2fsprogs-tests-s_basic_scan.patch
• patches/sles/libcom_err-no-init_error_table.patch
• patches/e2fsprogs-tests-f_uninit_blks-used_blk-uninit-not-set.patch
• patches/e2fsprogs-lfsck.patch
• patches/e2fsprogs-tests-f_extents_eh_max.patch
• patches/e2fsprogs-bigjournal.patch
• patches/e2fsprogs-specdotin.patch
• patches/e2fsprogs-tests-f_uninit_ino-ref_beyond_bg-itable-unused.patch
• patches/e2fsprogs-tests-f_extents_imbalanced_tree.patch
• patches/e2fsprogs-tests-f_extents_bad_blk.patch
• patches/sles/11/rpm/e2fsprogs-1.41.1-splash_support.patch
• patches/e2fsprogs-tests-f_expisize_ea_del.patch
• patches/e2fsprogs-tests-f_extents_ee_block.patch
• patches/e2fsprogs-tests-f_extents_orphan_blks.patch

[Andreas Dilger]

In change 315 in Gerrit, I've attached a patchset for e2fsprogs-1.42 which, among many other things, can build against the installed Lustre RPM files. In order to do this, run:

configure --with-lustre=/usr

and it will find the include files under @LUSTRE@/include, and the libraries under @LUSTRE@/lib

{64}

. It is a bit clunky because it adds both "-L @LUSTRE@/lib64 -L @LUSTRE/lib" to LDFLAGS, but ld should do the right thing. If there was an easier way to know in the configure.in file which one was correct it could be fixed.

[Brian Murrell (Inactive)]

I have not actually tried a build that way but I'm not convinced. Looking at the patch we can see the link command line is:

$(LD) -pthread $(ALL_LDFLAGS) $(LUSTRE_INC) $(LFSCK_OBJS) -o lfsck $(LUSTRE_LIB) -llustreapi $(DEPLIBS)

and $LUSTRE_INC and $LUSTRE_LIB are defined as:

LUSTRE_INC=-I @LUSTRE@/lustre/include -Wall
LUSTRE_LIB=-L @LUSTRE@/lustre/utils

and LUSTRE is defined in configure as:

1. Check whether --with-lustre was given.
   if test "${with_lustre+set}" = set; then :
   withval=$with_lustre; { $as_echo "$as_me:${as_lineno-$LINENO}: result: Lustre is in $withval" >&5
   $as_echo "Lustre is in $withval" >&6; }
   LUSTRE="$withval"
   else
   LUSTRE="/usr/src/lustre"
   fi

So it's possible that a --with-lustre=/usr/include might work where the liblustreapi.a library is just found in the regular search path.

it's ugly at best though. That macro to find LUSTRE could certainly do some more exhaustive checking for an installed lustre rpm. But even still, a build machine should not need the lustre RPM installed but should be able to work with a lustre-devel RPM that has the includes and libraries in it. The macro above would still need to be made better, to search for the locations that the lustre-devel RPM will drop the includes and libraries.

[Andreas Dilger]

The e2fsprogs build should be able to get liblustreapi.h and liblustreapi.a from the installed lustre RPM, but it has been a long time since I checked whether this worked.