[LU-14356] mkfs.lustre exited with 22 on Ubuntu 20.04.1 aarch64 Created: 21/Jan/21  Updated: 27/Jan/21  Resolved: 27/Jan/21

Status: Resolved
Project: Lustre
Component/s: None
Affects Version/s: Lustre 2.14.0
Fix Version/s: Lustre 2.14.0

Type: Bug Priority: Minor
Reporter: Jian Yu Assignee: Jian Yu
Resolution: Fixed Votes: 0
Labels: None

Severity: 3
Rank (Obsolete): 9223372036854775807

 Description   

 Running the following command on an Ubuntu 20.04.1 aarch64 server node exited with code 22:

$ sudo mkfs.lustre --mgs --fsname=lustre --mdt --index=0 --backfstype=ldiskfs --device-size=250000 --reformat /tmp/lustre-mdt1
$ echo $?
22

strace messages:

1611254854.585937 openat(AT_FDCWD, "/usr/lib/lustre/mount_osd_ldiskfs.so", O_RDONLY|O_CLOEXEC) = 3 <0.000052>
1611254854.586070 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0\267\0\1\0\0\0\0\34\0\0\0\0\0\0"..., 832) = 832 <0.000041>
1611254854.586185 fstat(3, {st_mode=S_IFREG|0644, st_size=119344, ...}) = 0 <0.000106>
1611254854.586362 mmap(NULL, 99064, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xffff93f34000 <0.000035>
1611254854.586456 mprotect(0xffff93f3a000, 65536, PROT_NONE) = 0 <0.000034>
1611254854.586543 mmap(0xffff93f4a000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6000) = 0xffff93f4a000 <0.000039>
1611254854.586654 mmap(0xffff93f4c000, 760, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xffff93f4c000 <0.000117>
1611254854.586857 close(3)              = 0 <0.000030>
1611254854.586976 munmap(0xffff93f34000, 99064) = 0 <0.000053>
1611254854.587104 openat(AT_FDCWD, "/usr/lib/lustre/mount_osd_ldiskfs.so", O_RDONLY|O_CLOEXEC) = 3 <0.000047>
1611254854.587250 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0\267\0\1\0\0\0\0\34\0\0\0\0\0\0"..., 832) = 832 <0.000035>
1611254854.587348 fstat(3, {st_mode=S_IFREG|0644, st_size=119344, ...}) = 0 <0.000029>
1611254854.587440 mmap(NULL, 99064, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xffff93f34000 <0.000032>
1611254854.587527 mprotect(0xffff93f3a000, 65536, PROT_NONE) = 0 <0.000036>
1611254854.587616 mmap(0xffff93f4a000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6000) = 0xffff93f4a000 <0.000036>
1611254854.587718 mmap(0xffff93f4c000, 760, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xffff93f4c000 <0.000032>
1611254854.587817 close(3)              = 0 <0.000032>
1611254854.587918 munmap(0xffff93f34000, 99064) = 0 <0.000045>
1611254854.588028 openat(AT_FDCWD, "/usr/lib/lustre/mount_osd_smfs.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) <0.000039>
1611254854.588149 openat(AT_FDCWD, "/usr/lib/lustre/mount_osd_reiserfs.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) <0.000037>
1611254854.588252 openat(AT_FDCWD, "/usr/lib/lustre/mount_osd_ldiskfs.so", O_RDONLY|O_CLOEXEC) = 3 <0.000038>
1611254854.588352 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0\267\0\1\0\0\0\0\34\0\0\0\0\0\0"..., 832) = 832 <0.000032>
1611254854.588449 fstat(3, {st_mode=S_IFREG|0644, st_size=119344, ...}) = 0 <0.000030>
1611254854.588542 mmap(NULL, 99064, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xffff93f34000 <0.000034>
1611254854.588631 mprotect(0xffff93f3a000, 65536, PROT_NONE) = 0 <0.000037>
1611254854.588722 mmap(0xffff93f4a000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6000) = 0xffff93f4a000 <0.000035>
1611254854.588820 mmap(0xffff93f4c000, 760, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xffff93f4c000 <0.000032>
1611254854.588915 close(3)              = 0 <0.000028>
1611254854.589012 munmap(0xffff93f34000, 99064) = 0 <0.000049>
1611254854.589119 openat(AT_FDCWD, "/usr/lib/lustre/mount_osd_zfs.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) <0.000037>
1611254854.589248 exit_group(22)        = ?
1611254854.589506 +++ exited with 22 +++


 Comments   
Comment by James A Simmons [ 21/Jan/21 ]

Looks like /usr/lib64/lustre/mount_osd_zfs.so is missing.

Comment by Jian Yu [ 21/Jan/21 ]

The backend fs type is ldiskfs and mount_osd_ldiskfs.so is located:

1611254854.588252 openat(AT_FDCWD, "/usr/lib/lustre/mount_osd_ldiskfs.so", O_RDONLY|O_CLOEXEC) = 3 <0.000038>
Comment by Jian Yu [ 22/Jan/21 ]

It turns out the calling of dlopen("/usr/lib/lustre/mount_osd_ldiskfs.so", RTLD_LAZY) in load_backfs_module() returns NULL.

Comment by Jian Yu [ 22/Jan/21 ]

I added dlerror() to print the error message from the failed dlopen() and got the following one:

/usr/lib/lustre/mount_osd_ldiskfs.so: undefined symbol: unix_io_manager
Comment by Jian Yu [ 22/Jan/21 ]

In lustre/utils/Makefile.am:

        $(CC) $(LDFLAGS) $(MNTMODLDFLAGS) -shared -export-dynamic -lext2fs \
                         -o mount_osd_ldiskfs.so \
                         `$(AR) -t libmount_utils_ldiskfs.a` \
                         $(MNTMODLIBS)

After I moved "-lext2fs" to the end of the command, the "undefined symbol: unix_io_manager" issue was resolved.

Comment by Gerrit Updater [ 23/Jan/21 ]

Jian Yu (yujian@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/41305
Subject: LU-14356 utils: change position of the -lext2fs option
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 4a13c875b567e6b8f9f4b035e468aaf9be64a59b

Comment by Gerrit Updater [ 27/Jan/21 ]

Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/41305/
Subject: LU-14356 utils: change position of the -lext2fs option
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 9cd651aead327ae4589b58dde5818b068c89b3e5

Comment by Peter Jones [ 27/Jan/21 ]

Landed for 2.14

Generated at Sat Feb 10 03:09:01 UTC 2024 using Jira 9.4.14#940014-sha1:734e6822bbf0d45eff9af51f82432957f73aa32c.