[LU-6163] 64-bit divides should use do_div()

• Status: Resolved
• Project: Lustre
• Affects Version/s: Lustre 2.7.0
• Fix Version/s: Lustre 2.9.0
• Type: Bug
• Priority: Minor
• Reporter: John Hammond
• Assignee: Amir Shehata (Inactive)
• Resolution: Fixed
• Labels: build, easy
• Severity: 3

Description

Andrew Uselton has pointer out that the i686 built modules are unusable on his 32-bit hardware due to undefined references to __umoddi3. Divides that that emit these calls should be rewritten to use do_div() or one of its friends. I grabbed the latest i686 RPMs and checked for these calls as follows:

wget 'https://build.hpdd.intel.com/job/lustre`master/arch=i686,build_type=server,distro=el6,ib_stack=inkernel/lastSuccessfulBuild/artifact/artifacts/RPMS/i686/lustre-modules-2.6.93-2.6.32_431.29.2.el6_lustre.g52480e5.i686_g52480e5.i686.rpm'
wget 'https://build.hpdd.intel.com/job/lustre-master/arch=i686,build_type=server,distro=el6,ib_stack=inkernel/lastSuccessfulBuild/artifact/artifacts/RPMS/i686/lustre-debuginfo-2.6.93-2.6.32_431.29.2.el6_lustre.g52480e5.i686_g52480e5.i686.rpm'
rpm2cpio lustre-modules-2.6.93-2.6.32_431.29.2.el6_lustre.g52480e5.i686_g52480e5.i686.rpm | cpio -itv
rpm2cpio lustre-modules-2.6.93-2.6.32_431.29.2.el6_lustre.g52480e5.i686_g52480e5.i686.rpm | cpio -idv
find lib -name '*.ko' -exec eu-unstrip {} usr/lib/debug/{}.debug \;
find /tmp/i686/usr -name '*.ko.debug' -exec objdump -dlr {} \; | grep -B30

Here is the slightly edited output:

kiblnd_get_completion_vector():
/usr/src/debug/lustre-2.6.93/lnet/klnds/o2iblnd/o2iblnd.c:696
   11bac:       89 c2                   mov    %eax,%edx
   11bae:       c1 fa 1f                sar    $0x1f,%edx
   11bb1:       89 04 24                mov    %eax,(%esp)
   11bb4:       89 f0                   mov    %esi,%eax
   11bb6:       89 54 24 04             mov    %edx,0x4(%esp)
   11bba:       89 fa                   mov    %edi,%edx
   11bbc:       e8 fc ff ff ff          call   11bbd <kiblnd_create_conn+0x87d>
                        11bbd: R_386_PC32       __umoddi3


delay_rule_match():
/usr/src/debug/lustre-2.6.93/lnet/lnet/net_fault.c:507
   3215d:       89 04 24                mov    %eax,(%esp)
   32160:       89 d8                   mov    %ebx,%eax
   32162:       c7 44 24 04 00 00 00    movl   $0x0,0x4(%esp)
   32169:       00
   3216a:       e8 fc ff ff ff          call   3216b <lnet_delay_rule_match_locked+0x2bb>
                        3216b: R_386_PC32       __umoddi3

drop_rule_match():
/usr/src/debug/lustre-2.6.93/lnet/lnet/net_fault.c:337
/usr/src/debug/lustre-2.6.93/lnet/lnet/net_fault.c:339
   33bd0:       89 0c 24                mov    %ecx,(%esp)
   33bd3:       c7 44 24 04 00 00 00    movl   $0x0,0x4(%esp)
   33bda:       00
   33bdb:       e8 fc ff ff ff          call   33bdc <lnet_drop_rule_match+0x25c>
                        33bdc: R_386_PC32       __umoddi3


lmv_hash_fnv1a():
/usr/src/debug/lustre-2.6.93/lustre/include/lustre_lmv.h:167
   5e62e:       8b 75 f0                mov    -0x10(%ebp),%esi
   5e631:       8b 45 dc                mov    -0x24(%ebp),%eax
   5e634:       8b 55 e0                mov    -0x20(%ebp),%edx
   5e637:       c7 44 24 04 00 00 00    movl   $0x0,0x4(%esp)
   5e63e:       00
   5e63f:       8b 5d e8                mov    -0x18(%ebp),%ebx
   5e642:       89 34 24                mov    %esi,(%esp)
   5e645:       89 4d e4                mov    %ecx,-0x1c(%ebp)
   5e648:       e8 fc ff ff ff          call   5e649 <lfsck_is_valid_slave_name_entry+0x199>
                        5e649: R_386_PC32       __umoddi3

Issue Links

• is duplicated by: LU-8835 More 64 bit divides without do_div.
• is related to: LU-6174 do_div() silently truncates divisor t...

Comments

[Bob Glossman (Inactive)]

This patch allowed me to do a 32 bit build of lustre with modules that don't report undefined symbols at load time. I'm pushing it here as an attachment to provide a possible starting point for a real fix and as a basis for discussion. Don't have enough confidence in it to submit as a proposed fix yet.

[James A Simmons]

Bob I just ran into this issue with the upstream client. The ko2iblnd version has been fixed already upstream. Its time to push this to master.

Looking at the patch cfs_rand() it returns a 32 bit integer so we don't need to do a do_div().

[Gerrit Updater]

James Simmons ([e-mail]) uploaded a new patch: http://review.whamcloud.com/18819
Subject: LU-6163 kernel: used do_div64() for 64 bit divides
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: b5c7321f04d22228bb3d46125cd82c0ef115267c

[James A Simmons]

I don't have 32 bit hardware so let me know if this fixes the issues on i386 platforms.

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch http://review.whamcloud.com/18819/
Subject: LU-6163 kernel: use do_div64() for 64 bit divides
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: e5ef85126953f866e155bb4f55cd85da963bde52

[Joseph Gmitter (Inactive)]

Landed to master for 2.9.0

[James A Simmons]

Bob and Andrew can you confirm this fixed lustre for 32 bit platforms?

[Andrew Uselton (Inactive)]

I actually recycled all that old 32-bit hardware before moving to Austin. I'd be happy to try it, but don't have a platform for it any more...Bob?
Andrew thinks, "dodged that bullet"

[Bob Glossman (Inactive)]

can't promise to test on 32 bits anytime soon. while I have a large set of 64 bit build and test environments at my fingertips I have no 32 bit ones atm. 32 bit is very low priority. While it's possible to set up 32 bit VMs for build and test purposes it's not something I do routinely.

[James A Simmons]

Rasberry PI anyone?