[LU-3121] lustre-initialization-1: Can't load module 'osd-zfs'

• Type: Bug
• Resolution: Fixed
• Priority: Blocker
• Affects Version/s: Lustre 2.4.1
• Labels: HB zfs
• Severity: 3
• Assignee: Chris Gearing (Inactive)
• Reporter: Maloo

Description

This issue was created by maloo for Li Wei <[e-mail]>

This issue relates to the following test suite run: https://maloo.whamcloud.com/test_sets/418e2830-9f9f-11e2-9f27-52540035b04c.

The sub-test lustre-initialization_1 failed with the following error:

Test system failed to start single suite, so abandoning all hope and giving up

Info required for matching: lustre-initialization-1 lustre-initialization_1

For some unknown reason, the MDS couldn't load osd-zfs.ko:

08:17:45:LNet: Accept all, port 7988
08:17:45:LustreError: 158-c: Can't load module 'osd-zfs'
08:17:45:LustreError: 2944:0:(genops.c:304:class_newdev()) OBD: unknown type: osd-zfs
08:17:45:LustreError: 2944:0:(obd_config.c:374:class_attach()) Cannot create device lustre-MDT0000-osd of type osd-zfs : -19
08:17:45:LustreError: 2944:0:(obd_mount.c:196:lustre_start_simple()) lustre-MDT0000-osd attach error -19
08:17:45:LustreError: 2944:0:(obd_mount_server.c:1664:server_fill_super()) Unable to start osd on lustre-mdt1/mdt1: -19
08:17:45:LustreError: 2944:0:(obd_mount.c:1264:lustre_fill_super()) Unable to mount  (-19)

Issue Links

• is related to: LU-2969 Fail to mount MDS as zfs on external OFED build

Comments

[Jian Yu]

Lustre Branch: master
Lustre Build: http://build.whamcloud.com/job/lustre-master/1495 (which contains the patch for LU-3269)
FSTYPE=zfs

The issue in this ticket still occurred:
https://maloo.whamcloud.com/test_sessions/754709ca-bd37-11e2-a548-52540035b04c

[Nathaniel Clark]

The above linked patch has landed for LU-3269.

[Nathaniel Clark]

If it's a module dependency ordering issue, it may be solved by http://review.whamcloud.com/6259 which fixes up lustre-osd-* / lustre-modules rpm dependencies.

[Nathaniel Clark]

I can reproduce these exact symptoms by installing all of lustre except lustre-osd-zfs normally, then install lustre-osd-zfs with --noscripts (causing it to skip running depmod). lustre-osd-zfs may skip depmod if the kernel isn't installed yet, still trying to figure out this exact vector.

[Nathaniel Clark]

rpms seem to install and run cleanly, must be an install issue of some sort. Possibly depmod issue?

[Jian Yu]

Another one: https://maloo.whamcloud.com/test_sets/cd712682-b72c-11e2-bd0f-52540035b04c

[Jian Yu]

This is blocking the patch review testing on zfs:
https://maloo.whamcloud.com/test_sets/f3082170-b678-11e2-a5af-52540035b04c

[Jian Yu]

Another instance: https://maloo.whamcloud.com/test_sets/2352a39e-b658-11e2-bf90-52540035b04c

[Zhenyu Xu]

another hit at https://maloo.whamcloud.com/test_sets/d170d43c-a767-11e2-b3cc-52540035b04c

[Li Wei (Inactive)]

Ah, there are also considerable amount of failures when testing other patches.

[Li Wei (Inactive)]

No problem. Ironically, my http://review.whamcloud.com/5785 has been hitting this issue every time recently, although I can't see any fault in the patch itself. Probably there really is a problem in the patch...