Details

    • 5847

    Description

      dkms is a cross distro mechanism for the building and maintenance of out-of-tree kernel modules.
      (http://linux.dell.com/dkms/) The attached patch adds dkms support to the debian/ubuntu packaging infrastructure.

      The current debian/ubuntu kernel module package uses module-assistant, which produces a deb which is tied to a specific kernel version, requiring the user to manually rebuild packages when the kernel is changed.

      The dkms package allows modules for multiple kernel versions to be packaged together into a single deb. Once installed, the package contains triggers so that when a new kernel is installed for which no pre-built lustre modules exists, dkms will automatically build and install them. This reduces maintenance overhead on client machines.

      dkms also works on redhat and sles; it should be possible to fold dkms support into the rpm build process.

      Attachments

        1. diff
          10 kB
        2. dkms.patch
          0.6 kB
        3. lustre.spec
          9 kB

        Issue Links

          Activity

            [LU-1032] Add dkms support for kernel modules

            Hello Brian,
            No I am not aware that something has been done to ship a DKMS tool RPM including my patch.
            But on the other hand, seems that upstream integration of my fix is done and we need to now use dkms-2.2.0.3-28 RPM/version from Fedora FC[19-21]/EPEL[5-7] distros.

            bfaccini Bruno Faccini (Inactive) added a comment - Hello Brian, No I am not aware that something has been done to ship a DKMS tool RPM including my patch. But on the other hand, seems that upstream integration of my fix is done and we need to now use dkms-2.2.0.3-28 RPM/version from Fedora FC [19-21] /EPEL [5-7] distros.

            bfaccini: there was mention at one point that we needed a patch in DKMS and that while we wait for upstream to ship that we'd build our own DKMS RPM for EL6 and ship that. Did anything ever come of that?

            brian Brian Murrell (Inactive) added a comment - bfaccini : there was mention at one point that we needed a patch in DKMS and that while we wait for upstream to ship that we'd build our own DKMS RPM for EL6 and ship that. Did anything ever come of that?
            pjones Peter Jones added a comment -

            Patches landed for 2.6

            pjones Peter Jones added a comment - Patches landed for 2.6

            Finally it has been decided to better use Brian's set of patches to get this DKMS RPM generated. After some little re-work+rebase both changes #5960 and #6019 have already landed and #6020 will soon.

            I have also updated TEI-1359 to detail the build procedure and also the basic testing requirements to be included in our build/test tools.

            Possible next steps will be, 1st to add the ability to create a Client only DKMS RPM and also later the ldiskfs modules when no Kernel patching will be necessary.

            bfaccini Bruno Faccini (Inactive) added a comment - Finally it has been decided to better use Brian's set of patches to get this DKMS RPM generated. After some little re-work+rebase both changes #5960 and #6019 have already landed and #6020 will soon. I have also updated TEI-1359 to detail the build procedure and also the basic testing requirements to be included in our build/test tools. Possible next steps will be, 1st to add the ability to create a Client only DKMS RPM and also later the ldiskfs modules when no Kernel patching will be necessary.

            Actually working on the best way to package (Requires/Conflicts/...) DKMS lustre-client-modules RPM.
            BTW, I just added a "Conflicts: " for the lustre-modules itself since we need to ensure that no dual modules install (lustre-modules RPM vs DKMS rebuild) can occur that may lead to unpredictable behavior.

            bfaccini Bruno Faccini (Inactive) added a comment - Actually working on the best way to package (Requires/Conflicts/...) DKMS lustre-client-modules RPM. BTW, I just added a "Conflicts: " for the lustre-modules itself since we need to ensure that no dual modules install (lustre-modules RPM vs DKMS rebuild) can occur that may lead to unpredictable behavior.

            TEI-1359 has been created, as a follow-on to TT-1112 (that has been mistakenly mixed/duped with TT-1244/TEI-74) in order to have DKMS RPMs testing become part of autotests.

            bfaccini Bruno Faccini (Inactive) added a comment - TEI-1359 has been created, as a follow-on to TT-1112 (that has been mistakenly mixed/duped with TT-1244/TEI-74) in order to have DKMS RPMs testing become part of autotests.
            bfaccini Bruno Faccini (Inactive) added a comment - - edited

            For IEEL (INTL-26), I have re-worked my original patch http://review.whamcloud.com/5284 that allows Lustre-Client modules re-build under DKMS control. It is patch-set #9.

            It should need further add-on to handle llite_lloop and ptlrpd_gss special cases, due to their respective configure-time and gss/krb5 dependencies on target platforms.

            I also think that may be more Requirements need to be set for this DKMS RPM.

            Also, Lustre-Server modules future DKMS re-build has been somewhat planned.

            bfaccini Bruno Faccini (Inactive) added a comment - - edited For IEEL (INTL-26), I have re-worked my original patch http://review.whamcloud.com/5284 that allows Lustre-Client modules re-build under DKMS control. It is patch-set #9. It should need further add-on to handle llite_lloop and ptlrpd_gss special cases, due to their respective configure-time and gss/krb5 dependencies on target platforms. I also think that may be more Requirements need to be set for this DKMS RPM. Also, Lustre-Server modules future DKMS re-build has been somewhat planned.

            The zfsonlinux repository was updated to include dkms packages for spl/zfs-0.6.2 and lustre-2.4.0. The update branch can be found here https://github.com/chaos/lustre/tree/v2_4_0-dkms, it includes refreshed versions of the following two patches which have been submitted before but not yet included. I can push refreshed copies if upstream is interested in adding dkms support.

            https://github.com/chaos/lustre/commit/f7aeb9dd20c5c7cd044d06757f0782d8da8d2f92 -
            https://github.com/chaos/lustre/commit/4afc95703208fcb50eb35ef72466e1d8af97b46a

            behlendorf Brian Behlendorf added a comment - The zfsonlinux repository was updated to include dkms packages for spl/zfs-0.6.2 and lustre-2.4.0. The update branch can be found here https://github.com/chaos/lustre/tree/v2_4_0-dkms , it includes refreshed versions of the following two patches which have been submitted before but not yet included. I can push refreshed copies if upstream is interested in adding dkms support. https://github.com/chaos/lustre/commit/f7aeb9dd20c5c7cd044d06757f0782d8da8d2f92 - https://github.com/chaos/lustre/commit/4afc95703208fcb50eb35ef72466e1d8af97b46a

            I don't want to step on anybodies toes, but I took a crack at this since I have some experience with dkms packaging. With the following three patches applied I'm able to create a lustre-dkms package and a matching user space lustre package. The lustre-dkms package includes support for the client and support for ZFS servers. I was forced to disable the ldiskfs functionality because we don't have reliable dkms packaging for it yet. The patches extend the existing functionality in the Lustre build system in the following ways:

            http://review.whamcloud.com/#change,5960 - zfs-0.6.1 kmod+dkms compatibility
            http://review.whamcloud.com/#change,6019 - Add Lustre DKMS spec file
            http://review.whamcloud.com/#change,6020 - Honor --disable-modules option in spec file

            To verify that the packages work as expected I applied them to a branch off the Lustre v2.3.63 tag. For reference I've pushed that branch to github, https://github.com/chaos/lustre/tree/v2_3_63-dkms. Using that branch I built the required packages and added them to ZFS on Linux, EPEL 6 repository, see http://zfsonlinux.org/epel.html. Next I created a pristine CentOS 6 image in a VM, added the ZoL repository and did a 'yum install lustre'. As expected everything gets installed and you have either a Lustre client or Lustre ZFS based server ready for use. Here's a transcript of the install process.

            https://gist.github.com/behlendorf/5358099#file-lustre-dkms-install

            If you want to try it yourself just run the following commands in either a RHEL or CentOS 6 environment.

            sudo yum localinstall --nogpgcheck http://archive.zfsonlinux.org/epel/zfs-release-1-2.el6.noarch.rpm
            sudo yum install lustre
            

            As I mentioned earlier the packages are based off the 2.3.63 tag. It would be great if we could get these changes merged before the 2.3.64 tag is made. There are a few fixes in master which resolve some of the warnings which are issue during the build. My intention is to refresh the lustre packages in the ZFS on Linux repositories only when new upstream tags are made. When Lustre 2.4.0 is tagged I'll just start tracking the Lustre maintenance releases.

            behlendorf Brian Behlendorf added a comment - I don't want to step on anybodies toes, but I took a crack at this since I have some experience with dkms packaging. With the following three patches applied I'm able to create a lustre-dkms package and a matching user space lustre package. The lustre-dkms package includes support for the client and support for ZFS servers. I was forced to disable the ldiskfs functionality because we don't have reliable dkms packaging for it yet. The patches extend the existing functionality in the Lustre build system in the following ways: http://review.whamcloud.com/#change,5960 - zfs-0.6.1 kmod+dkms compatibility http://review.whamcloud.com/#change,6019 - Add Lustre DKMS spec file http://review.whamcloud.com/#change,6020 - Honor --disable-modules option in spec file To verify that the packages work as expected I applied them to a branch off the Lustre v2.3.63 tag. For reference I've pushed that branch to github, https://github.com/chaos/lustre/tree/v2_3_63-dkms . Using that branch I built the required packages and added them to ZFS on Linux, EPEL 6 repository, see http://zfsonlinux.org/epel.html . Next I created a pristine CentOS 6 image in a VM, added the ZoL repository and did a 'yum install lustre'. As expected everything gets installed and you have either a Lustre client or Lustre ZFS based server ready for use. Here's a transcript of the install process. https://gist.github.com/behlendorf/5358099#file-lustre-dkms-install If you want to try it yourself just run the following commands in either a RHEL or CentOS 6 environment. sudo yum localinstall --nogpgcheck http://archive.zfsonlinux.org/epel/zfs-release-1-2.el6.noarch.rpm sudo yum install lustre As I mentioned earlier the packages are based off the 2.3.63 tag. It would be great if we could get these changes merged before the 2.3.64 tag is made. There are a few fixes in master which resolve some of the warnings which are issue during the build. My intention is to refresh the lustre packages in the ZFS on Linux repositories only when new upstream tags are made. When Lustre 2.4.0 is tagged I'll just start tracking the Lustre maintenance releases.

            I understand, and I think you did a good job considering the existing state of the Lustre build system!

            Last, I am not sure I fully understand what DKMS packages inter-dependencies support will provide, is it capable for example to re-start an odd_zfs rebuild when a new zfs is installed?

            I don't know if it handles that or not. What it should handle is when a new kernel is installed. It should have DKMS compile the modules in this specific order: spl, zfs, osd-zfs. If they are built in any other order, the compilation will fail and DKMS will abort.

            morrone Christopher Morrone (Inactive) added a comment - I understand, and I think you did a good job considering the existing state of the Lustre build system! Last, I am not sure I fully understand what DKMS packages inter-dependencies support will provide, is it capable for example to re-start an odd_zfs rebuild when a new zfs is installed? I don't know if it handles that or not. What it should handle is when a new kernel is installed. It should have DKMS compile the modules in this specific order: spl, zfs, osd-zfs. If they are built in any other order, the compilation will fail and DKMS will abort.

            I was a bit late for the odd_zfs DKMS stuff, and since I was expecting a lot of feed-back from you guys, I decided to push a very basic patch version.
            I am not disappointed with the results!
            Thank's already to all for your ideas and contributions that are of a great help. I can say that at least some of the questions/potental-problems Chris listed in his 2 last updates went to my mind too at the time I was writing the patch.
            But again, I decided to do it first according what I think is possible with current Lustre build capabilities.
            I also think having autonomous/stand-alone build of osd_zfs module or full/Server Lustre DKMS would be the mandatory/principal next step.
            Last, I am not sure I fully understand what DKMS packages inter-dependencies support will provide, is it capable for example to re-start an odd_zfs rebuild when a new zfs is installed ?

            bfaccini Bruno Faccini (Inactive) added a comment - I was a bit late for the odd_zfs DKMS stuff, and since I was expecting a lot of feed-back from you guys, I decided to push a very basic patch version. I am not disappointed with the results! Thank's already to all for your ideas and contributions that are of a great help. I can say that at least some of the questions/potental-problems Chris listed in his 2 last updates went to my mind too at the time I was writing the patch. But again, I decided to do it first according what I think is possible with current Lustre build capabilities. I also think having autonomous/stand-alone build of osd_zfs module or full/Server Lustre DKMS would be the mandatory/principal next step. Last, I am not sure I fully understand what DKMS packages inter-dependencies support will provide, is it capable for example to re-start an odd_zfs rebuild when a new zfs is installed ?

            People

              bfaccini Bruno Faccini (Inactive)
              gmpc@sanger.ac.uk Guy Coates
              Votes:
              0 Vote for this issue
              Watchers:
              20 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: