Details

    • Improvement
    • Resolution: Duplicate
    • Minor
    • None
    • None
    • 18,766
    • 7561

    Description

      We need to get upwards of 10k of ip2nets and 10k of route data loaded into LNet at module startup time.

      Attachments

        Issue Links

          Activity

            [LU-1071] able to handle many lnet routes
            doug Doug Oucharek (Inactive) added a comment - - edited

            Ticket LU-2950 has implement a way from user space to configure many LNet routes which will mirror what Dynamic Lnet Config will be doing. LU-2950 will be landing in 2.5 and is ready to cherry pick now.

            Note: this addresses routes, not ip2nets entries. Those have to wait for the full Dynamic LNet Config.

            doug Doug Oucharek (Inactive) added a comment - - edited Ticket LU-2950 has implement a way from user space to configure many LNet routes which will mirror what Dynamic Lnet Config will be doing. LU-2950 will be landing in 2.5 and is ready to cherry pick now. Note: this addresses routes, not ip2nets entries. Those have to wait for the full Dynamic LNet Config.

            Yes, the Dynamic LNet Config feature is landing in 2.4 and brings with it the ability to configure an almost unlimited number of routes. It does this by letting you dynamically configure one route at a time. That means you will still need a script in user space to take a list of routes and to invoke the "lent" utility for each route entry to get it configured.

            doug Doug Oucharek (Inactive) added a comment - Yes, the Dynamic LNet Config feature is landing in 2.4 and brings with it the ability to configure an almost unlimited number of routes. It does this by letting you dynamically configure one route at a time. That means you will still need a script in user space to take a list of routes and to invoke the "lent" utility for each route entry to get it configured.

            It seems Dynamic Lnet Config is moving. The main LU can be see at http://jira.whamcloud.com/browse/LU-2456.

            keith Keith Mannthey (Inactive) added a comment - It seems Dynamic Lnet Config is moving. The main LU can be see at http://jira.whamcloud.com/browse/LU-2456 .

            Without any update I will close this LU at the end of the month.

            keith Keith Mannthey (Inactive) added a comment - Without any update I will close this LU at the end of the month.

            This patch has not been resubmitted and there are outstanding issues covered in review. There is no solution proposed at this point.

            keith Keith Mannthey (Inactive) added a comment - This patch has not been resubmitted and there are outstanding issues covered in review. There is no solution proposed at this point.

            Now that OpenSFS seems to be moving the RFP process forward, there is a hope that the Dynamic LNet Config work can make the 2.4 release of Lustre.

            Would that be soon enough or will this patch be needed in an earlier release?

            doug Doug Oucharek (Inactive) added a comment - Now that OpenSFS seems to be moving the RFP process forward, there is a hope that the Dynamic LNet Config work can make the 2.4 release of Lustre. Would that be soon enough or will this patch be needed in an earlier release?

            Is there any update for dynamic-lnet-config or this patch? I would hope we can land something to fill this gap.

            keith Keith Mannthey (Inactive) added a comment - Is there any update for dynamic-lnet-config or this patch? I would hope we can land something to fill this gap.

            Xyratex MRP-342

            nrutman Nathan Rutman added a comment - Xyratex MRP-342

            I think it is agreed-ed we are not going to block this patch for the sake of "Dynamic LNet Config", you know as much about it as I do. It sounds like just using 'lctl add_route' it is not possible to accomplish what is needed since modprobe install hooks don't work.

            Sorry this patch is not ready to go in as is. The file handling is very best case scenario: at a minimum we need to be sure the the file is not larger than the fixed size, introducing code that might only read SOME of the data not good. I am sure this patch works in the specific usage it was designed for but I don't think it is asking a lot to make it a little more generally safe as it could be in the code base for awhile.

            I would also hope that we all want to understand what is/will be possible, ie what are the new maximum values. How many routes/ ip2nets can be added with this patch in place? So far 10k/10k has been stated but no indication has been given about the filesize.

            keith Keith Mannthey (Inactive) added a comment - I think it is agreed-ed we are not going to block this patch for the sake of "Dynamic LNet Config", you know as much about it as I do. It sounds like just using 'lctl add_route' it is not possible to accomplish what is needed since modprobe install hooks don't work. Sorry this patch is not ready to go in as is. The file handling is very best case scenario: at a minimum we need to be sure the the file is not larger than the fixed size, introducing code that might only read SOME of the data not good. I am sure this patch works in the specific usage it was designed for but I don't think it is asking a lot to make it a little more generally safe as it could be in the code base for awhile. I would also hope that we all want to understand what is/will be possible, ie what are the new maximum values. How many routes/ ip2nets can be added with this patch in place? So far 10k/10k has been stated but no indication has been given about the filesize.
            nic_henke@xyratex.com Nic Henke (Inactive) added a comment - - edited

            ip2nets: scripting here is not always possible, and frankly requires the duplication of code that is already in lnet.ko in the kernel.

            routes: we need the full set of routes available at mount time, for proper mount-on-boot for client nodes and automatic failover/reboot on server side. To prevent wrapping or rewriting mount.lustre, this needs to be handled in the module load chain. We had tried using modprobe install hooks for this, but the end result was racy and error prone. You can't hook into lnet.ko, as that requires the loading of all the LNDs before lctl --net XYX will work, etc.

            Given that we have a fairly low-impact patch that is already done and ready to go in, why would we hold it off for some unknown feature like Dynamic LNet Config ? If DLC fixes this in the future, we could replace it - but right now only one is coded and viable.

            Since Isaac and Doug both referenced "Dynamic LNet Config" - can you share the design? It is very hard to contribute code when we don't know the plans for the future.

            nic_henke@xyratex.com Nic Henke (Inactive) added a comment - - edited ip2nets: scripting here is not always possible, and frankly requires the duplication of code that is already in lnet.ko in the kernel. routes: we need the full set of routes available at mount time, for proper mount-on-boot for client nodes and automatic failover/reboot on server side. To prevent wrapping or rewriting mount.lustre, this needs to be handled in the module load chain. We had tried using modprobe install hooks for this, but the end result was racy and error prone. You can't hook into lnet.ko, as that requires the loading of all the LNDs before lctl --net XYX will work, etc. Given that we have a fairly low-impact patch that is already done and ready to go in, why would we hold it off for some unknown feature like Dynamic LNet Config ? If DLC fixes this in the future, we could replace it - but right now only one is coded and viable. Since Isaac and Doug both referenced "Dynamic LNet Config" - can you share the design? It is very hard to contribute code when we don't know the plans for the future.

            People

              ashehata Amir Shehata (Inactive)
              aboyko Alexander Boyko
              Votes:
              0 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: