[LU-6245] Untangle userland and kernel space support for libcfs Created: 13/Feb/15 Updated: 02/Mar/18 Resolved: 13/Aug/17 |
|
| Status: | Resolved |
| Project: | Lustre |
| Component/s: | None |
| Affects Version/s: | Lustre 2.8.0 |
| Fix Version/s: | Lustre 2.11.0 |
| Type: | Improvement | Priority: | Major |
| Reporter: | James A Simmons | Assignee: | James A Simmons |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | patch | ||
| Issue Links: |
|
||||||||||||||||||||||||||||||||||||
| Rank (Obsolete): | 17491 | ||||||||||||||||||||||||||||||||||||
| Description |
|
In times past libcfs was built for user space and kernel space. Today we no longer need to build a user land version of libcfs. This ticket will track the work of removing the libcfs abstraction to support both user space and kernel space. In the end we should have a minimal libcfs that only supports kernel space. |
| Comments |
| Comment by Gerrit Updater [ 13/Feb/15 ] |
|
James Simmons (uja.ornl@gmail.com) uploaded a new patch: http://review.whamcloud.com/13759 |
| Comment by John Hammond [ 13/Feb/15 ] |
|
James, Of course there are several ways to go about this but in my analysis the next step would be to sort out the LNet headers. Most are not needed for userspace and should be amended to generate an error if _KERNEL_ is not defined. Others like lib-types.h contain a mixture of wire types, definitions needed by kernel and user space, and definitions only needed by kernel space. This header and similar ones "hold on to" some of the libcfs cruft. |
| Comment by Gerrit Updater [ 13/Feb/15 ] |
|
James Simmons (uja.ornl@gmail.com) uploaded a new patch: http://review.whamcloud.com/13760 |
| Comment by James A Simmons [ 13/Feb/15 ] |
|
I have some patches that cleanup the LNET headers. The confusing thing with those headers is that user land definitions are located in a couple of headers. We need to talk to the LNET guys to discuss what goes where. |
| Comment by Gerrit Updater [ 17/Feb/15 ] |
|
James Simmons (uja.ornl@gmail.com) uploaded a new patch: http://review.whamcloud.com/13783 |
| Comment by Gerrit Updater [ 18/Feb/15 ] |
|
James Simmons (uja.ornl@gmail.com) uploaded a new patch: http://review.whamcloud.com/13792 |
| Comment by Gerrit Updater [ 18/Feb/15 ] |
|
James Simmons (uja.ornl@gmail.com) uploaded a new patch: http://review.whamcloud.com/13793 |
| Comment by Gerrit Updater [ 20/Feb/15 ] |
|
James Simmons (uja.ornl@gmail.com) uploaded a new patch: http://review.whamcloud.com/13835 |
| Comment by Gerrit Updater [ 23/Feb/15 ] |
|
James Simmons (uja.ornl@gmail.com) uploaded a new patch: http://review.whamcloud.com/13841 |
| Comment by Gerrit Updater [ 25/Feb/15 ] |
|
James Simmons (uja.ornl@gmail.com) uploaded a new patch: http://review.whamcloud.com/13874 |
| Comment by Gerrit Updater [ 26/Feb/15 ] |
|
James Simmons (uja.ornl@gmail.com) uploaded a new patch: http://review.whamcloud.com/13899 |
| Comment by Gerrit Updater [ 27/Feb/15 ] |
|
James Simmons (uja.ornl@gmail.com) uploaded a new patch: http://review.whamcloud.com/13917 |
| Comment by Gerrit Updater [ 25/Mar/15 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/13760/ |
| Comment by Gerrit Updater [ 25/Mar/15 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/13783/ |
| Comment by Gerrit Updater [ 25/Mar/15 ] |
|
John L. Hammond (john.hammond@intel.com) uploaded a new patch: http://review.whamcloud.com/14180 |
| Comment by Gerrit Updater [ 25/Mar/15 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/13792/ |
| Comment by Gerrit Updater [ 25/Mar/15 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/13793/ |
| Comment by Gerrit Updater [ 30/Mar/15 ] |
|
frank zago (fzago@cray.com) uploaded a new patch: http://review.whamcloud.com/14270 |
| Comment by Gerrit Updater [ 06/Apr/15 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/13835/ |
| Comment by Gerrit Updater [ 20/Apr/15 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/14270/ |
| Comment by Gerrit Updater [ 21/Apr/15 ] |
|
James Simmons (uja.ornl@yahoo.com) uploaded a new patch: http://review.whamcloud.com/14512 |
| Comment by Gerrit Updater [ 27/Apr/15 ] |
|
James Simmons (uja.ornl@yahoo.com) uploaded a new patch: http://review.whamcloud.com/14624 |
| Comment by Gerrit Updater [ 28/Apr/15 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/14180/ |
| Comment by Gerrit Updater [ 28/Apr/15 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/13841/ |
| Comment by Gerrit Updater [ 28/Apr/15 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/13874/ |
| Comment by Gerrit Updater [ 28/Apr/15 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/13899/ |
| Comment by Gerrit Updater [ 28/Apr/15 ] |
|
James Simmons (uja.ornl@yahoo.com) uploaded a new patch: http://review.whamcloud.com/14629 |
| Comment by Gerrit Updater [ 07/May/15 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/14629/ |
| Comment by James A Simmons [ 15/May/15 ] |
|
I have started the work on removing the random wrappers. Most of the cleanups are straight forward except for cfs_srand which is used to add to the entropy pool. Now starting in kernel 3.6 we have add_device_randomness which is only missing for RHEL6 kernels. The question is do we even need it. Should we just remove cfs_srand? |
| Comment by Jian Yu [ 20/May/15 ] |
|
Hi Andreas, |
| Comment by Gerrit Updater [ 29/May/15 ] |
|
James Simmons (uja.ornl@yahoo.com) uploaded a new patch: http://review.whamcloud.com/15083 |
| Comment by Gerrit Updater [ 30/May/15 ] |
|
James Simmons (uja.ornl@yahoo.com) uploaded a new patch: http://review.whamcloud.com/15085 |
| Comment by Gerrit Updater [ 30/May/15 ] |
|
James Simmons (uja.ornl@yahoo.com) uploaded a new patch: http://review.whamcloud.com/15086 |
| Comment by Gerrit Updater [ 05/Jun/15 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/15083/ |
| Comment by Gerrit Updater [ 10/Jun/15 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/14624/ |
| Comment by Gerrit Updater [ 10/Jun/15 ] |
|
James Simmons (uja.ornl@yahoo.com) uploaded a new patch: http://review.whamcloud.com/15197 |
| Comment by Gerrit Updater [ 10/Jun/15 ] |
|
James Simmons (uja.ornl@yahoo.com) uploaded a new patch: http://review.whamcloud.com/15200 |
| Comment by Gerrit Updater [ 10/Jun/15 ] |
|
James Simmons (uja.ornl@yahoo.com) uploaded a new patch: http://review.whamcloud.com/15208 |
| Comment by Gerrit Updater [ 11/Jun/15 ] |
|
James Simmons (uja.ornl@yahoo.com) uploaded a new patch: http://review.whamcloud.com/15217 |
| Comment by James A Simmons [ 11/Jun/15 ] |
|
The last two patches remove libcfs.h usage from the user land utilities. The libcfs.h header is still left in user land code via the lustre include headers. That will next on the list for cleanup and after that libcfs.h user land code can be removed. |
| Comment by Andreas Dilger [ 12/Jun/15 ] |
|
James, to answer your question on cfs_rand() and cfs_srand() and cfs_get_random_bytes() - these were very important for the Cray Catamount machine, which had tens of thousands of cores booting simultaneously with a single-threaded interrupt-free kernel. There was not any kind of h/w random number generator to use. That sometimes resulted in the identical random number entropy state across multiple nodes, which would cause problems with e.g. client UUIDs being the same on two clients, as well as lesser issues with initial lustre_handle, and initial XID value and other items that should be unique (if at least for debugging purposes). The solution was to include the NID into the random number pool, which was one of the very few things that was unique about each node. I think that if it is possible to call add_device_randomness() to include the NID into the entropy pool that would allow us to use the kernel PRNG with some assurance that the number sequences will be different across nodes. I suspect that for existing RHEL 6 installations that the chance of having the identical state across many nodes is low, most machines have h/w RNG these days via the CPU, and for the few places it matters we can still fold in the NID(s) directly. If the add_device_randomness() API is available with newer kernels then it should be used to include the NID(s) into the RNG pool, to avoid potential problems in the future if a Catamount-like system is revived again. These "serious" places all use cfs_get_random_bytes(). The cfs_rand() users in lod_qos.c and elsewhere doesn't need this special treatment, and obd_alloc_fail() could be removed entirely (I have an in-progress patch for that, if you want to take it over). One potential option is to have the RHEL6 add_device_randomness() wrapper hash the input bytes into a 64-bit mask that we can XOR into the values returned by the kernel's get_random_bytes() in cfs_get_random_bytes() (which can be dropped entirely once we drop RHEL6 support). Likewise ll_generate_random_uuid() could also be converted to generate_random_uuid() at that time. |
| Comment by Gerrit Updater [ 19/Jun/15 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/15085/ |
| Comment by Gerrit Updater [ 22/Jun/15 ] |
|
James Simmons (uja.ornl@yahoo.com) uploaded a new patch: http://review.whamcloud.com/15368 |
| Comment by Gerrit Updater [ 08/Jul/15 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/15086/ |
| Comment by Gerrit Updater [ 08/Jul/15 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/15208/ |
| Comment by Gerrit Updater [ 08/Jul/15 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/15217/ |
| Comment by Gerrit Updater [ 10/Jul/15 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/15368/ |
| Comment by Doug Oucharek (Inactive) [ 13/Jul/15 ] |
|
James: How is this ticket progressing? With all the patch activity, I'm not sure where it stands. Doug |
| Comment by Gerrit Updater [ 19/Jul/15 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/15197/ |
| Comment by James A Simmons [ 27/Jul/15 ] |
|
With the landing of the last patch the libcfs module code no longer has any user land code in it. The libcfs headers are left to cleanup. |
| Comment by Gerrit Updater [ 07/Aug/15 ] |
|
James Simmons (uja.ornl@yahoo.com) uploaded a new patch: http://review.whamcloud.com/15909 |
| Comment by Gerrit Updater [ 07/Aug/15 ] |
|
James Simmons (uja.ornl@yahoo.com) uploaded a new patch: http://review.whamcloud.com/15910 |
| Comment by Gerrit Updater [ 07/Aug/15 ] |
|
James Simmons (uja.ornl@yahoo.com) uploaded a new patch: http://review.whamcloud.com/15913 |
| Comment by Gerrit Updater [ 18/Aug/15 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/13917/ |
| Comment by Gerrit Updater [ 18/Aug/15 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/15909/ |
| Comment by Gerrit Updater [ 18/Aug/15 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/15913/ |
| Comment by Andreas Dilger [ 25/Aug/15 ] |
|
James, it looks like http://review.whamcloud.com/15910 is the last unlanded patch for this ticket. Are there more cleanup patches needed for this? If yes, can you please file a separate bug for 2.9.0 and we can close this one as fixed for 2.8.0, which simplifies tracking. |
| Comment by James A Simmons [ 25/Aug/15 ] |
|
Not currently but many more patches are needed. Wouldn't it be better for this ticket to be independent of any version. It is going to be a while before this is finished. |
| Comment by Gerrit Updater [ 29/Aug/15 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/15910/ |
| Comment by Peter Jones [ 31/Aug/15 ] |
|
I agree with James - removed fix version |
| Comment by Gerrit Updater [ 19/Oct/15 ] |
|
James Simmons (uja.ornl@yahoo.com) uploaded a new patch: http://review.whamcloud.com/16879 |
| Comment by Gerrit Updater [ 22/Oct/15 ] |
|
James Simmons (uja.ornl@yahoo.com) uploaded a new patch: http://review.whamcloud.com/16914 |
| Comment by Gerrit Updater [ 22/Oct/15 ] |
|
James Simmons (uja.ornl@yahoo.com) uploaded a new patch: http://review.whamcloud.com/16916 |
| Comment by Gerrit Updater [ 15/Nov/15 ] |
|
James Simmons (uja.ornl@yahoo.com) uploaded a new patch: http://review.whamcloud.com/17202 |
| Comment by Gerrit Updater [ 27/Nov/15 ] |
|
James Simmons (uja.ornl@yahoo.com) uploaded a new patch: http://review.whamcloud.com/17375 |
| Comment by Gerrit Updater [ 27/Nov/15 ] |
|
James Simmons (uja.ornl@yahoo.com) uploaded a new patch: http://review.whamcloud.com/17376 |
| Comment by Gerrit Updater [ 16/Dec/15 ] |
|
James Simmons (uja.ornl@yahoo.com) uploaded a new patch: http://review.whamcloud.com/17643 |
| Comment by Gerrit Updater [ 02/Feb/16 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/17643/ |
| Comment by Gerrit Updater [ 02/Feb/16 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/16914/ |
| Comment by Gerrit Updater [ 14/Mar/16 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/16879/ |
| Comment by Gerrit Updater [ 14/Mar/16 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/17202/ |
| Comment by Gerrit Updater [ 16/Mar/16 ] |
|
James Simmons (uja.ornl@yahoo.com) uploaded a new patch: http://review.whamcloud.com/18964 |
| Comment by Gerrit Updater [ 23/Mar/16 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/17376/ |
| Comment by Gerrit Updater [ 23/Mar/16 ] |
|
Oleg Drokin (oleg.drokin@intel.com) uploaded a new patch: http://review.whamcloud.com/19094 |
| Comment by James A Simmons [ 23/Mar/16 ] |
|
Patch 17376 was done before RHEL7 picked up the time64_t changes in later kernels. That API is now in the latest RHEL7 kernels which caused the breakage we are seeing. |
| Comment by Gerrit Updater [ 23/Mar/16 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/19094/ |
| Comment by Gerrit Updater [ 23/Mar/16 ] |
|
James Simmons (uja.ornl@yahoo.com) uploaded a new patch: http://review.whamcloud.com/19113 |
| Comment by Gerrit Updater [ 06/Apr/16 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/18964/ |
| Comment by Gerrit Updater [ 07/Apr/16 ] |
|
James Simmons (uja.ornl@yahoo.com) uploaded a new patch: http://review.whamcloud.com/19383 |
| Comment by Gerrit Updater [ 11/Apr/16 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/19113/ |
| Comment by Gerrit Updater [ 11/Apr/16 ] |
|
James Simmons (uja.ornl@yahoo.com) uploaded a new patch: http://review.whamcloud.com/19459 |
| Comment by Gerrit Updater [ 11/Apr/16 ] |
|
James Simmons (uja.ornl@yahoo.com) uploaded a new patch: http://review.whamcloud.com/19460 |
| Comment by Gerrit Updater [ 21/Apr/16 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/19383/ |
| Comment by Gerrit Updater [ 04/May/16 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/19459/ |
| Comment by Gerrit Updater [ 04/May/16 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/19460/ |
| Comment by Gerrit Updater [ 13/May/16 ] |
|
James Simmons (uja.ornl@yahoo.com) uploaded a new patch: http://review.whamcloud.com/20204 |
| Comment by Gerrit Updater [ 02/Jun/16 ] |
|
James Simmons (uja.ornl@yahoo.com) uploaded a new patch: http://review.whamcloud.com/20590 |
| Comment by Gerrit Updater [ 14/Jun/16 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/20204/ |
| Comment by Gerrit Updater [ 07/Jul/16 ] |
|
James Simmons (uja.ornl@yahoo.com) uploaded a new patch: http://review.whamcloud.com/21187 |
| Comment by Gerrit Updater [ 22/Aug/16 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/20590/ |
| Comment by Gerrit Updater [ 25/Aug/16 ] |
|
James Simmons (uja.ornl@yahoo.com) uploaded a new patch: http://review.whamcloud.com/22138 |
| Comment by Gerrit Updater [ 25/Aug/16 ] |
|
James Simmons (uja.ornl@yahoo.com) uploaded a new patch: http://review.whamcloud.com/22139 |
| Comment by Gerrit Updater [ 02/Sep/16 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/15200/ |
| Comment by Gerrit Updater [ 02/Sep/16 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/22139/ |
| Comment by Gerrit Updater [ 07/Sep/16 ] |
|
James Simmons (uja.ornl@yahoo.com) uploaded a new patch: http://review.whamcloud.com/22370 |
| Comment by Gerrit Updater [ 07/Sep/16 ] |
|
James Simmons (uja.ornl@yahoo.com) uploaded a new patch: http://review.whamcloud.com/22371 |
| Comment by Gerrit Updater [ 22/Sep/16 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/22370/ |
| Comment by Gerrit Updater [ 22/Sep/16 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/22371/ |
| Comment by Gerrit Updater [ 26/Sep/16 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/16916/ |
| Comment by Gerrit Updater [ 26/Sep/16 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/21187/ |
| Comment by Gerrit Updater [ 26/Sep/16 ] |
|
James Simmons (uja.ornl@yahoo.com) uploaded a new patch: http://review.whamcloud.com/22732 |
| Comment by Gerrit Updater [ 30/Sep/16 ] |
|
James Simmons (uja.ornl@yahoo.com) uploaded a new patch: http://review.whamcloud.com/22857 |
| Comment by Gerrit Updater [ 01/Oct/16 ] |
|
James Simmons (uja.ornl@yahoo.com) uploaded a new patch: http://review.whamcloud.com/22866 |
| Comment by Gerrit Updater [ 08/Oct/16 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/22732/ |
| Comment by Gerrit Updater [ 08/Oct/16 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/22857/ |
| Comment by Gerrit Updater [ 12/Nov/16 ] |
|
James Simmons (uja.ornl@yahoo.com) uploaded a new patch: http://review.whamcloud.com/23733 |
| Comment by Gerrit Updater [ 01/Jan/17 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch https://review.whamcloud.com/22866/ |
| Comment by Gerrit Updater [ 02/Jan/17 ] |
|
James Simmons (uja.ornl@yahoo.com) uploaded a new patch: https://review.whamcloud.com/24575 |
| Comment by Gerrit Updater [ 24/Jan/17 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch https://review.whamcloud.com/23733/ |
| Comment by Gerrit Updater [ 31/Jan/17 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch https://review.whamcloud.com/22138/ |
| Comment by Gerrit Updater [ 07/Feb/17 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch https://review.whamcloud.com/24575/ |
| Comment by Gerrit Updater [ 18/Jul/17 ] |
|
James Simmons (uja.ornl@yahoo.com) uploaded a new patch: https://review.whamcloud.com/28089 |
| Comment by Gerrit Updater [ 05/Aug/17 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch https://review.whamcloud.com/28089/ |
| Comment by James A Simmons [ 05/Aug/17 ] |
|
I have just one patch left that I will submit shortly. Then this work will finish!!! |
| Comment by James A Simmons [ 07/Aug/17 ] |
|
The only work left is the test application it_test.c pulling in interval_tree.c from the lustre kernel source. This shouldn't be happening so I was looking at implementing the functions from interval_tree.c that is needed for it_test. Now when I looked to see how to test these changes I noticed it_test isn' even used anywere!!! Is it |
| Comment by Gerrit Updater [ 07/Aug/17 ] |
|
James Simmons (uja.ornl@yahoo.com) uploaded a new patch: https://review.whamcloud.com/28406 |
| Comment by James A Simmons [ 07/Aug/17 ] |
|
Last patch for this ticket push!!!! |
| Comment by Gerrit Updater [ 13/Aug/17 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch https://review.whamcloud.com/28406/ |
| Comment by James A Simmons [ 13/Aug/17 ] |
|
After 5 years and 10K+ lines of code changes this project is over. |