[LU-2335] add scripts to extract Lustre client kernel code from Lustre tree Created: 15/Nov/12 Updated: 12/Aug/15 Resolved: 12/Aug/15 |
|
| Status: | Resolved |
| Project: | Lustre |
| Component/s: | None |
| Affects Version/s: | Lustre 2.5.0 |
| Fix Version/s: | None |
| Type: | Improvement | Priority: | Minor |
| Reporter: | Peng Tao | Assignee: | Andreas Dilger |
| Resolution: | Won't Fix | Votes: | 0 |
| Labels: | emc, patch | ||
| Rank (Obsolete): | 5567 |
| Description |
|
To get a relatively clean version of lustre client kernel code base, we can use a script to extract it from Lustre tree, and use external tools (like coan, http://coan2.sourceforge.net/index.php) to clean up macros generated by autoconfig. Through the method, we can see from time to time how clean the code has got. And when it reaches to certain acceptable level, we can request to merge it in upstream staging tree. |
| Comments |
| Comment by Peng Tao [ 21/Nov/12 ] |
|
Three preparing patches are submitted: They are necessary to clean up client code a bit to more easily fit into kernel kconfig/kbuild system. I will submit the cleanup scripts soon. |
| Comment by Peter Jones [ 26/Feb/13 ] |
|
Andreas It looks like you have already been reviewing these patches Peter |
| Comment by Peng Tao [ 14/Mar/13 ] |
|
Two patches to generate and copy lustre client-only code into a kernel tree, along with necessary kbuild/kconfig changes: For now, they work on top of the latest master plus the 3.8 kernel patches. With them, one can build lustre client code as external modules via: Or copy lustre client code into kernel tree via: And I admit that the scripts and kbuild makefiles will likely need to be modified in future when lustre changes its makefile etc. But it is good to have them maintained in one place. |
| Comment by Peng Tao [ 29/Mar/13 ] |
|
Added one more |
| Comment by Keith Mannthey (Inactive) [ 22/Jul/13 ] |
|
Ok I got a chance to sit down a look at http://review.whamcloud.com/5713 this weekend. It became quickly apparent the real issues are in http://review.whamcloud.com/5714 and the patches are dependant. The larger issue is how to manage Makefile and the list of files that the client truly needs. A 3rd-ish set of makefiles seem very easy to subject to bit-rot. I would like to take a look at this from the view of having the Lustre-client in the staging tree. Automating the files that we want to diff is a good thing. It would seem we should be able to script this out of the Lustre-git tree build/make system. I am open to investigating this. I don't feel strongly about the need to install a client in a Linux-tree (kconfig and Makefiles) as a client is present. I understand how useful it as before the client landed as an integration tool but I am not sure if we need it anymore. Open questions are: 1. Do we really want to auto generate makefiles to compare against the Linux-tree? |
| Comment by Peng Tao [ 22/Jul/13 ] |
|
One good thing with the extra build system is that I was able to actually build/test lustre client before copying it into a kernel tree. I'm OK to drop the makefile/Kconfig/copy side of scripts, to ease the burden of having to maintaining separate build system. Given that client is already in upstream kernel, it is unlikely they will be used again. And I agree with you that now we only need the ability to compare Lustre master with upstream client. IMO one possibly most straightforward way is to maintain a list of files used in upstream client, and have a script running to diff each of them with corresponding file in Lustre master. Of course the files in master need to be cleaned up with coan2 before diffing. |
| Comment by Keith Mannthey (Inactive) [ 27/Jul/13 ] |
|
I had some time to sit down and work on this and I wanted to give a quick update. I am perusing generating the list of files to care about automatically and I have a found a sane way. The base idea is to use configure to generate client only Lustre makefiles like this: ./configure --with-linux=<kernel> --disable-server --disable-tests --disable-doc --disable-utils The makefiles generated by configure map well to the Mainline makefiles in functionality and the makefiles give the needed data to gather the list of files. I am about 75% of the way through a script that pulls all the *.chS files from the Lustre makefiles. Once this is done I will pass the files through coan and diff them against the mainline files. |
| Comment by Andreas Dilger [ 12/Aug/15 ] |
|
The upstream kernel client code has diverged too much for this to be useful anymore. |