[LU-1434] Update Wireshark support for LNet and Lustre

• Type: New Feature
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.4.0
• Affects Version/s: Lustre 2.4.0
• Labels: LNet tools,
• Assignee: Doug Oucharek (Inactive)
• Reporter: Doug Oucharek (Inactive)

Description

There was work previously done to create two plugins for Wireshark to decode LNet and Lustre headers. That work is started in lustre/contrib as the files packet-lnet.c and packet-lustre.c.

Attempting to build these plugins revealed that Wireshark's API for dissectors has changed such they no longer build/run. Also, the Lustre plugin had a few syntax errors so I am not sure if it ever worked.

This feature request is to do the following upgrades to Wireshark support:

1- Get both plugins building under the latest 1.4.x version of Wireshark (will be backwards compatible with 1.2.x).
2- Move them to a wireshark subdirectory of lustre/contrib to isolate them better
3- Create a Makefile to build and install the plugins (one does not exist today).
4- If possible, automate the building of the plugins if the build system has the Wireshark source package installed.

A Wiki page should also be created to provide instruction on how to use Wireshark to look at LNet and Lustre packets.

Issue Links

• is related to: LUDOC-91 Update Wireshark support for LNet and Lustre Doc Changes
• Trackbacks: Project Ideas Information for Developers Working on Lustre will quickly introduce a developer to the always interesting and impressive world of HPC, where the systems are the largest in the world,...

Comments

[Jodi Levi (Inactive)]

http://review.whamcloud.com/#change,5520

[Nathaniel Clark]

Further lustre parsing enhancements
http://review.whamcloud.com/5155

[Nathaniel Clark]

Update to Makefile to compile code in the directory and install depending on user id:
http://review.whamcloud.com/4851

Updates to lustre parsing code that I needed to decode lustre packets while investigating LU-2277:
http://review.whamcloud.com/4852

[Doug Oucharek (Inactive)]

I've created a new patch with only the changes needed to build and run the wireshark LNet and Lustre decoders under wireshark-1.6.8.

This is under change: http://review.whamcloud.com/4393

There are known issues with the Lustre decoding which will need to be ironed out over time. Future improvements/fixes will be done under new patches.

[Andreas Dilger]

Do we really need to convert every line of code to the new format? In other words, is every line modified just to convert it to proper coding style, or is in needed due to changes in wireshark? Changing every line in the file in one patch will make it messy to inspect.

Is it possible to land the LNet decoding first, then do the Lustre decoding in a separate patch to reduce the patch size? This would also allow landing the working LNet code first while the Lustre code is being fixed.

As to the Lustre decoding, the RQF structures should describe all of the RPC formats in use today. I agree that getting all of the structures decoded would be great, but given that the current code is completely broken I wonder if 80% decoding is better than zero, and we can work on fixing this incrementally?

[Doug Oucharek (Inactive)]

Here is the current state of Wireshark support:

• All changes I have made to date are on a Git branch on my system ready to be pushed into Gerrit. I was holding off until we are past 2.3 being the master. We are beyond that point so I should be pushing soon.
• I have done a Makefile for manual building so people can build it for themselves when they want.
• There is a README in the directory explaining how to build and install the new Wireshark plugins.

There are, however, some problems:

1- The decoding for Lustre is HUGE. The one file is over 12K lines of code. The file for LNet decoding is just over 1K lines. This is going to be a very long and painful code inspection.
2- I updated all code for our coding guidelines so it passes our Git hooks. As such, Gerrit will flag all lines as changed. That means about 13.5K lines will go into Gerrit in one patch.
3- I've been able to verify the LNet decoding, but the Lustre decoding does have some errors. I'm sure it is over 80% correct, but there is a need for making corrections. I don't have the background to do this (nor is there a protocol map I can rely on for this). I need the help of the Lustre experts to verify the Lustre decoding and make the necessary changes.

So, what I'd like to do is this:

1- Submit what I have so far to Gerrit to start the review/correction process.
2- Have some of the experts at the Lustre protocol install the patch, build Wireshark support, and do captures to verify that the decoding is correct. When errors are found, send corrections to me and I'll accumulate them into a new updated patch.
3- When the decoding is correct and happy, do a final patch to automate the build process tying this into our build system.

Thoughts?

[Andreas Dilger]

Doug, what is the current stat of this code? It would probably be useful to submit these patches to Gerrit for inspection and landing.

Ideally we would hook the files into the Lustre build system and generate (possibly a separate RPM) binaries for Wireshark plugins for the kernels we support. This would probably need the build nodes to install the "wireshark devel" packages, and have a configure check to detect whether the headers/libraries are installed.