[LU-8223] De-Noise LNet neterr logs so they can be ON by default

• Type: Improvement
• Resolution: Unresolved
• Priority: Major
• Labels: lnet
• Severity: 3
• Assignee: Amir Shehata (Inactive)
• Reporter: Doug Oucharek (Inactive)

Description

LNet's neterr logs are turned off by default. I have been told this is due to the fact that they are very noisy. Logically, if logs are happening very frequently, they are not really errors then but normal operations. If they were errors, we should be fixing them.

The big problem here is that when a networking field issue happens, we have little to nothing in the logs to go on. Debugging requires that the problem be easy to reproduce with neterr turned on (not usually the case for production errors), or becomes a discipline of the mind (i.e. guesswork).

This ticket is for cleaning up the neterr logs to be true errors so we can have neterr logs on be default.

Issue Links

• is duplicated by: LU-2418 Add Way to Detect Dropped Packets on Production Systems
• is related to: LU-8980 Add tracepoint support to Lustre

Comments

[Doug Oucharek (Inactive)]

Ok, I can delay this patch until LU-8980 lands and then update it with any additional changes required.

[James A Simmons]

I already started the tracepoint work. See LU-8980.

[Doug Oucharek (Inactive)]

This is really just a 2.10 thing.  I don't expect anyone will want this backported.

When will the migration to tracepoint be taking place?

[James A Simmons]

Do you need to backport this to earlier lustre version or is this a lustre 2.10 thing. The reason I ask is that the lustre debugging code is being migrated to tracepoint. For tracepoint this can be addressed but it wouldn't be back portable.

[Gerrit Updater]

Doug Oucharek ([e-mail]) uploaded a new patch: http://review.whamcloud.com/20769
Subject: LU-8223 lnet: Fix use of NETERR logging
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 155086c0edeb1b938994f1aba7c243ad4d57133d

[Doug Oucharek (Inactive)]

I actually consider this a bug and not an improvement. Neterrors should never have gotten into this unusable state in the first place. Outside of development, they are useless because they are off.