[LU-1201] Lustre crypto hash cleanup

• Type: Improvement
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.3.0
• Assignee: WC Triage
• Reporter: Alexander Boyko

Description

Lustre use crypto hashes algo in two ways: PTLRPC (ptlrpc/sec_bulk.c), OST(ost/ost_handler.c)/OSC (osc/osc_request.c).
OST,OSC use crc32, crc32c, adler for checksumming (compute_checksum() function)
PTLRPC uses crc32, adler, md5, sha1-512 ( kernel crypto api for all, excluding crc32 adler)
All subsystems go through bulk pages and update checksum.
To resolve conflicts with different implementation of checksumming, a new crypto hash interface is needed at libcfs. It should use kernel crypto api for hash calculation for kernel modules, and lustre implementation for user mode. Previus checksum calculation should be changed to the new libcfs crypto hash api. And adding new hash algo would be a simple task.

Issue Links

• is related to: LU-744 Single client's performance degradation on 2.1

Comments

[Nathan Rutman]

Xyratex-bug-id: MRP-337
Xyratex-bug-id: MRP-471

[Peter Jones]

Landed for 2.3

[Alexander Boyko]

new review req at http://review.whamcloud.com/#change,3098
two lines code was changed. Build succcessful.

[Peter Jones]

patch reverted as breaks build for external OFED

[Peter Jones]

Landed for 2.3

[Shuichi Ihara (Inactive)]

Hi Jay,

Yes, I didn't write/read the larger file size than client's memory size to avoid LU-744 issue.

I opened new ticket on LU-1408 for single client's performance regression on 2.2. Also, during my testing for LU-1408, I saw another big performance difference between the latest b2_1 branch and 2.1.2RC0 tag. This is filed on LU-1413 as another regression.

So, please move foward to on LU-744, LU-1408 and LU-1413 for single client performance regression and different behaviors on various lustre version.

[Jinshan Xiong (Inactive)]

Hi Ihara, in LU-744 you saw 2.2 clients performed better than 2.1, however, here you have seen the opposite. Is this only because the file size is less than memory size?

[Shuichi Ihara (Inactive)]

No big performance differences between 2.2 and 2.2/w this patches, but still lower than 2.1.2.

LU-744, I've filed before, but I think this is another regression compared to what we saw here.
As far as I tested on 2.2, the performance goes down when write/read size exceed client's memory size. This is LU-744 and same problem happens on 2.1.x.

In order to avoid this regression for this checksum comparison among on 1.8.7, 2.1.2 and 2.2, I used file size < client's memory size.

Anyway, I don't think this regression is related to LU-1201, I will open the new ticket.

[Alexander Boyko]

Ihara, have you checksumming benchmark results without this patch(for the same hw and configuration)?

[Andreas Dilger]

It looks like there is already a bug LU-744 for tracking the 2.x performance degradation.