Whamcloud Community JIRA

Whamcloud Community JIRA https://jira.whamcloud.com This file is an XML representation of an issue en-us

9.4.14

940014

05-12-2023

[LU-2488] crc t10 dif pclmulqdq implementation https://jira.whamcloud.com/browse/LU-2488 Lustre LU-2488

crc t10 dif pclmulqdq implementation

New Feature Minor Closed Incomplete Keith Mannthey Alexander Boyko patch Thu, 13 Dec 2012 06:08:15 +0000 Thu, 26 Sep 2013 05:50:50 +0000 Thu, 26 Sep 2013 05:50:50 +0000 0 8 This patch adds crc t10 dif pclmulqdq implementation to libcfs. Result from t10 unit test: <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>Lustre: t10crc UT Lustre: Checking block size 512 loops 4096 ... Lustre: Speed: libcfs 1351MB/s kernel 259MB/s Lustre: PASS Lustre: Checking block size 4096 loops 4096 ... Lustre: Speed: libcfs 2111MB/s kernel 251MB/s Lustre: PASS </pre> </div></div> kernel - linux kernel table implementation. libcfs - pclmulqdq implementation. <a href="http://review.whamcloud.com/4822" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/4822</a> Presumably the intent of this is to add support for a t10 DIF mode for bulk RPCs? It would be good to include some background in the bug about how this code is planned to be used. This code will be used by client to calculate crc t10 dif, and provide this checksum to server with t10 dif/dix capable storage. So presumably you are going to add a new T10 bulk data checksum mode to the BRW protocol? I recall seeing some discussion to this effect, but I don't recall if there was a conclusion about what method would be used to send all of the T10 information in the RPC? This would increase the size of the BRW RPC by 256 * 8 bytes = 2kB for a 1MB RPC, and by 8kB for a 4MB RPC, which is pretty significant. Do you have any kind of HLD for the T10 DIF changes to the RPC? A good amount of kernels have T10 support at this point. Is the kernels T10 implementation not usable for your needs? Keith, if you are talking about crc T10, the kernel implementation is very slow and take much cpu resources. Andreas, you are right size of BRW RPC will be increased by sizeof(crc) * number of sectors for the bulk. Xyratex <a href="http://jira-nss.xy01.xyratex.com:8080/browse/MRP-511" class="external-link" target="_blank" rel="nofollow noopener">MRP-511</a> While this particular patch provides hardware support for the T10-DIF CRC algorithm for general use in the libcfs library, and is not concerned with any actual T10-DIF/DIX usage, I understand the reluctance to use apparently unnecessary code. This is the first patch in a series designed to provide end-to-end T10-DIX support for Lustre clients; I will open another, high-level ticket describing it. <a href="https://jira.whamcloud.com/browse/LU-2584" title="End-to-End Data Integrity with T10" class="issue-link" data-issue-key="LU-2584"><del>LU-2584</del></a> is parent ticket. It is possible to accelerate the Kernels T10 rather than re-implement the T10 framework in Lustre? Do we know why the T10 is not ASM accelerated in the kernel or the status of mainline in this regard? Keith, it will take time. The current kernel version is 3.9, but Lustre base on 2.6.32-279. May be I need to create and push t10 code to kernel in parallel. The T10 for Lustre has specific restrictions - only 512 and 4096 sector size and aligned data, using this the code is more productive than the general implementation. It seem the feature freeze for 2.5 is the end of July. It would be good to refresh this patch if the T10 code is targeted for the 2.5 release. Pleas, close this issue. This feature is not needed without main T10 code, I don`t see any progress on T10, and I will not spend time to update this patch. Close bug per request. Related LU-2584 Development Rank 1|hzvdsf: Rank (Obsolete) 5841