[LU-19905] FLR-EC: import ISA-L assembly versions to optimize EC calculations

• Type: Improvement
• Resolution: Unresolved
• Priority: Medium
• Affects Version/s: Lustre 2.18.0
• Labels: FLR-EC FLR-EC-RO FLR-EC-backlog
• Severity: 3
• Epic Link: FLR improvements tracker
• Assignee: Ronnie Sahlberg
• Reporter: Andreas Dilger

Description

The lustre/ec/ec_base.[ch] code implements the baseline functionality for Erasure Coding, imported from the ISA-L library erasusre_code tree (LU-12189, LU-19465).

This allows reasonable functional testing of the EC feature, but does not represent the optimized EC calculations that are possible. There are a wide variety of optimized assembly versions of the EC calculations for different CPU architectures (x86_64, aarch64, riscv) and instruction sets (SSE, AVX, AVX2, AVX512, etc) that could be used.

It makes sense as the FLR-EC project draws to a conclusion on implementing the core functionality to look toward performance optimizations for this code.

The optimized ISA-L EC implementations could be utilized directly by "lfs mirror extend/resync" in userspace with relatively little effort, likely some changes to the build chain to use the NASM compiler to build the code (maybe x86_64 only?), and selecting the right variant at runtime based on available CPU instructions.

In the kernel, care must also be taken to properly save and restore the extended register set that the optimized calculations are using, so that IO threads recovering data via EC during reads do not clobber userspace applications that are also using these registers.

Issue Links

• duplicates: LU-20016 FLR-EC: Tool to benchmark EC encode/decode speed
• is duplicated by: LU-19964 EC: integrate optimized assembler ISA-L routines
• is related to: LU-12669 FLR-EC: recover data from parity code
• is related to: LU-19465 FLR-EC: Move the erasue-coding code from the module and into a separate library
• is related to: LU-12189 FLR-EC: import ISA-L library in Lustre build
• is related to: LU-19074 FLR-EC: optimize ec_encode_data with lookup tables to replace gf_mul