Details
-
Bug
-
Resolution: Duplicate
-
Major
-
None
-
Lustre 2.11.0, Lustre 2.12.0
-
None
-
RHEL7.5 with default IB stack. Both clients and server back end running the default IB stack. All running lustre 2.11 but this affects 2.12 as well since LNet has not changed between version.s
-
3
-
9223372036854775807
Description
I found that I was unable to read the ones that were 5K in size. This occurs on the f2-util01 host that has a 2.11 client. It spews errors and eventually times out. To test if it was a 2.7 client <-> 2.11 client issue, I created another 5K text file from f2-util01. I found that I was unable to read that one as well. Identify what the issue was and pointed out that it was showing this in the logs:
—
LNetError: 106211:0:(o2iblnd_cb.c:571:kiblnd_fmr_map_tx()) Using FastReg with no GAPS support, but tx has gaps. Try setting use_fastreg_gaps to 1
LNetError: 106211:0:(o2iblnd_cb.c:571:kiblnd_fmr_map_tx()) Skipped 477 previous similar messages
LNetError: 106211:0:(o2iblnd_cb.c:1884:kiblnd_recv()) Can't setup PUT sink for 10.10.33.32@o2ib2: -93
LNetError: 106211:0:(o2iblnd_cb.c:1884:kiblnd_recv()) Skipped 477 previous similar messages
—
I then set that option in the ko2iblnd module parameters and brought everything back up but it was still encountering that issue.
If, however, I dd either of the files to /dev/null first I am then able to read them normally. This was the case for both user_fastreg_gaps set to 1 or 0.
