[LU-1517] no retry for the bulk operation

• Type: Bug
• Resolution: Fixed
• Priority: Critical
• Fix Version/s: Lustre 2.4.0, Lustre 2.1.4, Lustre 1.8.9
• Affects Version/s: Lustre 1.8.8
• Labels: patch
• Severity: 3
• Assignee: Keith Mannthey (Inactive)
• Reporter: Alexander Boyko

Description

lnd_cb.c:558:kgnilnd_setup_phys_buffer()) failed to allocate tx_phys
[2012-04-07 02:08:24][c5-0c0s5n2]LNet: 29099:0:(gnilnd_cb.c:1068:kgnilnd_tx_done()) $$ error -12 on tx 0xffff88000fe06b40-><?> id 0/0 state GNILND_TX_ALLOCD age 17481575s  msg@0xffff88000fe06bc0 m/v/ty/ck/pck/pl b00fbabe/8/3/0/78db/0 x0:GNILND_MSG_PUT_REQ
[2012-04-07 02:08:24][c5-0c0s5n2]LustreError: 29099:0:(events.c:198:client_bulk_callback()) event type 0, status -5, desc ffff880627c24000

The error is detected on both client and server; the server expects the client to retry but it doesn't. In the mean time, the OSS issues a lock callback to the client, but the client does not respond because it is waiting for the I/O to complete. Eventually the OSS evicts the client. Lustre does not retry the bulk op when it detects the error.

Issue Links

• Trackbacks: Lustre 1.8.x known issues tracker While testing against Lustre b18 branch, we would hit known bugs which were already reported in Lustre Bugzilla https://bugzilla.lustre.org/. In order to move away from relying on Bugzilla, we would create a JIRA

Comments

[Cory Spitz]

Johann, for master, have you seen LU-901 and change #4092? Or does it only partially address the fault?

[Johann Lombardi (Inactive)]

I'm fine with landing the patch on b1_8. That said, it seems that master suffers from the same issue, right?
If so, it would be great to push a patch against master.
Thanks in advance!

[Cory Spitz]

Understood. Trigerring this bug results in client eviction though. Just FYI.

[Keith Mannthey (Inactive)]

I just wanted to say that 1.8 is in more of a maintenance mode at this point. The patch looks fine but very few things are landing in 1.8 right now.

[Keith Mannthey (Inactive)]

The patch has been reviewed and is awaiting final merger.

[Cory Spitz]

Cray has been using this patch and it is effective.

[Colin Faber [X] (Inactive)]

Hi,

Have WC/Intel had time to review our proposed patch yet?

-cf

[Alexander Boyko]

Review request http://review.whamcloud.com/3102