[LU-5599] Lustre Error: Impossible state: 4

• Type: Bug
• Resolution: Cannot Reproduce
• Priority: Major
• Affects Version/s: Lustre 2.4.3
• Environment: Fedora 19 x86_64 on Washington Pass nodes, 1GbE & FDR IB

Epic/Theme

Kernel Lustre-2.4 Panic

• Severity: 3
• Project: Fast Forward
• Assignee: Bob Glossman (Inactive)
• Reporter: Brian Bernel (Inactive)

Description

Lustre Version: 2.4.52
Kernel: patchless_client
Build: v2_4_52 0-gfdd4844-CHANGED-3.9.9-302.fc19.x86_64

2nd Instance:
-----------------
From: Cledat, Romain E
Sent: Monday, September 08, 2014 4:34 PM
To: Bernel, BrianX D
Subject: Error

Message from syslogd@bar1 at Sep 8 16:08:51 ...
kernel:[1235195.162972] LustreError: 85031:0:(osc_lock.c:1129:osc_lock_enqueue()) ASSERTION( ols->ols_state == OLS_NEW ) failed: Impossible state: 4 r
Message from syslogd@bar1 at Sep 8 16:08:51 ...
kernel:[1235195.193211] LustreError: 85031:0:(osc_lock.c:1129:osc_lock_enqueue()) LBUG

1st Instance:
-----------------
From: Nickerson, Brian R
Sent: Thursday, August 14, 2014 3:44 PM
To: Bernel, BrianX D; Cledat, Romain E
Subject: Kernel crash details

Message from syslogd@bar4 at Aug 14 15:34:57 ...
kernel:[1216856.270451] LustreError: 42598:0:(osc_lock.c:1129:osc_lock_enqueue()) ASSERTION( ols->ols_state == OLS_NEW ) failed: Impossible state: 4

Message from syslogd@bar4 at Aug 14 15:34:57 ...
kernel:[1216856.271008] LustreError: 42598:0:(osc_lock.c:1129:osc_lock_enqueue()) LBUG

Message from syslogd@bar4 at Aug 14 15:34:57 ...
kernel:[1216856.271830] Kernel panic - not syncing: LBUG

Issue Links

• is related to: LU-5910 LustreError: osc_lock_enqueue() Impossible state: 4
• is related to: LU-5492 osc_lock.c:1165:osc_lock_enqueue()) ASSERTION( ols->ols_state == OLS_NEW ) failed: Impossible state: 6

Comments

[Keith Mannthey (Inactive)]

We do not need a kernel dump to get this information.

Can you upload /var/log/messages or "dmesg" right after this issue is hit?

[Bob Glossman (Inactive)]

Brian,
The stack dumps Cliff asked about a few comments ago is still needed. Any chance of getting those?

[Brian Bernel (Inactive)]

Hi Bob,

Thanks so much for your help on this. The Lustre kernel for X-Stack was put together by Gabrielle Paciucci. …As for what might have been going on at the time of the problem, the only thing I have to go on at this point is that git was involved. No kdump available, I’m afraid. A cursory look at the logs doesn’t show anything glaringly amiss, but it does corroborate that garret/git was in play in the minute leading up to the Lustre error.

(Thanks Josh, for directing me to reply by comment vs email)

Regards,

Brian

[Bob Glossman (Inactive)]

this may not be known but I wonder if there was any particular applications or load that was running near the time of the 2 reported panic instances.

[Bob Glossman (Inactive)]

the reported lustre version is "Lustre Version: 2.4.52". this suggests it was built from source or derived from a review build in between our standard release versions with names like 2.4.2 or 2.4.3. Could we get detail about how this lustre was generated or obtained? knowing the exact origin is very important to help us understand the problem.

[Peter Jones]

Bob

Could you please help with this issue?

Thanks

Peter

[Cliff White (Inactive)]

There should be a stack dump to go along with the ASSERTION - can you please acquire and attach to the ticket? In addition, it would be useful to have logs for some time before the actual assertion - are there any LustreErrors previous to this?