[LU-5730] intermittent I/O errors for some directories

• Type: Bug
• Resolution: Fixed
• Priority: Minor
• Affects Version/s: Lustre 2.5.2
• Environment: Lustre 2.5.2 on RHEL6 servers and clients, NFS exported, ACLs
• Severity: 3
• Assignee: Lai Siyao
• Reporter: Frederik Ferner (Inactive)

Description

Our users have reported an issue where the suddenly have problems editing a file in a directory, they also got I/O errors for example when trying to get the ACLs for that directory. In at least one instance the problem resolved itself overnight after we decided to investigate in more detail later, in another case today the problem went away when we renamed the problematic directory.

In the most recent instance today we had some time where we were able to attempt to understand the issue and this is what we found so far: While the problem persists, some clients are seeing I/O error on calling getfacl, other clients don't have any problems running the same commands and returned the expected results. Some machines access this directory over NFS, exported from one of our clients which was showing problems in this instance, they had the same issues. Attempting to edit a file in the problematic directory with vim came up with the message that the .swp file already exists even for new files. Creating new files in the directory, for example with touch, worked with no problem.

There are no error messages recorded by syslog on any of the machines involved.

We've mostly run out of ideas what to look for next to resolve this if it happens again.

Issue Links

• is related to: LU-6528 permission denied over NFS (intermittent
• is related to: LU-3727 LBUG (llite_nfs.c:281:ll_get_parent()) ASSERTION(body->valid & OBD_MD_FLID) failed

Comments

[Peter Jones]

Great - thanks for confirming

[Frederik Ferner (Inactive)]

A quick update. We have not had any reports from our users that they were still seeing this, not with the patched version and not with 2.7 which we are running now. So I guess this can be closed.

[Frederik Ferner (Inactive)]

Peter,

thanks for checking. Judging by the time it previously took our users previously to experience the issue, I might prefer to leave the ticket open a while longer. On the other hand, we can always re-open it if we see the same problem again.

However in the mean time, we are now seeing a (potentially) different problem with NFS, this time we don't get stale NFS file handle errors, we get permission denied instead when trying to create a file in a newly created directory. This is intermittent as well but reproducible. I'm currently unsure if I should continue using this ticket or if I should open a new ticket. I'm leaning towards opening a new ticket to avoid confusion. (I'm also currently still gathering information...)

Cheers,
Frederik

[Peter Jones]

Hi Frederik

I'm just checking in to see whether you are now comfortable to consider this issue resolved by the patch or whether you want to see a longer stretch without a reoccurrence.

Peter

[Peter Jones]

Thanks for the update Frederik. Keep us posted.

[Frederik Ferner (Inactive)]

Peter,

thanks for the confirmation. We have today upgraded our servers to 2.5.3 plus this suggested patch. Time will tell, we've so far only been able to reproduce this after the file system has been up for a few weeks.

Frederik