[LU-2025] cd'ing to a certain directory causes client eviction

• Status: Resolved
• Project: Lustre
• Affects Version/s: Lustre 1.8.7
• Type: Bug
• Priority: Major
• Reporter: Manish Patel (Inactive)
• Assignee: Yang Sheng
• Resolution: Cannot Reproduce
• Severity: 3

Description

Directory not accessible. When a CD is typed as follows:
cd /scratch1/portfolios/NCEPDEV/jcsda/noscrub/Yong.Chen/CRTM_ODPStraining/work_ir_v12.1_Zeus/ECMWF83/iasiB3_metop-a

The following error is generated after a ~6 minute wait:
cannot access iasiB3_metop-a: Cannot send after transport endpoint shutdown

ls of this directory hangs for more than 10 minutes. behavior is not see on any of the parent directories.

It appears that the MDT gets stuck and the client is eventually evicted. I have kernel messages from the the client as well as debug logs for the client and MDT.

Comments

[Kit Westneat (Inactive)]

The behavior can be reproduced on any client at will. Let me know if there is other information that could be useful, and I will try to get it.

[Peter Jones]

Yangsheng

Could you please comment on this one?

Thanks

Peter

[Kit Westneat (Inactive)]

Any update? This user is currently unable to access their data, so it's high priority for us.

Thanks,
Kit

[Yang Sheng]

Hi, Kit, I'll starting look into this issue. Could you please recollect the logs for me? They are look like not include the failure-point data. I cannot find the Directory name in fe7 log. And please trying to stop other operations while collect log. Of course, this looks like a product system and very busy. So i understand it not so easy to do. TIA.

[Oleg Drokin]

So it looks like some mds processes stuck while processing requests.
A messages from MDS would be useful.
Also sysrq-t from MDS

[Peter Jones]

As per DDN this is no longer a priority