[LU-17185] After deactivating OSTs, some clients see them as active

• Type: Bug
• Resolution: Unresolved
• Priority: Blocker
• Affects Version/s: Lustre 2.12.9

Environment

Client:
CentOS Linux release 7.9.2009 (Core)
3.10.0-1160.49.1.el7.x86_64
lustre-client-2.12.9-1.el7.x86_64
lustre-client-dkms-2.12.9-1.el7.noarch

Servers:
Servers:
CentOS Linux release 7.9.2009 (Core)
3.10.0-1160.49.1.el7_lustre.x86_64
Lustre 2.12.9

Epic/Theme

client mgs

• Severity: 3

Epic

client mgs

• Assignee: WC Triage
• Reporter: Roger Sersted

Description

After running lctl conf_param lustrefc-OST0018.osc.active=0 on the MGS for multiple OSTs, some clients see the OSTs as inactive and work just fine. Some clients see the OSTs as active and hang. The software stack is the same on working and non-working clients.

Here is some sample output from a working client:
4 IN osc lustrefc-OST0007-osc-ffff9d035c824000 c75633d5-3afe-6370-7e49-dcad475a6bc2 4
5 IN osc lustrefc-OST000e-osc-ffff9d035c824000 c75633d5-3afe-6370-7e49-dcad475a6bc2 4
6 IN osc lustrefc-OST000f-osc-ffff9d035c824000 c75633d5-3afe-6370-7e49-dcad475a6bc2 4

1. Note disabled OSTs not listed
2. lfs df -h partial ouput
   lustrefc-MDT0000_UUID 229.3G 61.3G 131.4G 32% /data[MDT:0]
   lustrefc-OST0012_UUID 64.9T 20.3T 44.6T 32% /data[OST:18]

Output from a non-working client
4 UP osc lustrefc-OST0007-osc-ffff958c1995b800 4910e4fd-accd-1685-c6d2-3418a29afbd1 3
5 UP osc lustrefc-OST000e-osc-ffff958c1995b800 4910e4fd-accd-1685-c6d2-3418a29afbd1 3
6 UP osc lustrefc-OST000f-osc-ffff958c1995b800 4910e4fd-accd-1685-c6d2-3418a29afbd1 3

1. lfs df -h partial output
   lustrefc-MDT0000_UUID 229.3G 61.3G 131.4G 32% /data[MDT:0]
   OST0007 : Invalid argument
   OST000e : Invalid argument
   OST000f : Invalid argument

This has rendered our cluster unusable.