[LU-8100] Missing MDTs in /proc/fs/lustre/lmv/lustre-clilmv-.../target_obd

• Type: Bug
• Resolution: Fixed
• Priority: Major
• Fix Version/s: Lustre 2.9.0
• Affects Version/s: Lustre 2.8.0
• Labels: llnl patch
• Environment: lustre-2.8.0_14_gd0cbf68-1.x86_64; 2.6.32-573.22.1.1chaos.ch5.4.x86_64
• Severity: 3
• Assignee: Lai Siyao
• Reporter: Giuseppe Di Natale (Inactive)

Description

Our test setup contains 16 MDTs. Our clients only ever see 10 MDTs. All output below is from a client with the file system mounted. All were formatted at the same time with the same script and same mgs nid.

Attempts to use any MDT that is not listed fails (i.e. lfs mkdir --index=10 will fail).

[root@catalyst320:mdc]# lfs mkdir --index=12 /p/lustre/dinatale/testdir
error on LL_IOC_LMV_SETSTRIPE '/p/lustre/dinatale/testdir' (3): No such device
error: mkdir: create stripe dir '/p/lustre/dinatale/testdir' failed
[root@catalyst320:mdc]# lfs mkdir --index=8 /p/lustre/dinatale/testdir
[root@catalyst320:mdc]# lfs getdirstripe /p/lustre/dinatale/testdir/        
/p/lustre/dinatale/testdir/
lmv_stripe_count: 0 lmv_stripe_offset: 8

[root@catalyst320:mdc]# cat /proc/fs/lustre/lmv/lustre-clilmv-ffff881003e14400/target_obd
0: lustre-MDT0000_UUID ACTIVE
1: lustre-MDT0001_UUID ACTIVE
2: lustre-MDT0002_UUID ACTIVE
3: lustre-MDT0003_UUID ACTIVE
4: lustre-MDT0004_UUID ACTIVE
5: lustre-MDT0005_UUID ACTIVE
6: lustre-MDT0006_UUID ACTIVE
7: lustre-MDT0007_UUID ACTIVE
8: lustre-MDT0008_UUID ACTIVE
9: lustre-MDT0009_UUID ACTIVE

ls /proc/fs/lustre/lmv/lustre-clilmv-ffff881003e14400/target_obds/
lustre-MDT0000-mdc-ffff881003e14400  lustre-MDT0006-mdc-ffff881003e14400  lustre-MDT0012-mdc-ffff881003e14400
lustre-MDT0001-mdc-ffff881003e14400  lustre-MDT0007-mdc-ffff881003e14400  lustre-MDT0013-mdc-ffff881003e14400
lustre-MDT0002-mdc-ffff881003e14400  lustre-MDT0008-mdc-ffff881003e14400  lustre-MDT0014-mdc-ffff881003e14400
lustre-MDT0003-mdc-ffff881003e14400  lustre-MDT0009-mdc-ffff881003e14400  lustre-MDT0015-mdc-ffff881003e14400
lustre-MDT0004-mdc-ffff881003e14400  lustre-MDT0010-mdc-ffff881003e14400
lustre-MDT0005-mdc-ffff881003e14400  lustre-MDT0011-mdc-ffff881003e14400

[root@catalyst320:mdc]# grep current_state */state
lustre-MDT0000-mdc-ffff881003e14400/state:current_state: FULL
lustre-MDT0001-mdc-ffff881003e14400/state:current_state: FULL
lustre-MDT0002-mdc-ffff881003e14400/state:current_state: FULL
lustre-MDT0003-mdc-ffff881003e14400/state:current_state: FULL
lustre-MDT0004-mdc-ffff881003e14400/state:current_state: FULL
lustre-MDT0005-mdc-ffff881003e14400/state:current_state: FULL
lustre-MDT0006-mdc-ffff881003e14400/state:current_state: FULL
lustre-MDT0007-mdc-ffff881003e14400/state:current_state: FULL
lustre-MDT0008-mdc-ffff881003e14400/state:current_state: FULL
lustre-MDT0009-mdc-ffff881003e14400/state:current_state: FULL
lustre-MDT0010-mdc-ffff881003e14400/state:current_state: FULL
lustre-MDT0011-mdc-ffff881003e14400/state:current_state: FULL
lustre-MDT0012-mdc-ffff881003e14400/state:current_state: FULL
lustre-MDT0013-mdc-ffff881003e14400/state:current_state: FULL
lustre-MDT0014-mdc-ffff881003e14400/state:current_state: FULL
lustre-MDT0015-mdc-ffff881003e14400/state:current_state: FULL

Comments

[Peter Jones]

Landed for 2.9

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch http://review.whamcloud.com/20546/
Subject: LU-8100 tests: Test the correctness of target_obd
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 1a8c46a434b20c1083fafbded4d39e6dbbdff37a

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch http://review.whamcloud.com/20336/
Subject: LU-8100 lmv: Correctly generate target_obd
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: e5bb5a373cb3548ec8f5989c7e08fa7a3c8259f3

[Gerrit Updater]

Giuseppe Di Natale ([e-mail]) uploaded a new patch: http://review.whamcloud.com/20546
Subject: LU-8100 conf-sanity: Test the correctness of target_obd
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: e7cd83e1a36a6ea790233055222eb052670c4324

[Andreas Dilger]

It should be possible to generate non-consecutive MDT indices. See conf-sanity test_56 for an example of this with large OST indices. Maybe renaming that to test_56a and adding test_56b for large MDT indices is the right way to go?

[Giuseppe Di Natale (Inactive)]

Went ahead and submitted a patch. It works on a VM setup I have, but I'm not 100% it's correct. I would also like to introduce a test to check this file. The test would require MDTs to have non-consecutive indices. Is it possible within the test framework to manually assign indices to MDTs or can someone point me to an example test?

[Gerrit Updater]

Giuseppe Di Natale ([e-mail]) uploaded a new patch: http://review.whamcloud.com/20336
Subject: LU-8100 lmv: Correctly generate target_obd
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: d8c2209167efb0fcbba5a1a390e07e800e938fec

[Lai Siyao]

No, thanks for your work!

[Giuseppe Di Natale (Inactive)]

Just to make sure no one else is working on this, I'll be submitting a patch soon.

[Giuseppe Di Natale (Inactive)]

I found the portion of the code you are talking about. I might already have a patch.