[LU-5984] server_mgc_set_fs()) can't set_fs -17 - Whamcloud Community JIRA

Details

Type: Bug
Resolution: Fixed
Priority: Blocker
Fix Version/s: None
Affects Version/s: Lustre 2.5.3
Labels:
- llnl

Severity:
2
Rank (Obsolete):
16699

Description

After upgrading Lustre to 2.5.3 (specifically lustre-2.5.3-2chaos) we're no longer able to start the MDS due to the following failure.

Lustre: Lustre: Build Version: 2.5.3-2chaos-2chaos--PRISTINE-2.6.32-431.29.2.1chaos.ch5.2.x86_64
LustreError: 13871:0:(obd_mount_server.c:313:server_mgc_set_fs()) can't set_fs -17
Lustre: fsv-MDT0000: Unable to start target: -17
LustreError: 13871:0:(obd_mount_server.c:845:lustre_disconnect_lwp()) fsv-MDT0000-lwp-MDT0000: Can't end config log fsv-client.
LustreError: 13871:0:(obd_mount_server.c:1419:server_put_super()) fsv-MDT0000: failed to disconnect lwp. (rc=-2)
LustreError: 13871:0:(obd_mount_server.c:1449:server_put_super()) no obd fsv-MDT0000
LustreError: 13871:0:(obd_mount_server.c:135:server_deregister_mount()) fsv-MDT0000 not registered
Lustre: server umount fsv-MDT0000 complete
LustreError: 13871:0:(obd_mount.c:1326:lustre_fill_super()) Unable to mount  (-17)

I took a look at the Lustre debug log and the failure is due to a problem creating the local copy of the config logs. This is a ZFS based MDS which is upgrading from 2.4.x so there was never a local CONFIGS directory.

I'll attach the full log but basically it seems to be correctly detecting there is no CONFIGS directory. Then it attempts to create the directory which fails with -17 EEXISTS. Given the debug log we have it's not clear why this fails since the directory clearly doesn't exist. We've mounted the MDT via the ZPL and verified this.

Hoping we could work around the issue we tried manually created the CONFIGS directory and added a copy of the llogs from the MGS. We also just tried creating an empty CONFIGS directory through the ZPL. In both cases this caused the MDS to LBUG on start as follows:

2014-12-04 11:10:50 LustreError: 16688:0:(osd_index.c:1313:osd_index_try()) ASSERTION( dt_object_exists(dt) ) failed:
2014-12-04 11:10:50 LustreError: 16688:0:(osd_index.c:1313:osd_index_try()) LBUG
2014-12-04 11:10:50 Pid: 16688, comm: mount.lustre
2014-12-04 11:10:50
2014-12-04 11:10:50 Call Trace:
2014-12-04 11:10:50  [<ffffffffa05d18f5>] libcfs_debug_dumpstack+0x55/0x80 [libcfs]
2014-12-04 11:10:50  [<ffffffffa05d1ef7>] lbug_with_loc+0x47/0xb0 [libcfs]
2014-12-04 11:10:50  [<ffffffffa0d623e4>] osd_index_try+0x224/0x470 [osd_zfs]
2014-12-04 11:10:50  [<ffffffffa0740d41>] dt_try_as_dir+0x41/0x60 [obdclass]
2014-12-04 11:10:50  [<ffffffffa0741351>] dt_lookup_dir+0x31/0x130 [obdclass]
2014-12-04 11:10:50  [<ffffffffa071f845>] llog_osd_open+0x475/0xbb0 [obdclass]
2014-12-04 11:10:50  [<ffffffffa06f15ba>] llog_open+0xba/0x2c0 [obdclass]
2014-12-04 11:10:50  [<ffffffffa06f5131>] llog_backup+0x61/0x500 [obdclass]
2014-12-04 11:10:50  [<ffffffff8128f540>] ? sprintf+0x40/0x50
2014-12-04 11:10:50  [<ffffffffa0d99757>] mgc_process_log+0x1177/0x18f0 [mgc]
2014-12-04 11:10:50  [<ffffffffa0d93360>] ? mgc_blocking_ast+0x0/0x810 [mgc]
2014-12-04 11:10:50  [<ffffffffa08991e0>] ? ldlm_completion_ast+0x0/0x920 [ptlrpc]
2014-12-04 11:10:50  [<ffffffffa0d9b4b5>] mgc_process_config+0x645/0x11d0 [mgc]
2014-12-04 11:10:50  [<ffffffffa07351c6>] lustre_process_log+0x256/0xa60 [obdclass]
2014-12-04 11:10:50  [<ffffffffa05e1971>] ? libcfs_debug_msg+0x41/0x50 [libcfs]
2014-12-04 11:10:50  [<ffffffffa05dc378>] ? libcfs_log_return+0x28/0x40 [libcfs]
2014-12-04 11:10:50  [<ffffffffa0766cb7>] server_start_targets+0x9e7/0x1db0 [obdclass]
2014-12-04 11:10:50  [<ffffffffa05dc378>] ? libcfs_log_return+0x28/0x40 [libcfs]
2014-12-04 11:10:50  [<ffffffffa0738876>] ? lustre_start_mgc+0x4b6/0x1e60 [obdclass]
2014-12-04 11:10:50  [<ffffffffa05e1971>] ? libcfs_debug_msg+0x41/0x50 [libcfs]
2014-12-04 11:10:50  [<ffffffffa0730760>] ? class_config_llog_handler+0x0/0x1880 [obdclass]
2014-12-04 11:10:50  [<ffffffffa076ceb8>] server_fill_super+0xb98/0x19e0 [obdclass]
2014-12-04 11:10:50  [<ffffffffa05dc378>] ? libcfs_log_return+0x28/0x40 [libcfs]
2014-12-04 11:10:50  [<ffffffffa073a3f8>] lustre_fill_super+0x1d8/0x550 [obdclass]
2014-12-04 11:10:50  [<ffffffffa073a220>] ? lustre_fill_super+0x0/0x550 [obdclass]
2014-12-04 11:10:50  [<ffffffff8118d1ef>] get_sb_nodev+0x5f/0xa0
2014-12-04 11:10:50  [<ffffffffa07320e5>] lustre_get_sb+0x25/0x30 [obdclass]
2014-12-04 11:10:50  [<ffffffff8118c82b>] vfs_kern_mount+0x7b/0x1b0
2014-12-04 11:10:50  [<ffffffff8118c9d2>] do_kern_mount+0x52/0x130
2014-12-04 11:10:50  [<ffffffff811ae21b>] do_mount+0x2fb/0x930
2014-12-04 11:10:50  [<ffffffff811ae8e0>] sys_mount+0x90/0xe0
2014-12-04 11:10:50  [<ffffffff8100b0b2>] system_call_fastpath+0x16/0x1b

At this point we're rolling back to the previous Lustre release in order to make the system available again.

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending
- Thumbnails
- List
- Download All

vesta-mds1.log
12 kB
04/Dec/14 8:27 PM

Issue Links

is related to

LU-2059 mgc to backup configuration on osd-based llogs

Resolved

Activity

[LU-5984] server_mgc_set_fs()) can't set_fs -17

Christopher Morrone (Inactive) added a comment - 08/Dec/14 10:08 PM

This is a ZFS filesystem, so there isn't much in the way of "older versions". The current filesystem in use was probably formatted under lustre 2.4.0.

Since there are 128 OI directories, I took a wild guess that the OI values are stored in those directories modulo 128, so 0x200000003 would be in oi.3. oi.3 does contain:

0x200000003:0x0:0x0
0x200000003:0x1:0x0
0x200000003:0x2:0x0
0x200000003:0x3:0x0
0x200000003:0x4:0x0
0x200000003:0x5:0x0
0x200000003:0x6:0x0
0x200000003:0x7:0x0

For 0x200000003:0x2 I see:

# \ls oi.3/0x200000003:0x2:0x0
dt-0x0  md-0x0

Christopher Morrone (Inactive) added a comment - 08/Dec/14 10:08 PM This is a ZFS filesystem, so there isn't much in the way of "older versions". The current filesystem in use was probably formatted under lustre 2.4.0. Since there are 128 OI directories, I took a wild guess that the OI values are stored in those directories modulo 128, so 0x200000003 would be in oi.3. oi.3 does contain: 0x200000003:0x0:0x0 0x200000003:0x1:0x0 0x200000003:0x2:0x0 0x200000003:0x3:0x0 0x200000003:0x4:0x0 0x200000003:0x5:0x0 0x200000003:0x6:0x0 0x200000003:0x7:0x0 For 0x200000003:0x2 I see: # \ls oi.3/0x200000003:0x2:0x0 dt-0x0 md-0x0

Mikhail Pershin added a comment - 08/Dec/14 3:46 PM

In 2.5 the MOUNT_CONFIGS_DIR is created by name in special FID sequence FID_SEQ_LOCAL_NAME = 0x200000003ULL. During mount the new object with FID 0x200000003:0x2 was generated for CONFIGS directory but for some reason it exists already in filesystem OI. I am not sure how that happens, maybe this sequence was used in older Lustre versions for something else, this is area to investigate further.

Local creation will not help because there are no IGIFs in ZFS, and created directory has no associated FID neither in direntry nor in OI.

That would be useful to inspect OI to find out are there FIDs with sequence 0x200000003 and how many as next step.

Mikhail Pershin added a comment - 08/Dec/14 3:46 PM In 2.5 the MOUNT_CONFIGS_DIR is created by name in special FID sequence FID_SEQ_LOCAL_NAME = 0x200000003ULL. During mount the new object with FID 0x200000003:0x2 was generated for CONFIGS directory but for some reason it exists already in filesystem OI. I am not sure how that happens, maybe this sequence was used in older Lustre versions for something else, this is area to investigate further. Local creation will not help because there are no IGIFs in ZFS, and created directory has no associated FID neither in direntry nor in OI. That would be useful to inspect OI to find out are there FIDs with sequence 0x200000003 and how many as next step.

Jian Yu added a comment - 05/Dec/14 8:25 AM

Here are the conf-sanity test_32[abd] clean upgrading test reports for Lustre 2.5.3 with ZFS:
https://testing.hpdd.intel.com/sub_tests/a9e5dbde-311c-11e4-b503-5254006e85c2
https://testing.hpdd.intel.com/sub_tests/a9f8a32c-311c-11e4-b503-5254006e85c2
https://testing.hpdd.intel.com/sub_tests/aa5d42be-311c-11e4-b503-5254006e85c2
All of the tests passed.

I also tried to perform clean upgrading from Lustre 2.4.3 to 2.5.3 with ZFS but could not reproduce the failure. After upgrading, the Lustre filesystem was mounted successfully.

The configuration was:

1 MGS/MDS node (different datasets for MGS and MDT)
2 OSS nodes (1 OST per node)
2 client nodes

After upgrading, on MGS/MDS node:

# zfs list
NAME               USED  AVAIL  REFER  MOUNTPOINT
lustre-mdt1        231M   457G   136K  /lustre-mdt1
lustre-mdt1/mdt1   229M   457G   229M  /lustre-mdt1/mdt1
lustre-mgs        5.89M   457G   136K  /lustre-mgs
lustre-mgs/mgs    5.34M   457G  5.34M  /lustre-mgs/mgs

# df
Filesystem       1K-blocks    Used Available Use% Mounted on
/dev/sda1         20642428 2041472  17552380  11% /
tmpfs             16393952       0  16393952   0% /dev/shm
lustre-mgs/mgs   475194880    5504 475187328   1% /mnt/mgs
lustre-mdt1/mdt1 475192704  234240 474956416   1% /mnt/mds1

Jian Yu added a comment - 05/Dec/14 8:25 AM Here are the conf-sanity test_32[abd] clean upgrading test reports for Lustre 2.5.3 with ZFS: https://testing.hpdd.intel.com/sub_tests/a9e5dbde-311c-11e4-b503-5254006e85c2 https://testing.hpdd.intel.com/sub_tests/a9f8a32c-311c-11e4-b503-5254006e85c2 https://testing.hpdd.intel.com/sub_tests/aa5d42be-311c-11e4-b503-5254006e85c2 All of the tests passed. I also tried to perform clean upgrading from Lustre 2.4.3 to 2.5.3 with ZFS but could not reproduce the failure. After upgrading, the Lustre filesystem was mounted successfully. The configuration was: 1 MGS/MDS node (different datasets for MGS and MDT) 2 OSS nodes (1 OST per node) 2 client nodes After upgrading, on MGS/MDS node: # zfs list NAME USED AVAIL REFER MOUNTPOINT lustre-mdt1 231M 457G 136K /lustre-mdt1 lustre-mdt1/mdt1 229M 457G 229M /lustre-mdt1/mdt1 lustre-mgs 5.89M 457G 136K /lustre-mgs lustre-mgs/mgs 5.34M 457G 5.34M /lustre-mgs/mgs # df Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda1 20642428 2041472 17552380 11% / tmpfs 16393952 0 16393952 0% /dev/shm lustre-mgs/mgs 475194880 5504 475187328 1% /mnt/mgs lustre-mdt1/mdt1 475192704 234240 474956416 1% /mnt/mds1

Andreas Dilger added a comment - 04/Dec/14 11:15 PM

I've also added Alex and Mike to the CC list. Looking at the osd-zfs/osd_oi.c code, it appears that it is looking up the "CONFIGS" directory by FID, but in ZFS that FID may only be referenced by the OI file and not have a filename:

static const struct named_oid oids[] = {
        { LAST_RECV_OID,                LAST_RCVD },
        { OFD_LAST_GROUP_OID,           "LAST_GROUP" },
        { LLOG_CATALOGS_OID,            "CATALOGS" },
        { MGS_CONFIGS_OID,              NULL /*MOUNT_CONFIGS_DIR*/ },

It isn't clear to me why only the MGS_CONFIGS_OID=4119 "named_oid" in osd-zfs doesn't have a filename in the namespace, unlike other OIDs, and in osd-ldiskfs. It doesn't appear that MGS_CONFIGS_OID is used directly in the code anywhere, only via MOUNT_CONFIGS_DIR, so at leas. While MOUNT_CONFIGS_DIR use is partly ldiskfs-specific for accessing the "mountdata" file (which is stored in dataset properties in ZFS), it is also used in the server_mgc_set_fs-> mgc_fs_setup-> local_file_find_or_create() callpath that is failing.

It also isn't clear why conf-sanity test_32[ab] upgrade tests did not show any problems, since there is definitely a test image lustre/tests/disk2_4-zfs.tar.bz2 (created with 2.4.0) that should run 2.4.0->current version upgrades for every review-zfs test. There is ~~LU-5649~~ that is specific to ZFS failing test_32 but it looks like it was only a short-term regression on 2.5 due to a patch that was later reverted before release ("~~LU-3489~~ osd-zfs: varsize index support").

Andreas Dilger added a comment - 04/Dec/14 11:15 PM I've also added Alex and Mike to the CC list. Looking at the osd-zfs/osd_oi.c code, it appears that it is looking up the "CONFIGS" directory by FID, but in ZFS that FID may only be referenced by the OI file and not have a filename: static const struct named_oid oids[] = { { LAST_RECV_OID, LAST_RCVD }, { OFD_LAST_GROUP_OID, "LAST_GROUP" }, { LLOG_CATALOGS_OID, "CATALOGS" }, { MGS_CONFIGS_OID, NULL /*MOUNT_CONFIGS_DIR*/ }, It isn't clear to me why only the MGS_CONFIGS_OID=4119 "named_oid" in osd-zfs doesn't have a filename in the namespace, unlike other OIDs, and in osd-ldiskfs. It doesn't appear that MGS_CONFIGS_OID is used directly in the code anywhere, only via MOUNT_CONFIGS_DIR, so at leas. While MOUNT_CONFIGS_DIR use is partly ldiskfs-specific for accessing the "mountdata" file (which is stored in dataset properties in ZFS), it is also used in the server_mgc_set_fs-> mgc_fs_setup-> local_file_find_or_create() callpath that is failing. It also isn't clear why conf-sanity test_32 [ab] upgrade tests did not show any problems, since there is definitely a test image lustre/tests/disk2_4-zfs.tar.bz2 (created with 2.4.0) that should run 2.4.0->current version upgrades for every review-zfs test. There is LU-5649 that is specific to ZFS failing test_32 but it looks like it was only a short-term regression on 2.5 due to a patch that was later reverted before release (" LU-3489 osd-zfs: varsize index support").

Peter Jones added a comment - 04/Dec/14 8:53 PM

Oleg is looking into this

Peter Jones added a comment - 04/Dec/14 8:53 PM Oleg is looking into this

Brian Behlendorf added a comment - 04/Dec/14 8:34 PM

A little additional information about the configuration on the MDS node. One thing to notice is particular is the MGS and MDT are running on the same node but use different datasets. They are also named slightly differently for historical reasons. This MGS originally only had configuration for a filesystem named lsv, then a second was added called fsv and the original lsv was retired.

$ zfs list
NAME                  USED  AVAIL  REFER  MOUNTPOINT
vesta-mds1           1.04T  1.02T    30K  /vesta-mds1
vesta-mds1/fsv-mdt0  1.03T  1.02T   864G  legacy
vesta-mds1/mgs       27.2M  1.02T  26.9M  /vesta-mds1/mgs

# Top level MDT directory
# ls
CATALOGS           oi.10   oi.112  oi.125  oi.23  oi.36  oi.49  oi.61  oi.74  oi.87  quota_master
O                  oi.100  oi.113  oi.126  oi.24  oi.37  oi.5   oi.62  oi.75  oi.88  quota_slave
PENDING            oi.101  oi.114  oi.127  oi.25  oi.38  oi.50  oi.63  oi.76  oi.89  seq-200000003-lastid
ROOT               oi.102  oi.115  oi.13   oi.26  oi.39  oi.51  oi.64  oi.77  oi.9   seq_ctl
capa_keys          oi.103  oi.116  oi.14   oi.27  oi.4   oi.52  oi.65  oi.78  oi.90  seq_srv
changelog_catalog  oi.104  oi.117  oi.15   oi.28  oi.40  oi.53  oi.66  oi.79  oi.91
changelog_users    oi.105  oi.118  oi.16   oi.29  oi.41  oi.54  oi.67  oi.8   oi.92
fld                oi.106  oi.119  oi.17   oi.3   oi.42  oi.55  oi.68  oi.80  oi.93
last_rcvd          oi.107  oi.12   oi.18   oi.30  oi.43  oi.56  oi.69  oi.81  oi.94
lfsck_bookmark     oi.108  oi.120  oi.19   oi.31  oi.44  oi.57  oi.7   oi.82  oi.95
lfsck_namespace    oi.109  oi.121  oi.2    oi.32  oi.45  oi.58  oi.70  oi.83  oi.96
lov_objid          oi.11   oi.122  oi.20   oi.33  oi.46  oi.59  oi.71  oi.84  oi.97
oi.0               oi.110  oi.123  oi.21   oi.34  oi.47  oi.6   oi.72  oi.85  oi.98
oi.1               oi.111  oi.124  oi.22   oi.35  oi.48  oi.60  oi.73  oi.86  oi.99

# Top level and CONFIGS MGS direcotry
$ ls
CONFIGS          oi.106  oi.117  oi.13  oi.24  oi.35  oi.46  oi.57  oi.68  oi.79  oi.9
NIDTBL_VERSIONS  oi.107  oi.118  oi.14  oi.25  oi.36  oi.47  oi.58  oi.69  oi.8   oi.90
O                oi.108  oi.119  oi.15  oi.26  oi.37  oi.48  oi.59  oi.7   oi.80  oi.91
oi.0             oi.109  oi.12   oi.16  oi.27  oi.38  oi.49  oi.6   oi.70  oi.81  oi.92
oi.1             oi.11   oi.120  oi.17  oi.28  oi.39  oi.5   oi.60  oi.71  oi.82  oi.93
oi.10            oi.110  oi.121  oi.18  oi.29  oi.4   oi.50  oi.61  oi.72  oi.83  oi.94
oi.100           oi.111  oi.122  oi.19  oi.3   oi.40  oi.51  oi.62  oi.73  oi.84  oi.95
oi.101           oi.112  oi.123  oi.2   oi.30  oi.41  oi.52  oi.63  oi.74  oi.85  oi.96
oi.102           oi.113  oi.124  oi.20  oi.31  oi.42  oi.53  oi.64  oi.75  oi.86  oi.97
oi.103           oi.114  oi.125  oi.21  oi.32  oi.43  oi.54  oi.65  oi.76  oi.87  oi.98
oi.104           oi.115  oi.126  oi.22  oi.33  oi.44  oi.55  oi.66  oi.77  oi.88  oi.99
oi.105           oi.116  oi.127  oi.23  oi.34  oi.45  oi.56  oi.67  oi.78  oi.89  seq-200000003-lastid

$ ls CONFIGS/
fsv-MDT0000  fsv-OST0018  fsv-OST0031  fsv-OST004a  lsv-OST0001  lsv-OST001a  lsv-OST0033  lsv-OST004c
fsv-OST0000  fsv-OST0019  fsv-OST0032  fsv-OST004b  lsv-OST0002  lsv-OST001b  lsv-OST0034  lsv-OST004d
fsv-OST0001  fsv-OST001a  fsv-OST0033  fsv-OST004c  lsv-OST0003  lsv-OST001c  lsv-OST0035  lsv-OST004e
fsv-OST0002  fsv-OST001b  fsv-OST0034  fsv-OST004d  lsv-OST0004  lsv-OST001d  lsv-OST0036  lsv-OST004f
fsv-OST0003  fsv-OST001c  fsv-OST0035  fsv-OST004e  lsv-OST0005  lsv-OST001e  lsv-OST0037  lsv-OST0050
fsv-OST0004  fsv-OST001d  fsv-OST0036  fsv-OST004f  lsv-OST0006  lsv-OST001f  lsv-OST0038  lsv-OST0051
fsv-OST0005  fsv-OST001e  fsv-OST0037  fsv-OST0050  lsv-OST0007  lsv-OST0020  lsv-OST0039  lsv-OST0052
fsv-OST0006  fsv-OST001f  fsv-OST0038  fsv-OST0051  lsv-OST0008  lsv-OST0021  lsv-OST003a  lsv-OST0053
fsv-OST0007  fsv-OST0020  fsv-OST0039  fsv-OST0052  lsv-OST0009  lsv-OST0022  lsv-OST003b  lsv-OST0054
fsv-OST0008  fsv-OST0021  fsv-OST003a  fsv-OST0053  lsv-OST000a  lsv-OST0023  lsv-OST003c  lsv-OST0055
fsv-OST0009  fsv-OST0022  fsv-OST003b  fsv-OST0054  lsv-OST000b  lsv-OST0024  lsv-OST003d  lsv-OST0056
fsv-OST000a  fsv-OST0023  fsv-OST003c  fsv-OST0055  lsv-OST000c  lsv-OST0025  lsv-OST003e  lsv-OST0057
fsv-OST000b  fsv-OST0024  fsv-OST003d  fsv-OST0056  lsv-OST000d  lsv-OST0026  lsv-OST003f  lsv-OST0058
fsv-OST000c  fsv-OST0025  fsv-OST003e  fsv-OST0057  lsv-OST000e  lsv-OST0027  lsv-OST0040  lsv-OST0059
fsv-OST000d  fsv-OST0026  fsv-OST003f  fsv-OST0058  lsv-OST000f  lsv-OST0028  lsv-OST0041  lsv-OST005a
fsv-OST000e  fsv-OST0027  fsv-OST0040  fsv-OST0059  lsv-OST0010  lsv-OST0029  lsv-OST0042  lsv-OST005b
fsv-OST000f  fsv-OST0028  fsv-OST0041  fsv-OST005a  lsv-OST0011  lsv-OST002a  lsv-OST0043  lsv-OST005c
fsv-OST0010  fsv-OST0029  fsv-OST0042  fsv-OST005b  lsv-OST0012  lsv-OST002b  lsv-OST0044  lsv-OST005d
fsv-OST0011  fsv-OST002a  fsv-OST0043  fsv-OST005c  lsv-OST0013  lsv-OST002c  lsv-OST0045  lsv-OST005e
fsv-OST0012  fsv-OST002b  fsv-OST0044  fsv-OST005d  lsv-OST0014  lsv-OST002d  lsv-OST0046  lsv-OST005f
fsv-OST0013  fsv-OST002c  fsv-OST0045  fsv-OST005e  lsv-OST0015  lsv-OST002e  lsv-OST0047  lsv-OST0060
fsv-OST0014  fsv-OST002d  fsv-OST0046  fsv-OST005f  lsv-OST0016  lsv-OST002f  lsv-OST0048  lsv-client
fsv-OST0015  fsv-OST002e  fsv-OST0047  fsv-client   lsv-OST0017  lsv-OST0030  lsv-OST0049  params
fsv-OST0016  fsv-OST002f  fsv-OST0048  fsv-params   lsv-OST0018  lsv-OST0031  lsv-OST004a  params-client
fsv-OST0017  fsv-OST0030  fsv-OST0049  lsv-MDT0000  lsv-OST0019  lsv-OST0032  lsv-OST004b

Brian Behlendorf added a comment - 04/Dec/14 8:34 PM A little additional information about the configuration on the MDS node. One thing to notice is particular is the MGS and MDT are running on the same node but use different datasets. They are also named slightly differently for historical reasons. This MGS originally only had configuration for a filesystem named lsv, then a second was added called fsv and the original lsv was retired. $ zfs list NAME USED AVAIL REFER MOUNTPOINT vesta-mds1 1.04T 1.02T 30K /vesta-mds1 vesta-mds1/fsv-mdt0 1.03T 1.02T 864G legacy vesta-mds1/mgs 27.2M 1.02T 26.9M /vesta-mds1/mgs # Top level MDT directory # ls CATALOGS oi.10 oi.112 oi.125 oi.23 oi.36 oi.49 oi.61 oi.74 oi.87 quota_master O oi.100 oi.113 oi.126 oi.24 oi.37 oi.5 oi.62 oi.75 oi.88 quota_slave PENDING oi.101 oi.114 oi.127 oi.25 oi.38 oi.50 oi.63 oi.76 oi.89 seq-200000003-lastid ROOT oi.102 oi.115 oi.13 oi.26 oi.39 oi.51 oi.64 oi.77 oi.9 seq_ctl capa_keys oi.103 oi.116 oi.14 oi.27 oi.4 oi.52 oi.65 oi.78 oi.90 seq_srv changelog_catalog oi.104 oi.117 oi.15 oi.28 oi.40 oi.53 oi.66 oi.79 oi.91 changelog_users oi.105 oi.118 oi.16 oi.29 oi.41 oi.54 oi.67 oi.8 oi.92 fld oi.106 oi.119 oi.17 oi.3 oi.42 oi.55 oi.68 oi.80 oi.93 last_rcvd oi.107 oi.12 oi.18 oi.30 oi.43 oi.56 oi.69 oi.81 oi.94 lfsck_bookmark oi.108 oi.120 oi.19 oi.31 oi.44 oi.57 oi.7 oi.82 oi.95 lfsck_namespace oi.109 oi.121 oi.2 oi.32 oi.45 oi.58 oi.70 oi.83 oi.96 lov_objid oi.11 oi.122 oi.20 oi.33 oi.46 oi.59 oi.71 oi.84 oi.97 oi.0 oi.110 oi.123 oi.21 oi.34 oi.47 oi.6 oi.72 oi.85 oi.98 oi.1 oi.111 oi.124 oi.22 oi.35 oi.48 oi.60 oi.73 oi.86 oi.99 # Top level and CONFIGS MGS direcotry $ ls CONFIGS oi.106 oi.117 oi.13 oi.24 oi.35 oi.46 oi.57 oi.68 oi.79 oi.9 NIDTBL_VERSIONS oi.107 oi.118 oi.14 oi.25 oi.36 oi.47 oi.58 oi.69 oi.8 oi.90 O oi.108 oi.119 oi.15 oi.26 oi.37 oi.48 oi.59 oi.7 oi.80 oi.91 oi.0 oi.109 oi.12 oi.16 oi.27 oi.38 oi.49 oi.6 oi.70 oi.81 oi.92 oi.1 oi.11 oi.120 oi.17 oi.28 oi.39 oi.5 oi.60 oi.71 oi.82 oi.93 oi.10 oi.110 oi.121 oi.18 oi.29 oi.4 oi.50 oi.61 oi.72 oi.83 oi.94 oi.100 oi.111 oi.122 oi.19 oi.3 oi.40 oi.51 oi.62 oi.73 oi.84 oi.95 oi.101 oi.112 oi.123 oi.2 oi.30 oi.41 oi.52 oi.63 oi.74 oi.85 oi.96 oi.102 oi.113 oi.124 oi.20 oi.31 oi.42 oi.53 oi.64 oi.75 oi.86 oi.97 oi.103 oi.114 oi.125 oi.21 oi.32 oi.43 oi.54 oi.65 oi.76 oi.87 oi.98 oi.104 oi.115 oi.126 oi.22 oi.33 oi.44 oi.55 oi.66 oi.77 oi.88 oi.99 oi.105 oi.116 oi.127 oi.23 oi.34 oi.45 oi.56 oi.67 oi.78 oi.89 seq-200000003-lastid $ ls CONFIGS/ fsv-MDT0000 fsv-OST0018 fsv-OST0031 fsv-OST004a lsv-OST0001 lsv-OST001a lsv-OST0033 lsv-OST004c fsv-OST0000 fsv-OST0019 fsv-OST0032 fsv-OST004b lsv-OST0002 lsv-OST001b lsv-OST0034 lsv-OST004d fsv-OST0001 fsv-OST001a fsv-OST0033 fsv-OST004c lsv-OST0003 lsv-OST001c lsv-OST0035 lsv-OST004e fsv-OST0002 fsv-OST001b fsv-OST0034 fsv-OST004d lsv-OST0004 lsv-OST001d lsv-OST0036 lsv-OST004f fsv-OST0003 fsv-OST001c fsv-OST0035 fsv-OST004e lsv-OST0005 lsv-OST001e lsv-OST0037 lsv-OST0050 fsv-OST0004 fsv-OST001d fsv-OST0036 fsv-OST004f lsv-OST0006 lsv-OST001f lsv-OST0038 lsv-OST0051 fsv-OST0005 fsv-OST001e fsv-OST0037 fsv-OST0050 lsv-OST0007 lsv-OST0020 lsv-OST0039 lsv-OST0052 fsv-OST0006 fsv-OST001f fsv-OST0038 fsv-OST0051 lsv-OST0008 lsv-OST0021 lsv-OST003a lsv-OST0053 fsv-OST0007 fsv-OST0020 fsv-OST0039 fsv-OST0052 lsv-OST0009 lsv-OST0022 lsv-OST003b lsv-OST0054 fsv-OST0008 fsv-OST0021 fsv-OST003a fsv-OST0053 lsv-OST000a lsv-OST0023 lsv-OST003c lsv-OST0055 fsv-OST0009 fsv-OST0022 fsv-OST003b fsv-OST0054 lsv-OST000b lsv-OST0024 lsv-OST003d lsv-OST0056 fsv-OST000a fsv-OST0023 fsv-OST003c fsv-OST0055 lsv-OST000c lsv-OST0025 lsv-OST003e lsv-OST0057 fsv-OST000b fsv-OST0024 fsv-OST003d fsv-OST0056 lsv-OST000d lsv-OST0026 lsv-OST003f lsv-OST0058 fsv-OST000c fsv-OST0025 fsv-OST003e fsv-OST0057 lsv-OST000e lsv-OST0027 lsv-OST0040 lsv-OST0059 fsv-OST000d fsv-OST0026 fsv-OST003f fsv-OST0058 lsv-OST000f lsv-OST0028 lsv-OST0041 lsv-OST005a fsv-OST000e fsv-OST0027 fsv-OST0040 fsv-OST0059 lsv-OST0010 lsv-OST0029 lsv-OST0042 lsv-OST005b fsv-OST000f fsv-OST0028 fsv-OST0041 fsv-OST005a lsv-OST0011 lsv-OST002a lsv-OST0043 lsv-OST005c fsv-OST0010 fsv-OST0029 fsv-OST0042 fsv-OST005b lsv-OST0012 lsv-OST002b lsv-OST0044 lsv-OST005d fsv-OST0011 fsv-OST002a fsv-OST0043 fsv-OST005c lsv-OST0013 lsv-OST002c lsv-OST0045 lsv-OST005e fsv-OST0012 fsv-OST002b fsv-OST0044 fsv-OST005d lsv-OST0014 lsv-OST002d lsv-OST0046 lsv-OST005f fsv-OST0013 fsv-OST002c fsv-OST0045 fsv-OST005e lsv-OST0015 lsv-OST002e lsv-OST0047 lsv-OST0060 fsv-OST0014 fsv-OST002d fsv-OST0046 fsv-OST005f lsv-OST0016 lsv-OST002f lsv-OST0048 lsv-client fsv-OST0015 fsv-OST002e fsv-OST0047 fsv-client lsv-OST0017 lsv-OST0030 lsv-OST0049 params fsv-OST0016 fsv-OST002f fsv-OST0048 fsv-params lsv-OST0018 lsv-OST0031 lsv-OST004a params-client fsv-OST0017 fsv-OST0030 fsv-OST0049 lsv-MDT0000 lsv-OST0019 lsv-OST0032 lsv-OST004b

server_mgc_set_fs()) can't set_fs -17

Details

Description

Attachments

Attachments

Issue Links

Activity

People

Dates