https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-7851 Lustre <p>Before upgrade, system was formatted as 2.7.1 RHEL6.7 zfs<br/> 1. upgrade OSS from 2.7.1 RHEL6.7 zfs to 2.8.0 RHEL7.1 zfs, MDS and clients were remained 2.7.1, ran sanity, hit known issues;<br/> 2. then upgrade MDS from 2.7.1 RHEL6.7 zfs to 2.8.0 RHEL7.1 zfs, cannot mount</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>[root@onyx-25 ~]# mount -t lustre -o acl,user_xattr lustre-mdt1/mdt1 /mnt/mds1 [ 1753.734535] Lustre: MGS: Connection restored to 3f133170-911b-365a-d7e8-6a31eadc485c (at 0@lo) [ 1755.859175] LustreError: 22512:0:(obd_config.c:1387:class_process_proc_param()) mdt: lustre-MDT0000 unknown param som=disabled [ 1755.878206] LustreError: 22512:0:(obd_config.c:1666:class_config_llog_handler()) MGC10.2.4.47@tcp: cfg command failed: rc = -38 [ 1755.896907] Lustre: cmd=cf00f 0:lustre-MDT0000 1:mdt.som=disabled [ 1755.896907] [ 1755.911461] LustreError: 15c-8: MGC10.2.4.47@tcp: The configuration from log 'lustre-MDT0000' failed (-38). This may be the result of communication errors between this node and the MGS, a bad configuration, or other errors. See the syslog for more information. [ 1755.946877] LustreError: 22417:0:(obd_mount_server.c:1309:server_start_targets()) failed to start server lustre-MDT0000: -38 [ 1755.962800] LustreError: 22417:0:(obd_mount_server.c:1798:server_fill_super()) Unable to start targets: -38 [ 1755.976801] Lustre: Failing over lustre-MDT0000 [ 1763.422747] Lustre: 22417:0:(client.c:2063:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1457156520/real 1457156520] req@ffff8807f1960000 x1527938522546332/t0(0) o251-&gt;MGC10.2.4.47@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1457156526 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 [ 1765.473299] Lustre: server umount lustre-MDT0000 complete [ 1765.481424] LustreError: 22417:0:(obd_mount.c:1426:lustre_fill_super()) Unable to mount (-38) mount.lustre: mount lustre-mdt1/mdt1 at /mnt/mds1 failed: Function not implemented </pre> </div></div> before upgrade, system was formatted as 2.7.1 RHEL6.7 zfs LU-7851

ZFS rolling upgrade: cannot mount MDS: mdt: lustre-MDT0000 unknown param som=disabled

Bug Critical Resolved Fixed Hongchao Zhang Sarah Liu Sat, 5 Mar 2016 05:54:45 +0000 Wed, 28 Feb 2018 20:35:14 +0000 Wed, 27 Apr 2016 16:03:46 +0000 Lustre 2.8.0 Lustre 2.9.0 0 7 <p>upgrade 2.5.5 RHEL6.6 zfs to 2.8 RHEL6.7 zfs also hit the issue. It seems this is a zfs specific problem, ldiskfs didn't hit it.</p> <p>Hi Hongchao,</p> <p>Can you please have a look at this issue?</p> <p>Thanks.<br/> Joe</p> <p>when SOM code was removed som parameter handling should not have been removed completely, but instead converted to a noop like some other compat params.</p> <p>This mostly would only happen on the fiesystem where sanity.sh was run that tries to enable and then disable SOM</p> <p>I also tried upgrade from 2.7.1 RHEL6.7 zfs to 2.8 RHEL6.7 zfs, it didn't hit this issue.</p> <p>this problem could be related to the sanity.sh test during upgrading.</p> <p>Prior to 2.8, the SOM is disabled by default (mdt_som_conf = 0) and there should be no such configuration by default,<br/> but the parameter "som=disabled" for "som=enabled" will be setup during running sanity.sh, the it could cause this problem<br/> during upgrading the MDS to 2.8 afterwards for the newly added configuration of "som".</p> <p>Hi Sarah,<br/> Could you please verify whether it is the case? Thanks!</p> <p>Hongchao: even when users of the actual option might be rare, we still need the patch to ignore it since it was a valid one.</p> <p>That said making sure it does not hit by default is still good.</p> <p>Oleg: Okay, I'll create a patch to ignore the configuration.</p> <p>Hongchao Zhang (hongchao.zhang@intel.com) uploaded a new patch: <a href="http://review.whamcloud.com/18834" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/18834</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-7851" title="ZFS rolling upgrade: cannot mount MDS: mdt: lustre-MDT0000 unknown param som=disabled" class="issue-link" data-issue-key="LU-7851"><del>LU-7851</del></a> mdt: skip SOM related configuration<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 1555ccad216290befa4f32d93b366b6440bcc9d2</p> <p>hi Hongchao,</p> <p>sorry for the late reply. So here is the scenario will hit the problem<br/> 1. format and setup system as 2.5.5/2.7.1 zfs<br/> 2. upgrade OSS only to 2.8 zfs<br/> *3. run sanity with 2.8 OSS, 2.5.5/2.7.1 MDS and clients<br/> 4. now upgrade MDS to 2.8 zfs, hit the problem</p> <p>Step 3 is the key, without running sanity with such config won't hit this problem.</p> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="http://review.whamcloud.com/18834/" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/18834/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-7851" title="ZFS rolling upgrade: cannot mount MDS: mdt: lustre-MDT0000 unknown param som=disabled" class="issue-link" data-issue-key="LU-7851"><del>LU-7851</del></a> mdt: skip SOM related configuration<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 109cdb83d77844e00ebc6d5df6bf73845cf9d45e</p> <p>Landed to master for 2.9.0</p> Related LU-6047 Development Rank 1|hzy3pr: Rank (Obsolete) 9223372036854775807 Severity