Both master vs b2_2 and b2_2 vs master hit this issue:
https://maloo.whamcloud.com/test_sessions/611d3f18-d058-11e1-9002-52540035b04c
https://maloo.whamcloud.com/test_sessions/dc6a13ea-d075-11e1-9002-52540035b04c
Jinshan,
Would you be able to take a look at this one and assign to yourself if it's appropriate for you to fix?
Thank you!
Jinshan will look into this one
We need to land commit 35a8ed2b2007d89c1f125f01f155232e7f511e98 to 2.2 to fix this problem.
What do you know. I have the patch for LU-1252 for b2_2 already available at http://review.whamcloud.com/#change,3008
Cool, thanks James.
James
Thanks! 
Peter
Jinshan,
Since we do have 2.2 servers, the 2.3 client should not crash. So we should create a patch on master to avoid the crash. Can you please look into this?
Thank you!
Hi Jodi,
master has already had this patch. The problem was 2.2 was released during the time when we're working on this patch. After we commit this patch to 2.2, then we're done.
Jinshan
Apparently this patch being will crash the 2.2 client, so we cannot retroactively fix that release, and the chance of making a 2.2.1 release is small. We need to have some mechanism to detect if the client is handling this swabbing correctly. The best method is to use an OBD_CONNECT flag being sent from the 2.3+ clients and checked by the 2.3+ servers to decide how the swabbing needs to be done.
I'm reluctant to use a separate flag for just fixing this rare bug. Oleg's suggestion is to re-use an existing OBD_CONNECT flag that is not currently being used for the MGS, which can be deprecated easily in the future. I would suggest OBD_CONNECT_GRANT, which is a flag that we can also soon deprecate for 2.x clients as well. Something like:
/* overload OBD_CONNECT_GRANT to fix rare 2.2/2.3 problem with mixed-endian * interop swabbing for IR mne_length field. This can be removed in the * future when we don't expect 2.2 clients running with 2.3+ servers. * See LU-1644 for details */ #define OBD_CONNECT_MNE_SWAB OBD_CONNECT_GRANT
A patch is pushed to: http://review.whamcloud.com/3548
to fix the comparability problem between 2.2 client and 2.3+ servers.
Landed for 2.3
Stillhit this error on 2.2 client<->2.3-tag2.2.94 interop testing
https://maloo.whamcloud.com/test_sessions/1f9fde02-f42e-11e1-b3b2-52540035b04c
I can't even mount 2.2 clients to master servers, with this error:
[root@client-17 ~]# uname -r 2.6.32-220.4.2.el6_lustre.g45b2fe8.x86_64 [root@client-17 ~]# rpm -qa |grep lustre lustre-2.2.0-2.6.32_220.4.2.el6_lustre.g45b2fe8.x86_64_g25a1427.x86_64 kernel-2.6.32-220.4.2.el6_lustre.g45b2fe8.x86_64 lustre-modules-2.2.0-2.6.32_220.4.2.el6_lustre.g45b2fe8.x86_64_g25a1427.x86_64 lustre-ldiskfs-3.3.0-2.6.32_220.4.2.el6_lustre.g45b2fe8.x86_64_g25a1427.x86_64 [root@client-17 ~]# mount -t lustre client-18@tcp:/lustre /mnt/lustre mount.lustre: mount client-18@tcp:/lustre at /mnt/lustre failed: Invalid argument This may have multiple causes. Is 'lustre' the correct filesystem name? Are the mount options correct? Check the syslog for more info. [root@client-17 ~]# dmesg Lustre: MGC10.10.4.18@tcp: Reactivating import Lustre: 6833:0:(obd_config.c:1002:class_process_config()) Ignoring unknown param jobid_var=procname_uid LustreError: 6833:0:(obd_config.c:1362:class_config_llog_handler()) Err -22 on cfg command: Lustre: cmd=cf00f 0:(null) 1:sys.jobid_var=procname_uid 2:procname_uid LustreError: 15b-f: MGC10.10.4.18@tcp: The configuration from log 'lustre-client'failed from the MGS (-22). Make sure this client and the MGS are running compatible versions of Lustre. LustreError: 15c-8: MGC10.10.4.18@tcp: The configuration from log 'lustre-client' failed (-22). This may be the result of communication errors between this node and the MGS, a bad configuration, or other errors. See the syslog for more information. LustreError: 6821:0:(llite_lib.c:978:ll_fill_super()) Unable to process log: -22 LustreError: 6736:0:(lov_obd.c:928:lov_cleanup()) lov tgt 0 not cleaned! deathrow=0, lovrc=1 LustreError: 6736:0:(lov_obd.c:928:lov_cleanup()) Skipped 3 previous similar messages LustreError: 6821:0:(ldlm_request.c:1170:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 6821:0:(ldlm_request.c:1796:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 Lustre: client ffff880329563000 umount complete LustreError: 6821:0:(obd_mount.c:2349:lustre_fill_super()) Unable to mount (-22)
any secret to do that successfully?
Patch is at: http://review.whamcloud.com/3897
Landed for 2.3 and 2.4
Andreas Dilger (andreas.dilger@intel.com) uploaded a new patch: https://review.whamcloud.com/32087
Subject: LU-1644 mgc: remove obsolete IR swabbing workaround
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: ee1e23c0b535f5726008b0de015cea813815d6de
Andreas Dilger (andreas.dilger@intel.com) uploaded a new patch: https://review.whamcloud.com/32088
Subject: LU-1644 ptlrpc: fix return type of boolean functions
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 764161ac692e9ff99ba48d39032a1adb0ff5351e
Oleg Drokin (oleg.drokin@intel.com) merged in patch https://review.whamcloud.com/32087/
Subject: LU-1644 mgc: remove obsolete IR swabbing workaround
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: a0c644fde3405bba6752885481f0fdfe05da1bcd
Oleg Drokin (oleg.drokin@intel.com) merged in patch https://review.whamcloud.com/32088/
Subject: LU-1644 ptlrpc: fix return type of boolean functions
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: e2cac9fb9baf43f48eb58334fb5044cece5395c0