[LU-3163] conf-sanity test_61 reformats MDT but not OST Created: 12/Apr/13  Updated: 29/Oct/15  Resolved: 05/Aug/15

Status: Resolved
Project: Lustre
Component/s: None
Affects Version/s: None
Fix Version/s: Lustre 2.8.0

Type: Bug Priority: Minor
Reporter: John Hammond Assignee: WC Triage
Resolution: Fixed Votes: 0
Labels: patch, test

Issue Links:
Related
is related to LU-3648 large_xattr_enabled in test-framework... Resolved
is related to LU-7134 Ensure ZFS hostid protection if servi... Resolved
Severity: 3
Rank (Obsolete): 7707

 Description   

I can't seem to get large_xattr enabled through the test framework so conf-sanity test_61 reformats the MDT. It does this after formatting, starting, and stopping the MDT and one OST. It does not however reformat the OST. So when the OST starts it asks the MDT "Are you my mother?" To which the MDT replies "No."

# git describe 
2.3.63-83-g0f1d647
# ls /tmp
# mount
/dev/vda1 on / type ext4 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
tmpfs on /dev/shm type tmpfs (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
192.168.122.1:/export on /export type nfs (rw,vers=4,addr=192.168.122.1,clientaddr=192.168.122.101)
# ONLY=61 sh ./lustre/tests/conf-sanity.sh 
Logging to shared log directory: /tmp/test_logs/1365797832
error: lustre_build_version: No such file or directory
opening /dev/lnet failed: No such device
hint: the kernel modules may not be loaded
IOC_LIBCFS_GET_NI error 19: No such device
only running test 61
Loading modules from /root/lustre-release/lustre/tests/..
detected 4 online CPUs by sysfs
Force libcfs to create 2 CPU partitions
../libcfs/libcfs/libcfs options: 'cpu_npartitions=2'
debug=-1
subsystem_debug=all -lnet -lnd -pinger
gss/krb5 is not supported
quota/lquota options: 'hash_lqs_cur_bits=3'
excepting tests: 32newtarball 59 64
skipping tests SLOW=no: 30a 31 45
Stopping clients: n /mnt/lustre (opts:)
Stopping clients: n /mnt/lustre2 (opts:)
Loading modules from /root/lustre-release/lustre/tests/..
detected 4 online CPUs by sysfs
Force libcfs to create 2 CPU partitions
debug=-1
subsystem_debug=all -lnet -lnd -pinger
gss/krb5 is not supported
Formatting mgs, mds, osts
Format mds1: /tmp/lustre-mdt1
Format ost1: /tmp/lustre-ost1
Format ost2: /tmp/lustre-ost2
start mds service on n
Starting mds1:   -o loop /tmp/lustre-mdt1 /mnt/mds1
Started lustre-MDT0000
start ost1 service on n
Starting ost1:   -o loop /tmp/lustre-ost1 /mnt/ost1
Started lustre-OST0000
n: osc.lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec
stop ost1 service on n
Stopping /mnt/ost1 (opts:-f) on n
stop mds service on n
Stopping /mnt/mds1 (opts:-f) on n
umount lustre on /mnt/lustre.....
stop ost1 service on n
stop mds service on n
modules unloaded.
umount lustre on /mnt/lustre.....
stop ost1 service on n
stop mds service on n
modules unloaded.


== conf-sanity test 61: large xattr == 15:17:52 (1365797872)
Loading modules from /root/lustre-release/lustre/tests/..
detected 4 online CPUs by sysfs
Force libcfs to create 2 CPU partitions
../libcfs/libcfs/libcfs options: 'cpu_npartitions=2 cpu_npartitions=2 cpu_npartitions=2'
debug=-1
subsystem_debug=all -lnet -lnd -pinger
gss/krb5 is not supported
quota/lquota options: 'hash_lqs_cur_bits=3'

   Permanent disk data:
Target:     lustre:MDT0000
Index:      0
Lustre FS:  lustre
Mount type: ldiskfs
Flags:      0x65
              (MDT MGS first_time update )
Persistent mount opts: user_xattr,errors=remount-ro
Parameters: sys.timeout=20 lov.stripesize=1048576 lov.stripecount=0 mdt.identity_upcall=/root/lustre-release/lustre/tests/../utils/l_getidentity

formatting backing filesystem ldiskfs on /dev/loop0
	target name  lustre:MDT0000
	4k blocks     50000
	options         -I 512 -i 2048 -q -O large_xattr,dirdata,uninit_bg,^extents,dir_nlink,quota,huge_file,flex_bg -E lazy_itable_init,lazy_journal_init -F
mkfs_cmd = mke2fs -j -b 4096 -L lustre:MDT0000   -I 512 -i 2048 -q -O large_xattr,dirdata,uninit_bg,^extents,dir_nlink,quota,huge_file,flex_bg -E lazy_itable_init,lazy_journal_init -F /dev/loop0 50000
Writing CONFIGS/mountdata
start mds service on n
Starting mds1:   -o loop /tmp/lustre-mdt1 /mnt/mds1
Started lustre-MDT0000
start ost1 service on n
Starting ost1:   -o loop /tmp/lustre-ost1 /mnt/ost1
mount.lustre: mount /dev/loop1 at /mnt/ost1 failed: No such device or address
The target service failed to start (bad config log?) (/dev/loop1).  See /var/log/messages.
Start of /tmp/lustre-ost1 on ost1 failed 6
mount lustre on /mnt/lustre.....
Starting client: n: -o user_xattr,flock n@tcp:/lustre /mnt/lustre
^C
# dmesg 
LNet: HW CPU cores: 4, npartitions: 2
alg: No test for crc32 (crc32-table)
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.3.63-g7223187-CHANGED-2.6.32-279.19.1.el6_lustre_gcov.x86_64
LNet: Added LNI 192.168.122.101@tcp [8/256/0/180]
LNet: Accept secure, port 988
Lustre: Echo OBD driver; http://www.lustre.org/
Lustre: DEBUG MARKER: only running test 61
Lustre: DEBUG MARKER: excepting tests: 32newtarball 59 64
Lustre: DEBUG MARKER: skipping tests SLOW=no: 30a 31 45
LDISKFS-fs (loop0): mounted filesystem with ordered data mode. quota=on. Opts: 
LDISKFS-fs (loop0): mounted filesystem with ordered data mode. quota=on. Opts: 
LDISKFS-fs (loop0): mounted filesystem with ordered data mode. quota=on. Opts: 
LDISKFS-fs (loop0): mounted filesystem with ordered data mode. quota=on. Opts: 
Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000
Lustre: Setting parameter lustre-clilov.lov.stripesize in log lustre-client
Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space
Lustre: srv-lustre-MDT0000: No data found on store. Initialize space
Lustre: lustre-MDT0000: new disk, initializing
Lustre: 11756:0:(mdt_lproc.c:380:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /root/lustre-release/lustre/tests/../utils/l_getidentity
LustreError: 11-0: lustre-MDT0000-lwp-MDT0000: Communicating with 0@lo, operation mds_connect failed with -11.
LDISKFS-fs (loop1): mounted filesystem with ordered data mode. quota=on. Opts: 
Lustre: lustre-OST0000: new disk, initializing
Lustre: srv-lustre-OST0000: No data found on store. Initialize space
Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec
LustreError: 11-0: lustre-OST0000-osc-MDT0000: Communicating with 0@lo, operation ost_statfs failed with -107.
Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
LustreError: 137-5: lustre-OST0000: Not available for connect from 0@lo (stopping)
Lustre: server umount lustre-OST0000 complete
Lustre: 12139:0:(client.c:1868:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1365797860/real 1365797860]  req@ffff8800c4234000 x1432142836924672/t0(0) o251->MGC192.168.122.101@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1365797866 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1
Lustre: server umount lustre-MDT0000 complete
LNet: Removed LNI 192.168.122.101@tcp
LNet: HW CPU cores: 4, npartitions: 2
alg: No test for crc32 (crc32-table)
alg: No test for adler32 (adler32-zlib)
Lustre: Lustre: Build Version: 2.3.63-g7223187-CHANGED-2.6.32-279.19.1.el6_lustre_gcov.x86_64
LNet: Added LNI 192.168.122.101@tcp [8/256/0/180]
LNet: Accept secure, port 988
Lustre: Echo OBD driver; http://www.lustre.org/
Lustre: DEBUG MARKER: == conf-sanity test 61: large xattr == 15:17:52 (1365797872)
LDISKFS-fs (loop0): mounted filesystem with ordered data mode. quota=on. Opts: 
LDISKFS-fs (loop0): mounted filesystem with ordered data mode. quota=on. Opts: 
Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000
Lustre: Setting parameter lustre-clilov.lov.stripesize in log lustre-client
Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space
Lustre: srv-lustre-MDT0000: No data found on store. Initialize space
Lustre: lustre-MDT0000: new disk, initializing
Lustre: 13302:0:(mdt_lproc.c:380:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /root/lustre-release/lustre/tests/../utils/l_getidentity
LustreError: 11-0: lustre-MDT0000-lwp-MDT0000: Communicating with 0@lo, operation mds_connect failed with -11.
LDISKFS-fs (loop1): mounted filesystem with ordered data mode. quota=on. Opts: 
LustreError: 13b-9: lustre-OST0000 claims to have registered, but this MGS does not know about it, preventing registration.
LustreError: 13390:0:(obd_mount_server.c:1123:server_register_target()) lustre-OST0000: error registering with the MGS: rc = -2 (not fatal)
LustreError: 13390:0:(obd_mount_server.c:1266:server_start_targets()) no server named lustre-OST0000 was started
LustreError: 13390:0:(obd_mount_server.c:1697:server_fill_super()) Unable to start targets: -6
LustreError: 13390:0:(obd_mount_server.c:844:lustre_disconnect_lwp()) lustre-MDT0000-lwp-OST0000: Can't end config log lustre-client.
LustreError: 13390:0:(obd_mount_server.c:1424:server_put_super()) lustre-OST0000: failed to disconnect lwp. (rc=-2)
LustreError: 13390:0:(obd_mount_server.c:1454:server_put_super()) no obd lustre-OST0000
Lustre: server umount lustre-OST0000 complete
LustreError: 13390:0:(obd_mount.c:1264:lustre_fill_super()) Unable to mount  (-6)
LustreError: 11-0: lustre-MDT0000-mdc-ffff88008863f800: Communicating with 0@lo, operation mds_connect failed with -11.
LustreError: 11-0: lustre-MDT0000-mdc-ffff88008863f800: Communicating with 0@lo, operation mds_connect failed with -11.
LustreError: 11-0: lustre-MDT0000-mdc-ffff88008863f800: Communicating with 0@lo, operation mds_connect failed with -11.
LustreError: 11-0: lustre-MDT0000-mdc-ffff88008863f800: Communicating with 0@lo, operation mds_connect failed with -11.
LustreError: 11-0: lustre-MDT0000-mdc-ffff88008863f800: Communicating with 0@lo, operation mds_connect failed with -11.
LustreError: 11-0: lustre-MDT0000-mdc-ffff88008863f800: Communicating with 0@lo, operation mds_connect failed with -11.
LustreError: Skipped 1 previous similar message
LustreError: 11-0: lustre-MDT0000-mdc-ffff88008863f800: Communicating with 0@lo, operation mds_connect failed with -11.
LustreError: Skipped 3 previous similar messages
LustreError: 11-0: lustre-MDT0000-mdc-ffff88008863f800: Communicating with 0@lo, operation mds_connect failed with -11.
LustreError: Skipped 6 previous similar messages
LustreError: 13438:0:(lmv_obd.c:1281:lmv_statfs()) can't stat MDS #0 (lustre-MDT0000-mdc-ffff88008863f800), error -11
Lustre: Unmounted lustre-client
LustreError: 13438:0:(obd_mount.c:1264:lustre_fill_super()) Unable to mount  (-11)


 Comments   
Comment by Andreas Dilger [ 12/Apr/13 ]

It should be possible to enable large xattrs on any MDS with "tune2fs -O large_xattr /dev/

{mdsdev}

".

I actually wrote http://review.whamcloud.com/4315 to enable large xattrs by default, but it has fallen off my plate to finish this patch and the requisite work in LU-908 needed to get it pushed upstream.

Comment by James Nunez (Inactive) [ 15/Nov/13 ]

LU-3648 should allow large_xattr to be properly detected, but this doesn't impact the issue of reformatting the MDS and not the OSTs.

Comment by Gerrit Updater [ 03/Jul/15 ]

Ashish Purkar (ashish.purkar@seagate.com) uploaded a new patch: http://review.whamcloud.com/15486
Subject: LU-3163 tests: Fix test_61 from conf-sanity test
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 0f4fb41804b4fa80bacf448fd570391e888fe600

Comment by Gerrit Updater [ 05/Aug/15 ]

Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/15486/
Subject: LU-3163 tests: Fix test_61 from conf-sanity test
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 8c9720c55d54e4407c99f371b701e01ec87c196a

Comment by Peter Jones [ 05/Aug/15 ]

Landed for 2.8

Generated at Sat Feb 10 01:31:31 UTC 2024 using Jira 9.4.14#940014-sha1:734e6822bbf0d45eff9af51f82432957f73aa32c.