[LU-15700] conf-sanity test 115 does not cleanup on mkfs failure

• Status: Resolved
• Project: Lustre
• Affects Version/s: Lustre 2.15.0
• Type: Bug
• Priority: Minor
• Reporter: Oleg Drokin
• Assignee: WC Triage
• Resolution: Duplicate
• Severity: 3

Description

conf-sanity test 115 does not cleanup when it runs out of space leading to all subsequent tests failing due to no space left in /tmp

 

== conf-sanity test 115: Access large xattr with inodes number over 2TB ========================================================== 05:31:00 (1648373460)

oleg219-server: 
oleg219-server: mkfs.lustre FATAL: Unable to build fs /dev/loop0 (256)
oleg219-server: 
oleg219-server: mkfs.lustre FATAL: mkfs failed 256
pdsh@oleg219-client: oleg219-server: ssh exited with exit code 1

   Permanent disk data:
Target:     lustre:MDT0000
Index:      0
Lustre FS:  lustre
Mount type: ldiskfs
Flags:      0x65
              (MDT MGS first_time update )
Persistent mount opts: user_xattr,errors=remount-ro
Parameters: sys.timeout=20 mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity

device size = 3145728MB
formatting backing filesystem ldiskfs on /dev/loop0
	target name   lustre:MDT0000
	kilobytes     3221225472
	options        -N 2247484000  -J size=4096 -I 1024 -q -O ea_inode,^resize_inode,meta_bg,dirdata,uninit_bg,^extents,dir_nlink,quota,project,huge_file,large_dir,flex_bg -E lazy_itable_init,lazy_journal_init="0" -F
mkfs_cmd = mke2fs -j -b 4096 -L lustre:MDT0000  -N 2247484000  -J size=4096 -I 1024 -q -O ea_inode,^resize_inode,meta_bg,dirdata,uninit_bg,^extents,dir_nlink,quota,project,huge_file,large_dir,flex_bg -E lazy_itable_init,lazy_journal_init="0" -F /dev/loop0 3221225472k

 SKIP: conf-sanity test_115 format large MDT failed

== conf-sanity test 116: big size MDT support ============ 05:45:12 (1648374312)
/usr/sbin/mkfs.xfs
Stopping clients: oleg219-client.virtnet /mnt/lustre (opts:)
Stopping clients: oleg219-client.virtnet /mnt/lustre2 (opts:)
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
Force libcfs to create 2 CPU partitions
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
loading modules on: 'oleg219-server'
oleg219-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 469: cannot create temp file for here-document: No space left on device
...
 conf-sanity test_116: @@@@@@ FAIL: mkfs temporary xfs image 

Issue Links

• is related to: LU-11915 conf-sanity test 115 is skipped or hangs
• is related to: LU-15789 conf-sanity test_115() cleanup_115(...

Comments

[Colin Faber]

artem_blagodarenko – heads up.

[Colin Faber]

"Test was being skipped in Janitor testing, probably should not block 2.15 release"

[Artem Blagodarenko (Inactive)]

cfaber, it looks strange. There is cleanup_115() that executed on skip and do "rm -f $TMP/$tdir/lustre-mdt" so large file should be deleted.

There can I see more logs for such a test session? 

Thanks.

[Andreas Dilger]

Artem, you could probably reproduce this easily in Oleg's Gerrit Janitor test system by pushing a patch that modified test_115 and test_116.

[Gerrit Updater]

"Artem Blagodarenko <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/47093
Subject: LU-15700 tests: check how conf_sanity_115 cleanup works
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 037e49dbf030101b5eb0312f3b5b8656948bdcb6

[Artem Blagodarenko (Inactive)]

 adilger , thank you very much for the idea. I have created  https://review.whamcloud.com/47093 that force "skip" in conf_sanity_115 and modify conf_sanity_116. Let's see resutls.

[Artem Blagodarenko (Inactive)]

The issue can be addressed by https://review.whamcloud.com/47153 from LU-15789

[Andreas Dilger]

Use LU-15789 for the fix.