[LU-3060] After upgrade from 1.8.9 to 2.4, hit FAIL: iozone did not fail with EDQUOT

• Status: Resolved
• Project: Lustre
• Affects Version/s: Lustre 2.4.0
• Type: Bug
• Priority: Minor
• Reporter: Sarah Liu
• Assignee: Niu Yawei (Inactive)
• Resolution: Duplicate

Environment

before upgrade: 1.8.9
after upgrade: lustre-master build #1354

• Severity: 3

Description

After upgrade system from 1.8.9 to lustre-master build #1354

client console shows:

Verifying user quota_2usr that ran out of quotas is still out of quotas after the upgrade...
+ local verbose=false
+ '[' xclient-15,client-5 = x--verbose ']'
+ local rnodes=client-15,client-5
+ shift
+ single_local_node client-15,client-5
+ '[' client-15,client-5 = client-15 ']'
+ local 'myPDSH=pdsh -t 120 -S -Rrsh -w'
+ '[' -z 'pdsh -t 120 -S -Rrsh -w' -o 'pdsh -t 120 -S -Rrsh -w' = no_dsh -o 'pdsh -t 120 -S -Rrsh -w' = rsh ']'
++ get_node_count 'client-15 client-5'
++ local 'nodes=client-15 client-5'
++ echo client-15 client-5
++ wc -w
+ export FANOUT=2
+ FANOUT=2
+ false
+ false
+ [[ pdsh -t 120 -S -Rrsh -w = *-N* ]]
+ sed -re 's/^[^:]*: //g'
+ pdsh -t 120 -S -Rrsh -w client-15,client-5 '(PATH=$PATH:/usr/lib64/lustre/utils:/usr/lib64/lustre/tests:/sbin:/usr/sbin; cd /usr/lib64/lustre/tests; sh -c "runas -u quota_2usr /usr/bin/iozone -i 0 -e -+d -w -r 1024 -s 1048576 -f /mnt/lustre/d0.upgrade-downgrade/quota_2usr/iozone.\$(hostname).\$(date +%s)")'
client-15: running as uid/gid/euid/egid 60001/60001/60001/60001, groups:
client-15:  [/usr/bin/iozone] [-i] [0] [-e] [-+d] [-w] [-r] [1024] [-s] [1048576] [-f] [/mnt/lustre/d0.upgrade-downgrade/quota_2usr/iozone.client-15.lab.whamcloud.com.1364513136]
client-5: running as uid/gid/euid/egid 60001/60001/60001/60001, groups:
client-5:  [/usr/bin/iozone] [-i] [0] [-e] [-+d] [-w] [-r] [1024] [-s] [1048576] [-f] [/mnt/lustre/d0.upgrade-downgrade/quota_2usr/iozone.client-5.1364513136]
	Iozone: Performance Test of File I/O
	        Version $Revision: 3.373 $
		Compiled for 64 bit mode.
		Build: linux-AMD64 

	Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
	             Al Slater, Scott Rhine, Mike Wisner, Ken Goss
	             Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
	             Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
	             Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,
	             Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,
	             Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer.

	Run began: Thu Mar 28 16:25:36 2013

	Include fsync in write timing
	>>> I/O Diagnostic mode enabled. <<<
	Performance measurements are invalid in this mode.
	Setting no_unlink
	Record Size 1024 KB
	File size set to 1048576 KB
	Command line used: /usr/bin/iozone -i 0 -e -+d -w -r 1024 -s 1048576 -f /mnt/lustre/d0.upgrade-downgrade/quota_2usr/iozone.client-15.lab.whamcloud.com.1364513136
	Output is in Kbytes/sec
	Time Resolution = 0.000001 seconds.
	Processor cache size set to 1024 Kbytes.
	Processor cache line size set to 32 bytes.
	File stride size set to 17 * record size.
                                                            random  random    bkwd   record   stride                                   
              KB  reclen   write rewrite    read    reread    read   write    read  rewrite     read   fwrite frewrite   fread  freread
	Iozone: Performance Test of File I/O
	        Version $Revision: 3.373 $
		Compiled for 64 bit mode.
		Build: linux-AMD64 

	Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
	             Al Slater, Scott Rhine, Mike Wisner, Ken Goss
	             Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
	             Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
	             Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,
	             Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,
	             Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer.

	Run began: Thu Mar 28 16:25:36 2013

	Include fsync in write timing
	>>> I/O Diagnostic mode enabled. <<<
	Performance measurements are invalid in this mode.
	Setting no_unlink
	Record Size 1024 KB
	File size set to 1048576 KB
	Command line used: /usr/bin/iozone -i 0 -e -+d -w -r 1024 -s 1048576 -f /mnt/lustre/d0.upgrade-downgrade/quota_2usr/iozone.client-5.1364513136
	Output is in Kbytes/sec
	Time Resolution = 0.000001 seconds.
	Processor cache size set to 1024 Kbytes.
	Processor cache line size set to 32 bytes.
	File stride size set to 17 * record size.
                                                            random  random    bkwd   record   stride                                   
              KB  reclen   write rewrite    read    reread    read   write    read  rewrite     read   fwrite frewrite   fread  freread
         1048576    1024

Sanity check failed. Do not deploy this filesystem in a production environment !
         1048576    1024

Sanity check failed. Do not deploy this filesystem in a production environment !
+ return 44
 upgrade-downgrade : @@@@@@ FAIL: iozone did not fail with EDQUOT 
Dumping lctl log to /home/sarah/test_logs/upgrade-downgrade..*.1364513136.log
Resetting fail_loc on all nodes...done.

Not sure is this is duplicate with LU-2548

Comments

[Sarah Liu]

debug and dmesg logs

[Jodi Levi (Inactive)]

Niu,
Could you please take a look at this one?
Thank you!

[Sarah Liu]

Upgrade from 2.1.5 to 2.4 also hit this issue. Is this a dup of LU-3026?

[Niu Yawei (Inactive)]

Yes, it looks duplicate of LU-3026.