[#LU-202] Test failure on test suite sanity-quota

[LU-202] Test failure on test suite sanity-quota Created: 07/Apr/11 Updated: 03/May/11 Resolved: 29/Apr/11
Status:	Resolved
Project:	Lustre
Component/s:	None
Affects Version/s:	Lustre 2.1.0
Fix Version/s:	Lustre 2.1.0

Type:

Bug

Priority:

Minor

Reporter:

Maloo

Assignee:

Niu Yawei (Inactive)

Resolution:

Fixed

Votes:

Labels:

None

Attachments:

LU-202-master.patch

mds-log.tar.gz

ost-log.tar.gz

Severity:

Rank (Obsolete):

5059

Description

This issue was created by maloo for sarah <sarah@whamcloud.com>

This issue relates to the following test suite run: https://maloo.whamcloud.com/test_sets/addd9066-616c-11e0-a2b4-52540025f9af.

run for 400MB test file
User quota (limit: 409600 kbytes)
Disk quotas for user quota_usr (uid 60000):
Filesystem kbytes quota limit grace files quota limit grace
/mnt/lustre 0 0 409600 - 0 0 0 -
lustre-MDT0000_UUID
0 - 1024 - 0 - 0 -
lustre-OST0000_UUID
0 - 1024 - - - - -
lustre-OST0001_UUID
0 - 1024 - - - - -
lustre-OST0002_UUID
0 - 1024 - - - - -
lustre-OST0003_UUID
0 - 1024 - - - - -
lustre-OST0004_UUID
0 - 1024 - - - - -
lustre-OST0005_UUID
0 - 1024 - - - - -
lustre-OST0006_UUID
0 - 1024 - - - - -
lustre-OST0007_UUID
0 - 1024 - - - - -
lustre-OST0008_UUID
0 - 1024 - - - - -
lustre-OST0009_UUID
0 - 1024 - - - - -
Step1: trigger quota with 0_DIRECT
Write half of file
running as uid/gid/euid/egid 60000/60000/60000/60000, groups:
[/usr/lib64/lustre/tests/directio] [write] [/mnt/lustre/d0.sanity-quota/d23/f23-0] [0] [200] [1048576]
directio on /mnt/lustre/d0.sanity-quota/d23/f23-0 for 200x1048576 bytes
Write error Success (rc = 160342016, len = 209715200)
Files for user (quota_usr):
File: `/mnt/lustre/d0.sanity-quota/d23/f23-0'
Size: 160342016 Blocks: 313176 IO Block: 2097152 regular file
Device: 2c54f966h/743766374d Inode: 144116144779821062 Links: 1
Access: (0644/~~rw-r~~r-) Uid: (60000/quota_usr) Gid: (60000/quota_usr)
Access: 2011-04-07 12:28:34.000000000 -0700
Modify: 2011-04-07 12:28:37.000000000 -0700
Change: 2011-04-07 12:28:37.000000000 -0700
sanity-quota test_23: @@@@@@ FAIL: (1) write failure, but expect success: 409600
Dumping lctl log to /logdir/test_logs/2011-04-07/082134/sanity-quota.test_23.*.1302204517.log
tar: Removing leading `/' from member names
/logdir/test_logs/2011-04-07/082134/sanity-quota-1302204517.tar.bz2

Comments

Comment by Sarah Liu [ 07/Apr/11 ]

not sure if test 25 and 28 are the same issue, if they are not, I will create separated tickets for them. the log can be found on Maloo listed above.

Comment by Peter Jones [ 13/Apr/11 ]

Niu

Could you please look into this one?

Thanks

Peter

Comment by Sarah Liu [ 15/Apr/11 ]

same issue when doing quota test:
https://maloo.whamcloud.com/test_sets/756e52c6-67e1-11e0-b32b-52540025f9af

Comment by Niu Yawei (Inactive) [ 17/Apr/11 ]

Hi, Sarah

There is some defects in the test_23() test script: The test_23() always set stripe count of the test file to 1, however, it always write OSTCOUNT * xx to the test file, and the check_whether_skip() always check if OST0 has 120000 (about 120M) free space, so sometimes, when the OSTCOUNT is big, the test might fail for ENOSPC.

In your test, you have 10 OSTS, so the test_23() will write 10 * 40 = 400M to the single stripe file, and OST0 has 160696(about 150M) free space, it passed the check_whether_skip(), but failed for ENOSPC when writing the 400M file.

I think this bug can be always reproduced on big OST cout system, I made a patch to fix the test script, please help me to verify if it works, thank you.

Comment by Niu Yawei (Inactive) [ 17/Apr/11 ]

Since test_23() failed, the test file isn't deleted properly, and it occupied about 150MB quota usage for test user, which makes the following test_25/26/28 failed for EDQUOT. I think whenever the test_23 pass successfully, the following failures will be gone too.

Comment by Sarah Liu [ 21/Apr/11 ]

Hi Niu,

here is the result link with your patch.
https://maloo.whamcloud.com/test_sets/dda11ff6-6ca8-11e0-b32b-52540025f9af

Comment by Sarah Liu [ 22/Apr/11 ]

mds log

Comment by Niu Yawei (Inactive) [ 25/Apr/11 ]

Thanks a lot, Sarah. Because the lower limit of quota unit size is 1M bytes, which means each OST has at least 1M non-reclaimble quota, thus, 4M test file size is too small for this test. I've updated the patch for review:

http://review.whamcloud.com/#change,434

Comment by Sarah Liu [ 25/Apr/11 ]

this one works https://maloo.whamcloud.com/test_sets/fc37cbaa-6f8b-11e0-b32b-52540025f9af

Comment by Build Master (Inactive) [ 28/Apr/11 ]

Integrated in lustre-b1_8 » x86_64,server,el5,inkernel #21
~~LU-202~~ fix the test_23 of sanity-quota.sh

Johann Lombardi : 3b270f6b97a8594e1863fd5a91362cfe4ffb8833
Files :

lustre/tests/sanity-quota.sh

Comment by Build Master (Inactive) [ 28/Apr/11 ]

Integrated in lustre-b1_8 » x86_64,client,el6,inkernel #21
~~LU-202~~ fix the test_23 of sanity-quota.sh

Johann Lombardi : 3b270f6b97a8594e1863fd5a91362cfe4ffb8833
Files :

lustre/tests/sanity-quota.sh

Comment by Build Master (Inactive) [ 28/Apr/11 ]

Integrated in lustre-b1_8 » x86_64,client,ubuntu1004,inkernel #21
~~LU-202~~ fix the test_23 of sanity-quota.sh

Johann Lombardi : 3b270f6b97a8594e1863fd5a91362cfe4ffb8833
Files :

lustre/tests/sanity-quota.sh

Comment by Build Master (Inactive) [ 28/Apr/11 ]

Integrated in lustre-b1_8 » x86_64,client,el5,ofa #21
~~LU-202~~ fix the test_23 of sanity-quota.sh

Johann Lombardi : 3b270f6b97a8594e1863fd5a91362cfe4ffb8833
Files :

lustre/tests/sanity-quota.sh

Comment by Build Master (Inactive) [ 28/Apr/11 ]

Integrated in lustre-b1_8 » i686,client,el6,inkernel #21
~~LU-202~~ fix the test_23 of sanity-quota.sh

Johann Lombardi : 3b270f6b97a8594e1863fd5a91362cfe4ffb8833
Files :

lustre/tests/sanity-quota.sh

Comment by Build Master (Inactive) [ 28/Apr/11 ]

Integrated in lustre-b1_8 » x86_64,client,el5,inkernel #21
~~LU-202~~ fix the test_23 of sanity-quota.sh

Johann Lombardi : 3b270f6b97a8594e1863fd5a91362cfe4ffb8833
Files :

lustre/tests/sanity-quota.sh

Comment by Build Master (Inactive) [ 28/Apr/11 ]

Integrated in lustre-b1_8 » i686,server,el5,inkernel #21
~~LU-202~~ fix the test_23 of sanity-quota.sh

Johann Lombardi : 3b270f6b97a8594e1863fd5a91362cfe4ffb8833
Files :

lustre/tests/sanity-quota.sh

Comment by Build Master (Inactive) [ 28/Apr/11 ]

Integrated in lustre-b1_8 » i686,server,el5,ofa #21
~~LU-202~~ fix the test_23 of sanity-quota.sh

Johann Lombardi : 3b270f6b97a8594e1863fd5a91362cfe4ffb8833
Files :

lustre/tests/sanity-quota.sh

Comment by Build Master (Inactive) [ 28/Apr/11 ]

Integrated in lustre-b1_8 » x86_64,server,el5,ofa #21
~~LU-202~~ fix the test_23 of sanity-quota.sh

Johann Lombardi : 3b270f6b97a8594e1863fd5a91362cfe4ffb8833
Files :

lustre/tests/sanity-quota.sh

Comment by Build Master (Inactive) [ 28/Apr/11 ]

Integrated in lustre-b1_8 » i686,client,el5,ofa #21
~~LU-202~~ fix the test_23 of sanity-quota.sh

Johann Lombardi : 3b270f6b97a8594e1863fd5a91362cfe4ffb8833
Files :

lustre/tests/sanity-quota.sh

Comment by Build Master (Inactive) [ 28/Apr/11 ]

Integrated in lustre-b1_8 » i686,client,el5,inkernel #21
~~LU-202~~ fix the test_23 of sanity-quota.sh

Johann Lombardi : 3b270f6b97a8594e1863fd5a91362cfe4ffb8833
Files :

lustre/tests/sanity-quota.sh