[LU-2836] Test failure on test suite sanity-quota, subtest test_3

• Type: Bug
• Resolution: Fixed
• Priority: Major
• Fix Version/s: Lustre 2.4.0
• Affects Version/s: Lustre 2.4.0
• Labels: HB zfs
• Severity: 3
• Assignee: WC Triage
• Reporter: Maloo

Description

This issue was created by maloo for Li Wei <[e-mail]>

This issue relates to the following test suite run: https://maloo.whamcloud.com/test_sets/03c8681a-7af8-11e2-b916-52540035b04c.

The sub-test test_3 failed with the following error:

write failure, but expect success

Info required for matching: sanity-quota 3

Issue Links

• is duplicated by: LU-2837 Test failure on test suite sanity-quota, subtest test_6
• is related to: LU-4422 Failure on test suite sanity-quota test_6: file not growed in 30 seconds 2088960\/2088960
• is related to: LU-2874 Test timeout failure on test suite replay-ost-single test_8a: timeout on wait for dd
• is related to: LU-2887 sanity-quota test_12a: slow due to ZFS VMs sharing single disk

Comments

[Gerrit Updater]

Nathaniel Clark ([e-mail]) uploaded a new patch: https://review.whamcloud.com/32956
Subject: LU-2836 tests: Re-enable sanity-quota/3,6 for ZFS
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: ec4474bf153b79cb5ac5163f5eec00518ee1a709

[Andreas Dilger]

This test is still being skipped for ZFS. A patch needs to be submitted to remove it from ALWAYS_EXCEPT, with Test-Parameters: sanity-quota ostfilesystemtype=zfs run enough times that we are confident that it is passing consistently.

[James A Simmons]

Old blocker for unsupported version

[Andreas Dilger]

Bug was closed, but sanity-quota test_3 and test_6 are still being skipped for ZFS filesystems

[Peter Jones]

Landed for 2.4

[Niu Yawei (Inactive)]

This one is for ldiskfs, it's little bit different with other failures for zfs.

[Nathaniel Clark]

This is a slow dd command, which is related other slow dd commands.

[Niu Yawei (Inactive)]

http://review.whamcloud.com/5539

[Niu Yawei (Inactive)]

When approaching quota limit, client turns to sync write (one page by one page), on some slow test system, data flush could take very long time (longer than grace time), then grace could expires before we start testing.

I think we'd enlarge the grace time for the test_3.

LU-2837 has the similar problem, we'd enlarge the waiting time for test_6 either.

[Niu Yawei (Inactive)]

14:39:16:running as uid/gid/euid/egid 60000/60000/60000/60000, groups:
14:39:16: [dd] [if=/dev/zero] [bs=1M] [of=/mnt/lustre/d0.sanity-quota/d3/f.sanity-quota.3-0] [count=1]
14:39:16:1+0 records in
14:39:16:1+0 records out
14:39:16:1048576 bytes (1.0 MB) copied, 0.0296897 s, 35.3 MB/s
14:39:27:Write to exceed soft limit
14:39:27:running as uid/gid/euid/egid 60000/60000/60000/60000, groups:
14:39:27: [dd] [if=/dev/zero] [of=/mnt/lustre/d0.sanity-quota/d3/f.sanity-quota.3-0] [bs=1K] [count=10] [seek=1024]
14:39:27:10+0 records in
14:39:27:10+0 records out
14:39:27:10240 bytes (10 kB) copied, 9.24401 s, 1.1 kB/s
14:39:28:Disk quotas for user quota_usr (uid 60000):

It used 11 seconds to flush the data, so the grace has expired before the "Writing before timer goes off"

14:39:30:Write before timer goes off
14:39:30:running as uid/gid/euid/egid 60000/60000/60000/60000, groups:
14:39:30: [dd] [if=/dev/zero] [of=/mnt/lustre/d0.sanity-quota/d3/f.sanity-quota.3-0] [bs=1K] [count=10] [seek=2048]
14:39:30:dd: writing `/mnt/lustre/d0.sanity-quota/d3/f.sanity-quota.3-0': Disk quota exceeded

Not sure why flush 1M bytes data takes so long.