[LU-10367] FIO Fails to run with libaio

• Type: Bug
• Resolution: Cannot Reproduce
• Priority: Major
• Affects Version/s: Lustre 2.10.1
• Environment: Lustre 2.10.1 RHEL 7.4
• Severity: 3
• Assignee: Andriy Skulysh
• Reporter: Keith Mannthey (Inactive)

Description

While doing some bench marking with FIO is seems libaio is not working with Lustre.

FIO can be found here: https://github.com/axboe/fio/

When running with libaio it quickly fails with error messages. A sample FIO configuration file is below:

fsync=1
directory=/mnt/lustre 
direct=1
numjobs=32
ioengine=libaio
blocksize=64k
readwrite=readwrite
size=65536m
group_reporting

Sample error messages:

.....
lustre: Laying out IO file(s) (1 file(s) / 65536MB)
fio: io_u error on file /mnt/lustre/lustre.21.0: Unknown error -131072: read offset=0, buflen=65536
fio: io_u error on file /mnt/lustre/lustre.21.0: Unknown error -131072: read offset=65536, buflen=65536
fio: pid=51260, err=-131072/file:io_u.c:1708, func=io_u error, error=Unknown error -131072
...

Removing libaio options allows FIO to run.  

Fsync and Direct really slow Lustre down but they run. 

Other tests  work as expected. 

Issue Links

• is related to: LU-11825 Remove LU-8964/pio feature & supporting framework
• is related to: LU-8964 use parallel I/O to improve performance on machines with slow single thread performance

Comments

[Aurelien Degremont (Inactive)]

The problem is still present in 2.10 LTS. The provided patch seems to fix it.

Could be interesting for 2.10.7?

[Peter Jones]

ok - thanks

[Andriy Skulysh]

The patch isn't needed after LU-11825

[James A Simmons]

Now that the PIO code has been removed is this a problem still?

[Gerrit Updater]

Andriy Skulysh ([e-mail]) uploaded a new patch: https://review.whamcloud.com/33716
Subject: LU-10367 llite: aio read returns doubled bytes count
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 823b81a1b911ce92612e5953ca4d3135bd01b2a7

[Thomas Favre-Bulle (Inactive)]

Hello,

I'm having the same problem with 2.10.3.

Async IO simply won't work (libaio), directIO are much slower than what we get with 2.7 (same hardware).

Any idea where this could come from please?

Thank you,

[Homer Li (Inactive)]

Yes, I have the same problem.

It 's lustre 2.10.2 +  zfs 0.7.3