[LU-6007] FIEMAP fails xfstests's fiemap-tester

• Status: Open
• Project: Lustre
• Affects Version/s: Lustre 2.7.0
• Type: Bug
• Priority: Major
• Reporter: Frank Zago (Inactive)
• Assignee: WC Triage
• Resolution: Unresolved
• Severity: 3

Description

Following resolution of bug LU-5933, I tried to run fiemap-tester from xfstests, and saw it fail. I don't know whether it's a Lustre or some bad assumption, but the same test works fine on ext4, and presumably all filesystem supported by the xfs test suite.

xfstests is easy to get and install. See http://xfs.org/index.php/Getting_the_latest_source_code. fiemap-tester will fail within seconds, with no particular parameter besides the name of a file on Lustre.

# ./src/fiemap-tester /mnt/lustre/qwerty
preallocation not supported, disabling
Starting infinite run, if you don't see any output then its working properly.
ERROR: couldn't find extent at 197
map is 'HHHHHHDHDDDHDDDHDHHHDHHHHHDHDHDDDDHDDDDHDHDDDDDDDHHHDHDHHHDDHHHDHHDHHDDHDDDHHHHHHDDDDHHDDHDDDDDDHHHDDHDDDHHHHHHDDHDDHDHDDHDHDDDHHDDHDDHHDDHHHHDDHHHDHDDDHDDDDDHHDHHDHHHHDHDDHHHDDDHHDDHDDDHDDHHDDHHDHDDDHDHDDHDDDDDHDDHHDHHHHHHDDDHHHHDDHDHHHDHHHHDDDDHHHHHHDHHHHDHDHDDHHHDHDDHHDDDDHHHHDHDHHDHDDHHDDDHDDHHHDHHDDHHHDHHHHDDDDHDHHDHHHHDHDDDHDDDHDHHDHHDDDHDHDHDDDHHHDHHHHHDDDDHHDDHDHHDHHHDHHDHHHHDHHHHHQ'
logical: [       6..       6] phys:    34998..   34998 flags: 0x80000000 tot: 1
logical: [       8..      10] phys:    34999..   35001 flags: 0x80000000 tot: 3
logical: [      12..      14] phys:    35002..   35004 flags: 0x80000000 tot: 3
...
logical: [     187..     188] phys:    35094..   35095 flags: 0x80000000 tot: 2
logical: [     191..     192] phys:    35096..   35097 flags: 0x80000000 tot: 2
logical: [     195..     195] phys:    35098..   35098 flags: 0x80000000 tot: 1
Problem comparing fiemap and map

Related to LU-920.

Issue Links

• is related to: LU-1941 ZFS FIEMAP support
• is related to: LU-1923 filefrag with large fiemap buffer cra...
• is related to: LU-5933 FIEMAP: FIEMAP_EXTENT_LAST is not alw...
• is related to: LU-10335 Ubuntu1604 client sanity-130a: FAIL: ...
• is related to: LU-920 include xfstests into regular testing...

Comments

[Andreas Dilger]

I would love to get xfstests running on Lustre on a regular basis.

[Frank Zago (Inactive)]

FIEMAP is broken at least 2 ways.

• FIEMAP_EXTENT_LAST is set even when the extent is not the last one
• fm_start is somewhat ignored.

Here's two sample output from the same test, on Lustre and ext4.

Lustre head of tree:

$./src/fiemap-tester /mnt/lustre/qwe -S 7 -r 1

*** NEW RUN ***
FS_IOC_FIEMAP query fm_start=0, fm_length=1142784, fm_extent_count=279
FS_IOC_FIEMAP result, fm_mapped_extents=75
logical: [       1..       6] phys:    35008..   35013 flags: 0x80000000 tot: 6
logical: [       8..       9] phys:    35014..   35015 flags: 0x80000000 tot: 2
logical: [      12..      12] phys:    35016..   35016 flags: 0x80000000 tot: 1
.....
logical: [     272..     272] phys:    35141..   35141 flags: 0x80000000 tot: 1
logical: [     275..     276] phys:    35142..   35143 flags: 0x80000000 tot: 2
logical: [     278..     281] phys:    35144..   35147 flags: 0x80000001 tot: 4
FS_IOC_FIEMAP query fm_start=311296, fm_length=1142784, fm_extent_count=279
FS_IOC_FIEMAP result, fm_mapped_extents=90
logical: [       1..       6] phys:    35008..   35013 flags: 0x80000000 tot: 6
logical: [       8..       9] phys:    35014..   35015 flags: 0x80000000 tot: 2
logical: [      12..      12] phys:    35016..   35016 flags: 0x80000000 tot: 1
.....
logical: [     341..     341] phys:    35178..   35178 flags: 0x80000000 tot: 1
logical: [     343..     350] phys:    35179..   35186 flags: 0x80000000 tot: 8
logical: [     354..     355] phys:    35187..   35188 flags: 0x80000001 tot: 2
FS_IOC_FIEMAP query fm_start=684032, fm_length=1142784, fm_extent_count=279
FS_IOC_FIEMAP result, fm_mapped_extents=97
logical: [       1..       6] phys:    35008..   35013 flags: 0x80000000 tot: 6
logical: [       8..       9] phys:    35014..   35015 flags: 0x80000000 tot: 2
logical: [      12..      12] phys:    35016..   35016 flags: 0x80000000 tot: 1
.....

ext4:

*** NEW RUN ***
FS_IOC_FIEMAP query fm_start=0, fm_length=1142784, fm_extent_count=279
FS_IOC_FIEMAP result, fm_mapped_extents=126
logical: [       0..       1] phys:  2563767.. 2563768 flags: 0x800 tot: 2
logical: [       2..       3] phys:  4423998.. 4423999 flags: 0x000 tot: 2
logical: [       4..       7] phys:  2563769.. 2563772 flags: 0x800 tot: 4
.....
logical: [     273..     274] phys:  4430097.. 4430098 flags: 0x000 tot: 2
logical: [     275..     275] phys:  4430099.. 4430099 flags: 0x800 tot: 1
logical: [     278..     278] phys:  4430102.. 4430102 flags: 0x800 tot: 1
FS_IOC_FIEMAP query fm_start=520192, fm_length=1142784, fm_extent_count=279
FS_IOC_FIEMAP result, fm_mapped_extents=107
logical: [     127..     127] phys:  4425471.. 4425471 flags: 0x800 tot: 1
logical: [     129..     129] phys:  4445185.. 4445185 flags: 0x000 tot: 1
logical: [     130..     130] phys:  4445186.. 4445186 flags: 0x800 tot: 1
.....
logical: [     377..     377] phys:  4430201.. 4430201 flags: 0x000 tot: 1
logical: [     380..     380] phys:  4430204.. 4430204 flags: 0x000 tot: 1
logical: [     382..     382] phys:  4430206.. 4430206 flags: 0x801 tot: 1
FS_IOC_FIEMAP query fm_start=962560, fm_length=1142784, fm_extent_count=279
FS_IOC_FIEMAP result, fm_mapped_extents=60
logical: [     234..     235] phys:  4445290.. 4445291 flags: 0x800 tot: 2
logical: [     238..     240] phys:  4445294.. 4445296 flags: 0x800 tot: 3
logical: [     242..     242] phys:  4445298.. 4445298 flags: 0x800 tot: 1
.....

[Frank Zago (Inactive)]

Patch to add more debug traces to xfstests' fiemap-tester.c

[Frank Zago (Inactive)]

I've looked into that a bit more. basically fiemap will lie when the caller queries the file in chunks. Some application relying on that information will then read corrupted data.

I suggest increasing the severity of this bug.