[LU-10370] "truncate" does not update blocks count on client

• Status: Resolved
• Project: Lustre
• Affects Version/s: Lustre 2.10.1
• Fix Version/s: Lustre 2.12.0
• Type: Bug
• Priority: Minor
• Reporter: Andreas Dilger
• Assignee: Arshad Hussain
• Resolution: Fixed
• Labels: patch
• Severity: 3

Description

Running truncate --size=N file on a file larger than {{N} will correctly update and return the file size to the client, but does not update the client's idea of the block count. Running the following test fails on my 2.10.1 system:

$ dd if=1k bs=1M /myth/tmp/largefile
$ truncate --size 512M /myth/tmp/largefile

The size is updated on the client to 512MB, but the blocks are left at 1GB in the client cache.

Issue Links

• is related to: LU-3606 Implement fallocate() support for ldi...
• is related to: LU-9538 Size on MDT with guarantee of eventua...
• is related to: LU-11778 interop: sanity test 317 fails with '...

Comments

[Andreas Dilger]

The OSS should be replying with the object attributes (blocks, size, timestamps) from the truncate, and these should be updating the client's cached state.

[Arshad Hussain]

Hi Andreas,

Will osc_io.c:osc_cache_truncate_end() be the correct place to add the changes. What should be the correct approach for this fix. Could you help out here with the correct approach to follow for this? I can have a look at this one and then adapt the same for fallocate.

Thanks

Dec 18 09:48:10 mrpel7 kernel: Call Trace:
Dec 18 09:48:10 mrpel7 kernel: [<ffffffff81633ec4>] dump_stack+0x19/0x1b
Dec 18 09:48:10 mrpel7 kernel: [<ffffffffa0aa5a55>] osc_cache_truncate_end+0x6d5/0xc10 [osc]
Dec 18 09:48:10 mrpel7 kernel: [<ffffffffa08a0bac>] ? ptlrpc_request_bufs_pack+0x1ec/0x480 [ptlrpc]
Dec 18 09:48:10 mrpel7 kernel: [<ffffffffa0a96272>] ? osc_page_gang_lookup+0x272/0x320 [osc]
Dec 18 09:48:10 mrpel7 kernel: [<ffffffffa0a90400>] ? osc_io_data_version_end+0x190/0x190 [osc]
Dec 18 09:48:10 mrpel7 kernel: [<ffffffffa0a8ffe2>] osc_io_setattr_end+0x162/0x1b0 [osc]
Dec 18 09:48:10 mrpel7 kernel: [<ffffffffa0b1b480>] ? lov_io_iter_fini_wrapper+0x50/0x50 [lov]
Dec 18 09:48:10 mrpel7 kernel: [<ffffffffa068b8fd>] cl_io_end+0x5d/0x150 [obdclass]
Dec 18 09:48:10 mrpel7 kernel: [<ffffffffa0b1b55b>] lov_io_end_wrapper+0xdb/0xe0 [lov]
Dec 18 09:48:10 mrpel7 kernel: [<ffffffffa0b1bbb5>] lov_io_call.isra.5+0x85/0x140 [lov]
Dec 18 09:48:10 mrpel7 kernel: [<ffffffffa0b1bca6>] lov_io_end+0x36/0xb0 [lov]
Dec 18 09:48:10 mrpel7 kernel: [<ffffffffa068b8fd>] cl_io_end+0x5d/0x150 [obdclass]
Dec 18 09:48:10 mrpel7 kernel: [<ffffffffa068df8f>] cl_io_loop+0x13f/0xc70 [obdclass]
Dec 18 09:48:10 mrpel7 kernel: [<ffffffffa10ea870>] cl_setattr_ost+0x280/0x3d0 [lustre]
Dec 18 09:48:10 mrpel7 kernel: [<ffffffffa10c4ea5>] ll_setattr_raw+0x1165/0x1270 [lustre]
Dec 18 09:48:10 mrpel7 kernel: [<ffffffffa10c501c>] ll_setattr+0x6c/0xd0 [lustre]
Dec 18 09:48:10 mrpel7 kernel: [<ffffffff811fb6b9>] notify_change+0x279/0x3d0
Dec 18 09:48:10 mrpel7 kernel: [<ffffffff811dc7a3>] do_truncate+0x73/0xc0
Dec 18 09:48:10 mrpel7 kernel: [<ffffffff811e1038>] ? __sb_start_write+0x58/0x110
Dec 18 09:48:10 mrpel7 kernel: [<ffffffff811dcb4c>] do_sys_ftruncate.constprop.17+0x11c/0x180
Dec 18 09:48:10 mrpel7 kernel: [<ffffffff811dcbee>] SyS_ftruncate+0xe/0x10
Dec 18 09:48:10 mrpel7 kernel: [<ffffffff816448c9>] system_call_fastpath+0x16/0x1b

[Andreas Dilger]

Jinshan or Bobijam,
can you please comment on the best approach for this. The client should have the blocks count updated by the reply from OST_PUNCH, since it will currently keep the old (pre-truncate) blocks count in cache until the DLM lock is cancelled, which may be an hour or more.

This may also be a problem for LSOM, since the client's update of the LSOM state on the MDT will not be correct after a truncate if it is caching the wrong data.

[Gerrit Updater]

Arshad Hussain ([e-mail]) uploaded a new patch: https://review.whamcloud.com/31073
Subject: LU-10370 osc: truncate does not update blocks count on client
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 4b5055a464d1b2d190d5b7f49f68f7e4a60653c6

[Arshad Hussain]

Hi Andreas,

Debugging OST_PUNCH found that server does not reply back with updated blocks count to client. Presently OST_PUNCH RPC is processed under lustretre/ofd/ofd_dev.c:ofd_punch_hdl(). Here server does not update repbody with updated block count.

Addressing this allows the client to fetch these values in RPC reply. Under osc_io.c:osc_io_setattr_end() I could update object attributes with updated values received from server calling cl_object_attr_update(). Ive added tests to verify truncate reflects block count correctly for sparse files as well.

We can format and push a patch for review if this looks like a good solution. 

The output of truncate run on Sparse file:
===============================

1. yes 'a' | dd of=/mnt/lustre/sparstest bs=4096 count=2 seek=5
   2+0 records in
   2+0 records out
   8192 bytes (8.2 kB) copied, 0.001071 s, 7.6 MB/s

1. stat /mnt/lustre/sparstest
     File: '/mnt/lustre/sparstest'
     Size: 28672           Blocks: 16         IO Block: 4194304 regular file
   Device: 2c54f966h/743766374d    Inode: 144115205272502281  Links: 1
   Access: (0644/rw-rr-)  Uid: (    0/    root)   Gid: (    0/    root)
   Access: 2018-02-08 18:24:18.000000000 +0530
   Modify: 2018-02-08 18:24:18.000000000 +0530
   Change: 2018-02-08 18:24:18.000000000 +0530
    Birth: -

1. truncate -s24500 /mnt/lustre/sparstest

1. stat /mnt/lustre/sparstest
     File: '/mnt/lustre/sparstest'
     Size: 24500           Blocks: 8          IO Block: 4194304 regular file
   Device: 2c54f966h/743766374d    Inode: 144115205272502281  Links: 1
   Access: (0644/rw-rr-)  Uid: (    0/    root)   Gid: (    0/    root)
   Access: 2018-02-08 18:24:18.000000000 +0530
   Modify: 2018-02-08 18:24:42.000000000 +0530
   Change: 2018-02-08 18:24:42.000000000 +0530
    Birth: -

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/31073/
Subject: LU-10370 ofd: truncate does not update blocks count on client
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 6115eb7fd55abca12c110535be4f777e3eba2b10

[Peter Jones]

Landed for 2.12