[LU-2626] lustre_rsync produces only "/.lustrerepl" folder structure

• Status: Resolved
• Project: Lustre
• Affects Version/s: Lustre 2.3.0, Lustre 2.4.0, Lustre 2.1.4
• Fix Version/s: Lustre 2.5.0
• Type: Bug
• Priority: Critical
• Reporter: Iakov Pustilnik
• Assignee: WC Triage
• Resolution: Fixed

Environment

Linux 2.6.32-279.14.1.el6_lustre.x86_64 #1 SMP

• Severity: 3

Description

I have Lustre 2.1.4, RHEL 6.3 with two separate Lustre filesystems (mount points /array1 & /arra2).

[root@n02 ~]# lfs df -h
UUID bytes Used Available Use% Mounted on
lustre1-MDT0000_UUID 69.1G 460.6M 68.7G 1% /array1[MDT:0]
lustre1-OST0000_UUID 1.8T 467.6M 1.8T 0% /array1[OST:0]

filesystem summary: 1.8T 467.6M 1.8T 0% /array1

UUID bytes Used Available Use% Mounted on
lustre2-MDT0000_UUID 69.1G 460.5M 68.7G 1% /array2[MDT:0]
lustre2-OST0000_UUID 1.8T 467.6M 1.8T 0% /array2[OST:0]

filesystem summary: 1.8T 467.6M 1.8T 0% /array2

These are the commands:

[root@n02 ~]# lctl --device lustre1-MDT0000 changelog_register
lustre1-MDT0000: Registered changelog userid 'cl1'

[root@n02 ~]# lustre_rsync --source=/array1 --target=/array2 --mdt=lustre1-MDT0000 --user=cl1 --statuslog sync.log --verbose
Lustre filesystem: lustre1
MDT device: lustre1-MDT0000
Source: /array1
Target: /array2
Statuslog: sync.log
Changelog registration: cl1
Starting changelog record: 0
Errors: 0
lustre_rsync took 0 seconds
Changelog records consumed: 1

Next i put some data to /array1:

[root@n02 ~]# cp rpmbuild/ /array1/ -r

[root@n02 ~]# lustre_rsync --statuslog sync.log --verbose
Lustre filesystem: lustre1
MDT device: lustre1-MDT0000
Source: /array1
Target: /array2
Statuslog: sync.log
Changelog registration: cl1
Starting changelog record: 2
Errors: 0
lustre_rsync took 1 seconds
Changelog records consumed: 8

[root@n02 ~]# la /array1/
total 1048600
drwxr-xr-x 4 root root 4096 Jan 16 17:33 .
dr-xr-xr-x. 28 root root 4096 Jan 16 15:53 ..
rw-rr- 1 root root 1073741824 Jan 16 17:29 1G
drwxr-xr-x 2 root root 4096 Jan 16 17:26 .lustre
drwxr-xr-x 4 root root 4096 Jan 16 17:33 rpmbuild

[root@n02 ~]# la /array2/
total 1048600
drwxr-xr-x 4 root root 4096 Jan 16 17:32 .
dr-xr-xr-x. 28 root root 4096 Jan 16 15:53 ..
rw-rr- 1 root root 1073741824 Jan 16 17:29 1G
drwxr-xr-x 2 root root 4096 Jan 16 17:26 .lustre
drwxr-xr-x 5 root root 4096 Jan 16 17:33 .lustrerepl

[root@n02 ~]# la /array2/.lustrerepl/
total 212
drwxr-xr-x 5 root root 4096 Jan 16 17:33 .
drwxr-xr-x 4 root root 4096 Jan 16 17:32 ..
drwxr-xr-x 2 root root 4096 Jan 16 17:33 [0x200000400:0x2:0x0]
drwxr-xr-x 2 root root 4096 Jan 16 17:33 [0x200000400:0x3:0x0]
rw-rr- 1 root root 74 Jan 16 17:33 [0x200000400:0x4:0x0]
rw-rr- 1 root root 182554 Jan 16 17:33 [0x200000400:0x5:0x0]
rw-rr- 1 root root 49 Jan 16 17:33 [0x200000400:0x6:0x0]
drwxr-xr-x 2 root root 4096 Jan 16 17:33 [0x200000400:0x7:0x0]
rw-rr- 1 root root 1276 Jan 16 17:33 [0x200000400:0x8:0x0]

Second filesystem /array2 contains only /.lustrerepl folder structure, but not the original files.

Comments

[Iakov Pustilnik]

Tried with Lustre 2.3.0 - it's the same.

Maybe i'm doing something wrong? Does anybody here has succeed with lustre_rsync at all?

[Iakov Pustilnik]

for me a bugfix for this error is quite urgent.

i've downloaded and compiled master:HEAD from git, but i'm quite new to lustre. so if you could give me some directions where to start & what could be possible reason for this error - i can try fixin' it by myself...

[Andreas Dilger]

There is ongoing lustre-rsync testing for Lustre:

https://maloo.whamcloud.com/test_sets/query?utf8=%E2%9C%93&test_set[test_set_script_id]=a980fbe8-51b8-11e0-bb3d-52540025f9af&test_set[status]=&test_set[query_bugs]=&test_session[test_host]=&test_session[test_group]=full&test_session[user_id]=&test_session[query_date]=&test_session[query_recent_period]=2419200&test_node[os_type_id]=&test_node[distribution_type_id]=&test_node[architecture_type_id]=&test_node[file_system_type_id]=&test_node[lustre_branch_id]=&test_node_network[network_type_id]=&commit=Update+results

though these tests are only run once per day, and not on every patch that is landed. There does appear to be a problem with interoperability testing with the current development branch and older releases, but this will be addressed before the final 2.4 release is made.

I've submitted a patch to enable all of the current lustre-rsync tests to run for every patch, which will not fix your specific problem, but at least will ensure that we get better testing coverage of this feature.
http://review.whamcloud.com/5214