[#LU-989] Move client's import destruction from disconnect to cleanup phase

[LU-989] Move client's import destruction from disconnect to cleanup phase Created: 13/Jan/12 Updated: 21/Nov/12 Resolved: 05/Apr/12
Status:	Resolved
Project:	Lustre
Component/s:	None
Affects Version/s:	None
Fix Version/s:	Lustre 2.1.2

Type:

Improvement

Priority:

Minor

Reporter:

Andriy Skulysh

Assignee:

WC Triage

Resolution:

Fixed

Votes:

Labels:

None

Rank (Obsolete):

4664

Description

import creates during setup process, but destroyed during disconnect process.It looks wrong.
The patch allows connect usage after disconnect.

Comments

Comment by Andriy Skulysh [ 13/Jan/12 ]

CODE: http://review.whamcloud.com/1963

Comment by Andreas Dilger [ 23/Feb/12 ]

Hi Andriy,
can you please provide some details from the original MRP-288 ticket, such as how this problem was hit or if it was found during code inspection, what problem was seen (e.g. crash, unable to cleanup, etc), and any other information that is relevant.

Comment by Andriy Skulysh [ 23/Feb/12 ]

The original problem is panic during interruption of size=3 obdfilter-survey with Ctrl-C and test restart.
Connect issue was fixed in ~~LU-590~~. Proper obdfilter-survey cleanup on interruption was added in ~~LU-590~~ also.
This patch only prevents possible panics with wrong device operations.
Original crash dump:

BUG: unable to handle kernel NULL pointer dereference at 0000000000000058
IP: [<ffffffffa06644e8>] client_connect_import+0x1a8/0x550 [ptlrpc]
PGD 31ed55067 PUD 30d245067 PMD 0
Oops: 0002 1 SMP
last sysfs file: /sys/devices/pci0000:80/0000:80:05.0/0000:84:00.0/local_cpus
CPU 14
Modules linked in: obdecho(U) lustre(U) obdfilter(U) ost(U) osd_ldiskfs(U) cmm(U) fsfilt_ldiskfs(U) mdt(U) mdd(U) mds(U) mgs(U) ldiskfs(U) mgc(U) lov(U) osc(U) mdc(U) lmv(U) fid(U) fld(U) lquota(U) ptlrpc(U) obdclass(U) lvfs(U) ksocklnd(U) lnet(U) libcfs(U) jbd fuse nfsd lockd nfs_acl auth_rpcgss exportfs autofs4 ipmi_devintf ipmi_si ipmi_msghandler 8021q garp fcoe libfcoe stp libfc scsi_transport_fc scsi_tgt llc sunrpc cpufreq_ondemand acpi_cpufreq freq_table ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm ib_addr ipv6 ib_sa dm_mirror dm_region_hash dm_log uinput mlx4_ib ib_mad ib_core mlx4_en mlx4_core sg igb dca serio_raw ghes hed i2c_i801 i2c_core iTCO_wdt iTCO_vendor_support i7core_edac edac_core shpchp ext4 mbcache jbd2 sd_mod crc_t10dif ata_generic pata_acpi ata_piix dm_mod [last unloaded: libcfs]

Modules linked in: obdecho(U) lustre(U) obdfilter(U) ost(U) osd_ldiskfs(U) cmm(U) fsfilt_ldiskfs(U) mdt(U) mdd(U) mds(U) mgs(U) ldiskfs(U) mgc(U) lov(U) osc(U) mdc(U) lmv(U) fid(U) fld(U) lquota(U) ptlrpc(U) obdclass(U) lvfs(U) ksocklnd(U) lnet(U) libcfs(U) jbd fuse nfsd lockd nfs_acl auth_rpcgss exportfs autofs4 ipmi_devintf ipmi_si ipmi_msghandler 8021q garp fcoe libfcoe stp libfc scsi_transport_fc scsi_tgt llc sunrpc cpufreq_ondemand acpi_cpufreq freq_table ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm ib_addr ipv6 ib_sa dm_mirror dm_region_hash dm_log uinput mlx4_ib ib_mad ib_core mlx4_en mlx4_core sg igb dca serio_raw ghes hed i2c_i801 i2c_core iTCO_wdt iTCO_vendor_support i7core_edac edac_core shpchp ext4 mbcache jbd2 sd_mod crc_t10dif ata_generic pata_acpi ata_piix dm_mod [last unloaded: libcfs]
Pid: 15180, comm: lctl Not tainted 2.6.32-131.2.1.el6_lustre6d10ba94 #1 X8DTH-i/6/iF/6F
RIP: 0010:[<ffffffffa06644e8>] [<ffffffffa06644e8>] client_connect_import+0x1a8/0x550 [ptlrpc]
RSP: 0018:ffff88030fbf39a8 EFLAGS: 00010286
RAX: 6296895f57cebfc0 RBX: ffff88030e390138 RCX: 0000000000000007
RDX: 0000000000000000 RSI: ffff88030e3904e8 RDI: 0000000000000000
RBP: ffff88030fbf3a38 R08: ffff88030e3904e8 R09: 0000000000000000
R10: ffff8803077221c0 R11: ffff88030e390138 R12: 0000000000000000
R13: ffff8803077221c0 R14: ffff8802fc19a7c8 R15: 0000000000000000
FS: 00007fce9695f700(0000) GS:ffff8801c58c0000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 0000000000000058 CR3: 0000000300757000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process lctl (pid: 15180, threadinfo ffff88030fbf2000, task ffff88030080c0c0)
Stack:
ffff88030fbf3a18 ffffffffa05ac6ec ffff88030fbf3a08 0000000000000246
<0> ffff880300001828 ffff880300000050 ffffffffa0ba3e80 ffff88030f56c280
<0> 0000000000000010 ffff88030e390648 6296895f57cebfc0 ffff88030fbf3b48
Call Trace:
[<ffffffffa05ac6ec>] ? lu_site_init+0x28c/0x3b0 [obdclass]
[<ffffffffa0bd7261>] echo_device_alloc+0xb01/0x1710 [obdecho]
[<ffffffffa058f2ef>] obd_setup+0x1ff/0x330 [obdclass]
[<ffffffffa058b4d1>] ? class_handle_hash+0xa1/0x280 [obdclass]
[<ffffffffa058f629>] class_setup+0x209/0xa50 [obdclass]
[<ffffffff8110d4e7>] ? unlock_page+0x27/0x30
[<ffffffffa0575cc6>] ? class_name2dev+0x56/0xd0 [obdclass]
[<ffffffffa05971cc>] class_process_config+0xd6c/0x1fd0 [obdclass]
[<ffffffffa048ca13>] ? cfs_alloc+0x63/0x90 [libcfs]
[<ffffffffa0574004>] class_handle_ioctl+0x21a4/0x2380 [obdclass]
[<ffffffff811382f8>] ? handle_mm_fault+0x1d8/0x2a0
[<ffffffff8117255a>] ? do_sync_read+0xfa/0x140
[<ffffffffa055d313>] obd_class_ioctl+0x53/0x240 [obdclass]
[<ffffffff81184f12>] vfs_ioctl+0x22/0xa0
[<ffffffff811850b4>] do_vfs_ioctl+0x84/0x580
[<ffffffff8107db97>] ? do_sigaction+0x197/0x1d0
[<ffffffff81185631>] sys_ioctl+0x81/0xa0
[<ffffffff8100b172>] system_call_fastpath+0x16/0x1b
Code: 00 00 00 83 83 60 05 00 00 01 48 8d 7d c0 e8 a0 34 f1 ff 49 89 06 48 83 bb 48 01 00 00 00 0f 84 db 02 00 00 48 8b 45 c0 4c 89 ff <49> 89 47 58 e8 1f a8 05 00 85 c0 41 89 c4 0f 85 84 01 00 00 4d
RIP [<ffffffffa06644e8>] client_connect_import+0x1a8/0x550 [ptlrpc]
RSP <ffff88030fbf39a8>
CR2: 0000000000000058
--~~[ end trace b05b7589c7ded66e ]~~--
Kernel panic - not syncing: Fatal exception
Pid: 15180, comm: lctl Tainted: G D ---------------- 2.6.32-131.2.1.el6_lustre6d10ba94 #1
Call Trace:
[<ffffffff814db088>] ? panic+0x78/0x143
[<ffffffff814df0d4>] ? oops_end+0xe4/0x100
[<ffffffff81040cdb>] ? no_context+0xfb/0x260
[<ffffffff81040f65>] ? __bad_area_nosemaphore+0x125/0x1e0
[<ffffffff8104108e>] ? bad_area+0x4e/0x60
[<ffffffff810417b3>] ? __do_page_fault+0x3c3/0x480
[<ffffffffa049a522>] ? cfs_hash_bd_add_locked+0x62/0x90 [libcfs]
[<ffffffffa049b9de>] ? cfs_hash_dual_bd_findadd_locked+0xce/0x100 [libcfs]
[<ffffffffa049a384>] ? cfs_hash_dual_bd_unlock+0x34/0x60 [libcfs]
[<ffffffffa049dd0d>] ? cfs_hash_find_or_add+0x9d/0x190 [libcfs]
[<ffffffff814e109e>] ? do_page_fault+0x3e/0xa0
[<ffffffff814de445>] ? page_fault+0x25/0x30
[<ffffffffa06644e8>] ? client_connect_import+0x1a8/0x550 [ptlrpc]
[<ffffffffa06644d0>] ? client_connect_import+0x190/0x550 [ptlrpc]
[<ffffffffa05ac6ec>] ? lu_site_init+0x28c/0x3b0 [obdclass]
[<ffffffffa0bd7261>] ? echo_device_alloc+0xb01/0x1710 [obdecho]
[<ffffffffa058f2ef>] ? obd_setup+0x1ff/0x330 [obdclass]
[<ffffffffa058b4d1>] ? class_handle_hash+0xa1/0x280 [obdclass]
[<ffffffffa058f629>] ? class_setup+0x209/0xa50 [obdclass]
[<ffffffff8110d4e7>] ? unlock_page+0x27/0x30
[<ffffffffa0575cc6>] ? class_name2dev+0x56/0xd0 [obdclass]
[<ffffffffa05971cc>] ? class_process_config+0xd6c/0x1fd0 [obdclass]
[<ffffffffa048ca13>] ? cfs_alloc+0x63/0x90 [libcfs]
[<ffffffffa0574004>] ? class_handle_ioctl+0x21a4/0x2380 [obdclass]
[<ffffffff811382f8>] ? handle_mm_fault+0x1d8/0x2a0
[<ffffffff8117255a>] ? do_sync_read+0xfa/0x140
[<ffffffffa055d313>] ? obd_class_ioctl+0x53/0x240 [obdclass]
[<ffffffff81184f12>] ? vfs_ioctl+0x22/0xa0
[<ffffffff811850b4>] ? do_vfs_ioctl+0x84/0x580
[<ffffffff8107db97>] ? do_sigaction+0x197/0x1d0
[<ffffffff81185631>] ? sys_ioctl+0x81/0xa0
[<ffffffff8100b172>] ? system_call_fastpath+0x16/0x1b