[LU-6399] Libcfs socket cleanup

• Type: Improvement
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.12.0
• Labels: patch
• Assignee: Amir Shehata (Inactive)
• Reporter: Alexander Boyko

Description

The ioctl request was designed to get all needed information through socket from usermode. But the same patterns with tricks was used at kernel by linux libcfs. The patch changes this behavior from socket to kernel socket. For libcfs_sock_ioctl tricks with usermode call removed, kernel_sock_ioctl is used instead. But this call handle SIOCGIFADDR and SIOCGIFNETMASK. For SIOCGIFFLAGS we take device flag directly. Function libcfs_ipif_enumerate() which are used SIOCGIFCONF command totally rewriten to get device names without ioctl requests.

Issue Links

• is related to: LU-11893 doesn't handle logical network interface properly.
• is related to: LU-11838 Support linux kernel version 4.18

Comments

[James A Simmons]

See ticket https://jira.whamcloud.com/browse/LU-11838.

Its going to take me a bit to get everything ready.

[Jeremy Filizetti]

Sure

[James A Simmons]

I need to rework this code to support newer kernels (4.18) since kernel_sock_ioctl() has disappeared. Would you be wiling to try those patches.

[Jeremy Filizetti]

Patch e37335b2d8c2dc74badea8978ab3f46a63621265 breaks the ability to use IP address labels similar to the following.  This worked in 2.10:

[root@test-2 ~]# ip addr add 10.10.3.25/16 dev ib0 label ib0:ld
[root@test-2 ~]# modprobe lustre
modprobe: ERROR: could not insert 'lustre': Network is down
[root@test-2 ~]# dmesg
[250263.134595] LNet: HW NUMA nodes: 2, HW CPU cores: 56, npartitions: 2
[250263.138270] alg: No test for adler32 (adler32-zlib)
[250263.954199] Lustre: Lustre: Build Version: 2.12.50_45_g0af247b
[250264.176689] LNet: Using FastReg for registration
[250264.491592] LNet: Added LNI 10.10.3.22@o2ib [128/2048/0/180]
[250264.491629] LNetError: 268850:0:(lib-socket.c:105:lnet_ipif_query()) Can't get flags for interface ib0:ld
[250264.501305] LNetError: 268850:0:(o2iblnd.c:2879:kiblnd_create_dev()) Can't query IPoIB interface ib0:ld: -19
[250264.511224] LNetError: 105-4: Error -100 starting up LNI o2ib
[250265.960442] LNet: Removed LNI 10.10.3.22@o2ib
[250265.960780] LustreError: 268850:0:(events.c:625:ptlrpc_init_portals()) network initialisation failed
[root@test-2 ~]# grep network /etc/modprobe.d/lustre.conf
options lnet networks=o2ib(ib0),o2ib1(ib0:ld),tcp0(eno1)

The -ENODEV comes from lnet/lnet/lib-socket.c line 73:

 46 static int
 47 lnet_sock_ioctl(int cmd, unsigned long arg)
 48 {
...
 62         if (cmd == SIOCGIFFLAGS) {
 63                 /* This cmd is used only to get IFF_UP flag */
 64                 struct ifreq *ifr = (struct ifreq *) arg;
 65                 struct net_device *dev;
 66 
 67                 dev = dev_get_by_name(sock_net(sock->sk), ifr->ifr_name);
 68                 if (dev) {
 69                         ifr->ifr_flags = dev->flags;
 70                         dev_put(dev);
 71                         rc = 0;
 72                 } else {
 73                         rc = -ENODEV;
 74                 }
 75         } else {
 76                 rc = kernel_sock_ioctl(sock, cmd, arg);
 77         }
 78         sock_release(sock);
 79 
 80         return rc;
 81 }

 

 

 

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/14179/
Subject: LU-6399 lnet: socket cleanup
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: e37335b2d8c2dc74badea8978ab3f46a63621265

[James A Simmons]

Amir, Isaac, or Liang can you take a look at this patch.

[Oleg Drokin]

While working in this area, also need to keep in mind how to better tie this into upstream kernel works, there were some run-ups into our use of network ioctls in the past that led to some breakage even, so neatifying that code would be great too.

[Gerrit Updater]

Alexander Boyko ([e-mail]) uploaded a new patch: http://review.whamcloud.com/14179
Subject: LU-6399 libcfs: socket cleanup
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: d6c455bc33245bb6dfc3b594f0df8cd5f064cc00