[LU-15747] UDSP: udsp_single_net_07 failed as "less than expected traffic on the prioritized peer nid"

• Status: Open
• Project: Lustre
• Affects Version/s: Lustre 2.15.0
• Type: Bug
• Priority: Minor
• Reporter: Sarah Liu
• Assignee: Cyril Bordage
• Resolution: Unresolved
• Severity: 3

Description

version=2.15.0_RC2_6_g09fe899

This is from LUTF output

lutf>>> suites['udsp'].scripts['udsp_single_net_07'].run()
('nids: ', ['10.240.43.102@tcp', '10.240.43.109@tcp', '10.240.43.110@tcp', '10.240.43.117@tcp'])
None
(b'udsp:\n    - idx: 0\n      src: [0-255].[0-255].[0-255].[0-255]@tcp\n      dst: 10.240.43.100@tcp\n      rte: NA\n', 0)
([{'local NI(s)': [{'CPT': '[0]', 'dev cpt': -1, 'dropped_stats': {'ack': 0, 'get': 0, 'hello': 0, 'put': 0, 'reply': 0}, 'health stats': {'aborted': 0, 'dropped': 0, 'error': 0, 'fatal_error': 0, 'health value': 1000, 'interrupts': 0, 'next_ping': 0, 'no route': 0, 'ping_count': 0, 'timeouts': 0}, 'lnd tunables': {'conns_per_peer': 1}, 'nid': '10.240.43.100@tcp', 'received_stats': {'ack': 0, 'get': 1, 'hello': 0, 'put': 1, 'reply': 0}, 'sent_stats': {'ack': 0, 'get': 1, 'hello': 0, 'put': 1, 'reply': 0}, 'statistics': {'drop_count': 0, 'recv_count': 2, 'send_count': 2}, 'udsp info': {'net priority': -1, 'nid priority': -1}}, {'CPT': '[0]', 'dev cpt': -1, 'dropped_stats': {'ack': 0, 'get': 0, 'hello': 0, 'put': 0, 'reply': 0}, 'health stats': {'aborted': 0, 'dropped': 0, 'error': 0, 'fatal_error': 0, 'health value': 1000, 'interrupts': 0, 'next_ping': 0, 'no route': 0, 'ping_count': 0, 'timeouts': 0}, 'lnd tunables': {'conns_per_peer': 1}, 'nid': '10.240.43.111@tcp', 'received_stats': {'ack': 1, 'get': 0, 'hello': 0, 'put': 0, 'reply': 0}, 'sent_stats': {'ack': 0, 'get': 0, 'hello': 0, 'put': 1, 'reply': 0}, 'statistics': {'drop_count': 0, 'recv_count': 1, 'send_count': 1}, 'udsp info': {'net priority': -1, 'nid priority': -1}}, {'CPT': '[0]', 'dev cpt': -1, 'dropped_stats': {'ack': 0, 'get': 0, 'hello': 0, 'put': 0, 'reply': 0}, 'health stats': {'aborted': 0, 'dropped': 0, 'error': 0, 'fatal_error': 0, 'health value': 1000, 'interrupts': 0, 'next_ping': 0, 'no route': 0, 'ping_count': 0, 'timeouts': 0}, 'lnd tunables': {'conns_per_peer': 1}, 'nid': '10.240.43.113@tcp', 'received_stats': {'ack': 0, 'get': 0, 'hello': 0, 'put': 0, 'reply': 0}, 'sent_stats': {'ack': 0, 'get': 0, 'hello': 0, 'put': 0, 'reply': 0}, 'statistics': {'drop_count': 0, 'recv_count': 0, 'send_count': 0}, 'udsp info': {'net priority': -1, 'nid priority': -1}}, {'CPT': '[0]', 'dev cpt': -1, 'dropped_stats': {'ack': 0, 'get': 0, 'hello': 0, 'put': 0, 'reply': 0}, 'health stats': {'aborted': 0, 'dropped': 0, 'error': 0, 'fatal_error': 0, 'health value': 1000, 'interrupts': 0, 'next_ping': 0, 'no route': 0, 'ping_count': 0, 'timeouts': 0}, 'lnd tunables': {'conns_per_peer': 1}, 'nid': '10.240.43.115@tcp', 'received_stats': {'ack': 0, 'get': 0, 'hello': 0, 'put': 0, 'reply': 0}, 'sent_stats': {'ack': 0, 'get': 0, 'hello': 0, 'put': 0, 'reply': 0}, 'statistics': {'drop_count': 0, 'recv_count': 0, 'send_count': 0}, 'udsp info': {'net priority': -1, 'nid priority': -1}}], 'net type': 'tcp'}],)
([{'local NI(s)': [{'CPT': '[0]', 'dev cpt': -1, 'dropped_stats': {'ack': 0, 'get': 0, 'hello': 0, 'put': 0, 'reply': 0}, 'health stats': {'aborted': 0, 'dropped': 0, 'error': 0, 'fatal_error': 0, 'health value': 1000, 'interrupts': 0, 'next_ping': 0, 'no route': 0, 'ping_count': 0, 'timeouts': 0}, 'lnd tunables': {'conns_per_peer': 1}, 'nid': '10.240.43.100@tcp', 'received_stats': {'ack': 0, 'get': 9, 'hello': 0, 'put': 1, 'reply': 0}, 'sent_stats': {'ack': 0, 'get': 9, 'hello': 0, 'put': 1, 'reply': 0}, 'statistics': {'drop_count': 0, 'recv_count': 10, 'send_count': 10}, 'udsp info': {'net priority': -1, 'nid priority': -1}}, {'CPT': '[0]', 'dev cpt': -1, 'dropped_stats': {'ack': 0, 'get': 0, 'hello': 0, 'put': 0, 'reply': 0}, 'health stats': {'aborted': 0, 'dropped': 0, 'error': 0, 'fatal_error': 0, 'health value': 1000, 'interrupts': 0, 'next_ping': 0, 'no route': 0, 'ping_count': 0, 'timeouts': 0}, 'lnd tunables': {'conns_per_peer': 1}, 'nid': '10.240.43.111@tcp', 'received_stats': {'ack': 1, 'get': 0, 'hello': 0, 'put': 0, 'reply': 0}, 'sent_stats': {'ack': 0, 'get': 0, 'hello': 0, 'put': 1, 'reply': 0}, 'statistics': {'drop_count': 0, 'recv_count': 1, 'send_count': 1}, 'udsp info': {'net priority': -1, 'nid priority': -1}}, {'CPT': '[0]', 'dev cpt': -1, 'dropped_stats': {'ack': 0, 'get': 0, 'hello': 0, 'put': 0, 'reply': 0}, 'health stats': {'aborted': 0, 'dropped': 0, 'error': 0, 'fatal_error': 0, 'health value': 1000, 'interrupts': 0, 'next_ping': 0, 'no route': 0, 'ping_count': 0, 'timeouts': 0}, 'lnd tunables': {'conns_per_peer': 1}, 'nid': '10.240.43.113@tcp', 'received_stats': {'ack': 0, 'get': 1, 'hello': 0, 'put': 0, 'reply': 0}, 'sent_stats': {'ack': 0, 'get': 1, 'hello': 0, 'put': 0, 'reply': 0}, 'statistics': {'drop_count': 0, 'recv_count': 1, 'send_count': 1}, 'udsp info': {'net priority': -1, 'nid priority': -1}}, {'CPT': '[0]', 'dev cpt': -1, 'dropped_stats': {'ack': 0, 'get': 0, 'hello': 0, 'put': 0, 'reply': 0}, 'health stats': {'aborted': 0, 'dropped': 0, 'error': 0, 'fatal_error': 0, 'health value': 1000, 'interrupts': 0, 'next_ping': 0, 'no route': 0, 'ping_count': 0, 'timeouts': 0}, 'lnd tunables': {'conns_per_peer': 1}, 'nid': '10.240.43.115@tcp', 'received_stats': {'ack': 0, 'get': 1, 'hello': 0, 'put': 0, 'reply': 0}, 'sent_stats': {'ack': 0, 'get': 1, 'hello': 0, 'put': 0, 'reply': 0}, 'statistics': {'drop_count': 0, 'recv_count': 1, 'send_count': 1}, 'udsp info': {'net priority': -1, 'nid priority': -1}}], 'net type': 'tcp'}],)
({0: 2, 1: 1, 2: 0, 3: 0}, {0: 10, 1: 1, 2: 1, 3: 1})
('less than expected traffic on the prioritized peer nid',)

According to UDSP test plan, single_net_07 tests

Setup: configure single network, 3 NIDs on the network, 3 NIDs on the local peer
Add UDSP rule that creates 3 NID pairs such that one of the peer NIDs is not in any pair
Start traffic
Stop traffic
Verify that the peer NID that is not part of any pair was not used (less used?)
Delete UDSP rule
Start traffic
Stop traffic
Verify that all NIDs are used