Skip to content

Commit

Permalink
snappi_tests/multidut: Fixturize a few cases in pfc folder. (#15919)
Browse files Browse the repository at this point in the history 
Description of PR
In continuation of reducing the time to run for pfc cases, this PR refactors some testcases from pfc folder to use the new fixtures: setup_ports_and_dut and disable_pfcwd. This reduces the runtime for each of these cases by 4-5 minutes, by avoiding the final config-reload from the sanity-checker-teardown kicking in.

Approach
What is the motivation for this PR?
to reduce the runtime of the testcases.
Reuse the code available in fixtures instead of writing same code in multiple cases.
How did you do it?
By removing the repeat code, and using the already established fixtures.

How did you verify/test it?
Ran it on my TB, all of them have passed.

=============================================================================================== PASSES ===============================================================================================

_______________________________________________________________ test_pfc_pause_single_lossless_prio[multidut_port_info0-yy39top-lc4|3] _______________________________________________________________
_______________________________________________________________ test_pfc_pause_single_lossless_prio[multidut_port_info0-yy39top-lc4|4] _______________________________________________________________
_______________________________________________________________ test_pfc_pause_single_lossless_prio[multidut_port_info1-yy39top-lc4|3] _______________________________________________________________
_______________________________________________________________ test_pfc_pause_single_lossless_prio[multidut_port_info1-yy39top-lc4|4] _______________________________________________________________
------------------------------------------------------ generated xml file: /run_logs/ixia/rerun/2024-12-05-05-25-18/tr_2024-12-05-05-25-18.xml -------------------------------------------------------
INFO:root:Can not get Allure report URL. Please check logs
--------------------------------------------------------------------------------------- live log sessionfinish ---------------------------------------------------------------------------------------
05:59:34 __init__.pytest_terminal_summary         L0067 INFO   | Can not get Allure report URL. Please check logs
====================================================================================== short test summary info =======================================================================================
PASSED snappi_tests/multidut/pfc/test_multidut_pfc_pause_lossless_with_snappi.py::test_pfc_pause_single_lossless_prio[multidut_port_info0-yy39top-lc4|3]
PASSED snappi_tests/multidut/pfc/test_multidut_pfc_pause_lossless_with_snappi.py::test_pfc_pause_single_lossless_prio[multidut_port_info0-yy39top-lc4|4]
PASSED snappi_tests/multidut/pfc/test_multidut_pfc_pause_lossless_with_snappi.py::test_pfc_pause_single_lossless_prio[multidut_port_info1-yy39top-lc4|3]
PASSED snappi_tests/multidut/pfc/test_multidut_pfc_pause_lossless_with_snappi.py::test_pfc_pause_single_lossless_prio[multidut_port_info1-yy39top-lc4|4]
============================================================================= 4 passed, 7 warnings in 2054.37s (0:34:14) =============================================================================
sonic@snappi-sonic-mgmt-vanilla-202405-t2:/data/tests$ 


 m2o_fluctuating_lossless:
--------------------------------------------------------------------------------------- live log sessionfinish ---------------------------------------------------------------------------------------
06:41:29 __init__.pytest_terminal_summary         L0067 INFO   | Can not get Allure report URL. Please check logs
====================================================================================== short test summary info =======================================================================================
PASSED snappi_tests/multidut/pfc/test_m2o_fluctuating_lossless.py::test_m2o_fluctuating_lossless[multidut_port_info0]
PASSED snappi_tests/multidut/pfc/test_m2o_fluctuating_lossless.py::test_m2o_fluctuating_lossless[multidut_port_info1]
============================================================================= 2 passed, 5 warnings in 410.73s (0:06:50) ==============================================================================
sonic@snappi-sonic-mgmt-vanilla-202405-t2:/data/tests$ 
9. snappi_tests/multidut/pfc/test_m2o_fluctuating_lossless.py  test_m2o_fluctuating_lossless[multidut_port_info0]
--------------------------------------------------------------------------------------- live log sessionfinish ---------------------------------------------------------------------------------------
06:55:40 __init__.pytest_terminal_summary         L0067 INFO   | Can not get Allure report URL. Please check logs
====================================================================================== short test summary info =======================================================================================
PASSED snappi_tests/multidut/pfc/test_lossless_response_to_external_pause_storms.py::test_lossless_response_to_external_pause_storms_test[multidut_port_info0]
PASSED snappi_tests/multidut/pfc/test_lossless_response_to_external_pause_storms.py::test_lossless_response_to_external_pause_storms_test[multidut_port_info1]
============================================================================= 2 passed, 5 warnings in 424.78s (0:07:04) ==============================================================================
sonic@snappi-sonic-mgmt-vanilla-202405-t2:/data/tests$ 

co-authorized by: jianquanye@microsoft.com
  • Loading branch information
@rraghav-cisco @mssonicbld
rraghav-cisco authored and mssonicbld committed Jan 2, 2025
1 parent 2854c74 commit 0c32259
Show file tree
Hide file tree
Showing 6 changed files with 62 additions and 181 deletions.
41 changes: 11 additions & 30 deletions tests/snappi_tests/multidut/pfc/test_lossless_response_to_external_pause_storms.py
View file
Original file line number Diff line number Diff line change
@@ -1,15 +1,14 @@
import pytest
import logging
from tests.common.helpers.assertions import pytest_assert
from tests.common.fixtures.conn_graph_facts import conn_graph_facts, fanout_graph_facts, \
fanout_graph_facts_multidut # noqa: F401
from tests.common.snappi_tests.snappi_fixtures import snappi_api_serv_ip, snappi_api_serv_port, \
get_snappi_ports_single_dut, snappi_testbed_config, \
get_snappi_ports_multi_dut, is_snappi_multidut, \
snappi_api, snappi_dut_base_config, get_snappi_ports, get_snappi_ports_for_rdma, cleanup_config # noqa: F401
from tests.common.snappi_tests.qos_fixtures import prio_dscp_map, \
lossless_prio_list # noqa: F401
from tests.snappi_tests.variables import MULTIDUT_PORT_INFO, MULTIDUT_TESTBED
lossless_prio_list, disable_pfcwd # noqa: F401
from tests.snappi_tests.files.helper import multidut_port_info, setup_ports_and_dut, reboot_duts # noqa: F401
from tests.snappi_tests.multidut.pfc.files.lossless_response_to_external_pause_storms_helper import (
run_lossless_response_to_external_pause_storms_test,
)
Expand All @@ -18,7 +17,11 @@
pytestmark = [pytest.mark.topology('multidut-tgen', 'tgen')]


@pytest.mark.parametrize("multidut_port_info", MULTIDUT_PORT_INFO[MULTIDUT_TESTBED])
@pytest.fixture(autouse=True)
def number_of_tx_rx_ports():
yield (2, 1)


def test_lossless_response_to_external_pause_storms_test(snappi_api, # noqa: F811
conn_graph_facts, # noqa: F811
fanout_graph_facts_multidut, # noqa: F811
Expand All @@ -27,8 +30,9 @@ def test_lossless_response_to_external_pause_storms_test(snappi_api,
lossless_prio_list, # noqa: F811
tbinfo, # noqa: F811
get_snappi_ports, # noqa: F811
multidut_port_info,
): # noqa: F811
setup_ports_and_dut, # noqa: F811
disable_pfcwd # noqa: F811
):
"""
Run PFC lossless response to external pause storm with many to one traffic pattern
Expand Down Expand Up @@ -57,31 +61,8 @@ def test_lossless_response_to_external_pause_storms_test(snappi_api,
Returns:
N/A
"""
for testbed_subtype, rdma_ports in multidut_port_info.items():
tx_port_count = 2
rx_port_count = 1
snappi_port_list = get_snappi_ports
pytest_assert(len(snappi_port_list) >= tx_port_count + rx_port_count,
"Need Minimum of 3 ports defined in ansible/files/*links.csv file")

pytest_assert(len(rdma_ports['tx_ports']) >= tx_port_count,
'MULTIDUT_PORT_INFO doesn\'t have the required Tx ports defined for \
testbed {}, subtype {} in variables.py'.
format(MULTIDUT_TESTBED, testbed_subtype))
testbed_config, port_config_list, snappi_ports = setup_ports_and_dut

pytest_assert(len(rdma_ports['rx_ports']) >= rx_port_count,
'MULTIDUT_PORT_INFO doesn\'t have the required Rx ports defined for \
testbed {}, subtype {} in variables.py'.
format(MULTIDUT_TESTBED, testbed_subtype))
logger.info('Running test for testbed subtype: {}'.format(testbed_subtype))
if is_snappi_multidut(duthosts):
snappi_ports = get_snappi_ports_for_rdma(snappi_port_list, rdma_ports,
tx_port_count, rx_port_count, MULTIDUT_TESTBED)
else:
snappi_ports = get_snappi_ports
testbed_config, port_config_list, snappi_ports = snappi_dut_base_config(duthosts,
snappi_ports,
snappi_api)
all_prio_list = prio_dscp_map.keys()
test_prio_list = lossless_prio_list
pause_prio_list = test_prio_list
Expand Down
40 changes: 10 additions & 30 deletions tests/snappi_tests/multidut/pfc/test_lossless_response_to_throttling_pause_storms.py
View file
Original file line number Diff line number Diff line change
@@ -1,24 +1,27 @@
import pytest
import logging
from tests.common.helpers.assertions import pytest_assert
from tests.common.fixtures.conn_graph_facts import conn_graph_facts, fanout_graph_facts, \
fanout_graph_facts_multidut # noqa: F401
from tests.common.snappi_tests.snappi_fixtures import snappi_api_serv_ip, snappi_api_serv_port, \
get_snappi_ports_single_dut, snappi_testbed_config, \
get_snappi_ports_multi_dut, is_snappi_multidut, \
snappi_api, snappi_dut_base_config, get_snappi_ports, get_snappi_ports_for_rdma, cleanup_config # noqa: F401
from tests.common.snappi_tests.qos_fixtures import prio_dscp_map, \
lossless_prio_list # noqa: F401
from tests.snappi_tests.variables import MULTIDUT_PORT_INFO, MULTIDUT_TESTBED
lossless_prio_list, disable_pfcwd # noqa: F401
from tests.snappi_tests.multidut.pfc.files.lossless_response_to_throttling_pause_storms_helper import (
run_lossless_response_to_throttling_pause_storms_test)
from tests.common.snappi_tests.snappi_test_params import SnappiTestParams
from tests.snappi_tests.files.helper import setup_ports_and_dut, multidut_port_info # noqa: F401
from tests.common.snappi_tests.variables import pfcQueueGroupSize, pfcQueueValueDict # noqa: F401
logger = logging.getLogger(__name__)
pytestmark = [pytest.mark.topology('multidut-tgen', 'tgen')]


@pytest.mark.parametrize("multidut_port_info", MULTIDUT_PORT_INFO[MULTIDUT_TESTBED])
@pytest.fixture(autouse=True)
def number_of_tx_rx_ports():
yield (2, 1)


def test_lossless_response_to_throttling_pause_storms(snappi_api, # noqa: F811
conn_graph_facts, # noqa: F811
fanout_graph_facts_multidut, # noqa: F811
Expand All @@ -27,7 +30,8 @@ def test_lossless_response_to_throttling_pause_storms(snappi_api,
lossless_prio_list, # noqa: F811
get_snappi_ports, # noqa: F811
tbinfo, # noqa: F811
multidut_port_info): # noqa: F811
disable_pfcwd, # noqa: F811
setup_ports_and_dut): # noqa: F811

"""
Run PFC lossless response to throttling pause storms
Expand Down Expand Up @@ -59,31 +63,7 @@ def test_lossless_response_to_throttling_pause_storms(snappi_api,
Returns:
N/A
"""
for testbed_subtype, rdma_ports in multidut_port_info.items():
tx_port_count = 2
rx_port_count = 1
snappi_port_list = get_snappi_ports
pytest_assert(len(snappi_port_list) >= tx_port_count + rx_port_count,
"Need Minimum of 3 ports defined in ansible/files/*links.csv file")

pytest_assert(len(rdma_ports['tx_ports']) >= tx_port_count,
'MULTIDUT_PORT_INFO doesn\'t have the required Tx ports defined for \
testbed {}, subtype {} in variables.py'.
format(MULTIDUT_TESTBED, testbed_subtype))

pytest_assert(len(rdma_ports['rx_ports']) >= rx_port_count,
'MULTIDUT_PORT_INFO doesn\'t have the required Rx ports defined for \
testbed {}, subtype {} in variables.py'.
format(MULTIDUT_TESTBED, testbed_subtype))
logger.info('Running test for testbed subtype: {}'.format(testbed_subtype))
if is_snappi_multidut(duthosts):
snappi_ports = get_snappi_ports_for_rdma(snappi_port_list, rdma_ports,
tx_port_count, rx_port_count, MULTIDUT_TESTBED)
else:
snappi_ports = get_snappi_ports
testbed_config, port_config_list, snappi_ports = snappi_dut_base_config(duthosts,
snappi_ports,
snappi_api)
testbed_config, port_config_list, snappi_ports = setup_ports_and_dut

all_prio_list = prio_dscp_map.keys()
test_prio_list = lossless_prio_list
Expand Down
40 changes: 10 additions & 30 deletions tests/snappi_tests/multidut/pfc/test_m2o_fluctuating_lossless.py
View file
Original file line number Diff line number Diff line change
@@ -1,22 +1,25 @@
import pytest
import logging
from tests.common.helpers.assertions import pytest_assert
from tests.common.fixtures.conn_graph_facts import conn_graph_facts, fanout_graph_facts, \
fanout_graph_facts_multidut # noqa: F401
from tests.common.snappi_tests.snappi_fixtures import snappi_api_serv_ip, snappi_api_serv_port, \
get_snappi_ports_single_dut, snappi_testbed_config, \
get_snappi_ports_multi_dut, is_snappi_multidut, \
snappi_api, snappi_dut_base_config, get_snappi_ports, get_snappi_ports_for_rdma, cleanup_config # noqa: F401
from tests.common.snappi_tests.qos_fixtures import prio_dscp_map, \
lossless_prio_list # noqa: F401
from tests.snappi_tests.variables import MULTIDUT_PORT_INFO, MULTIDUT_TESTBED
lossless_prio_list, disable_pfcwd # noqa: F401
from tests.snappi_tests.files.helper import multidut_port_info, setup_ports_and_dut # noqa: F401
from tests.snappi_tests.multidut.pfc.files.m2o_fluctuating_lossless_helper import run_m2o_fluctuating_lossless_test
from tests.common.snappi_tests.snappi_test_params import SnappiTestParams
logger = logging.getLogger(__name__)
pytestmark = [pytest.mark.topology('multidut-tgen', 'tgen')]


@pytest.mark.parametrize("multidut_port_info", MULTIDUT_PORT_INFO[MULTIDUT_TESTBED])
@pytest.fixture(autouse=True)
def number_of_tx_rx_ports():
yield (2, 1)


def test_m2o_fluctuating_lossless(snappi_api, # noqa: F811
conn_graph_facts, # noqa: F811
fanout_graph_facts_multidut, # noqa: F811
Expand All @@ -25,7 +28,8 @@ def test_m2o_fluctuating_lossless(snappi_api, # noqa: F811
lossless_prio_list, # noqa: F811
get_snappi_ports, # noqa: F811
tbinfo, # noqa: F811
multidut_port_info): # noqa: F811
disable_pfcwd, # noqa: F811
setup_ports_and_dut): # noqa: F811

"""
Run PFC Fluctuating Lossless Traffic Congestion with many to one traffic pattern
Expand Down Expand Up @@ -56,31 +60,7 @@ def test_m2o_fluctuating_lossless(snappi_api, # noqa: F811
Returns:
N/A
"""
for testbed_subtype, rdma_ports in multidut_port_info.items():
tx_port_count = 2
rx_port_count = 1
snappi_port_list = get_snappi_ports
pytest_assert(len(snappi_port_list) >= tx_port_count + rx_port_count,
"Need Minimum of 3 ports defined in ansible/files/*links.csv file")

pytest_assert(len(rdma_ports['tx_ports']) >= tx_port_count,
'MULTIDUT_PORT_INFO doesn\'t have the required Tx ports defined for \
testbed {}, subtype {} in variables.py'.
format(MULTIDUT_TESTBED, testbed_subtype))

pytest_assert(len(rdma_ports['rx_ports']) >= rx_port_count,
'MULTIDUT_PORT_INFO doesn\'t have the required Rx ports defined for \
testbed {}, subtype {} in variables.py'.
format(MULTIDUT_TESTBED, testbed_subtype))
logger.info('Running test for testbed subtype: {}'.format(testbed_subtype))
if is_snappi_multidut(duthosts):
snappi_ports = get_snappi_ports_for_rdma(snappi_port_list, rdma_ports,
tx_port_count, rx_port_count, MULTIDUT_TESTBED)
else:
snappi_ports = get_snappi_ports
testbed_config, port_config_list, snappi_ports = snappi_dut_base_config(duthosts,
snappi_ports,
snappi_api)
testbed_config, port_config_list, snappi_ports = setup_ports_and_dut

all_prio_list = prio_dscp_map.keys()
test_prio_list = lossless_prio_list
Expand Down
40 changes: 10 additions & 30 deletions tests/snappi_tests/multidut/pfc/test_m2o_oversubscribe_lossless.py
View file
Original file line number Diff line number Diff line change
@@ -1,24 +1,27 @@
import pytest
import logging
from tests.common.helpers.assertions import pytest_assert
from tests.common.fixtures.conn_graph_facts import conn_graph_facts, fanout_graph_facts, \
fanout_graph_facts_multidut # noqa: F401
from tests.common.snappi_tests.snappi_fixtures import snappi_api_serv_ip, snappi_api_serv_port, \
get_snappi_ports_single_dut, snappi_testbed_config, \
get_snappi_ports_multi_dut, is_snappi_multidut, \
snappi_api, snappi_dut_base_config, get_snappi_ports, get_snappi_ports_for_rdma, cleanup_config # noqa: F401
from tests.common.snappi_tests.qos_fixtures import prio_dscp_map, \
lossless_prio_list # noqa: F401
from tests.snappi_tests.variables import MULTIDUT_PORT_INFO, MULTIDUT_TESTBED
lossless_prio_list, disable_pfcwd # noqa: F401
from tests.snappi_tests.multidut.pfc.files.m2o_oversubscribe_lossless_helper import (
run_m2o_oversubscribe_lossless_test
)
from tests.common.snappi_tests.snappi_test_params import SnappiTestParams
from tests.snappi_tests.files.helper import setup_ports_and_dut, multidut_port_info # noqa: F401
logger = logging.getLogger(__name__)
pytestmark = [pytest.mark.topology('multidut-tgen', 'tgen')]


@pytest.mark.parametrize("multidut_port_info", MULTIDUT_PORT_INFO[MULTIDUT_TESTBED])
@pytest.fixture(autouse=True)
def number_of_tx_rx_ports():
yield (2, 1)


def test_m2o_oversubscribe_lossless(snappi_api, # noqa: F811
conn_graph_facts, # noqa: F811
fanout_graph_facts_multidut, # noqa: F811
Expand All @@ -27,7 +30,8 @@ def test_m2o_oversubscribe_lossless(snappi_api, # n
lossless_prio_list, # noqa: F811
get_snappi_ports, # noqa: F811
tbinfo,
multidut_port_info): # noqa: F811
disable_pfcwd, # noqa: F811
setup_ports_and_dut): # noqa: F811

"""
Run PFC oversubsription lossless for many to one traffic pattern
Expand Down Expand Up @@ -56,31 +60,7 @@ def test_m2o_oversubscribe_lossless(snappi_api, # n
Returns:
N/A
"""
for testbed_subtype, rdma_ports in multidut_port_info.items():
tx_port_count = 2
rx_port_count = 1
snappi_port_list = get_snappi_ports
pytest_assert(len(snappi_port_list) >= tx_port_count + rx_port_count,
"Need Minimum of 3 ports defined in ansible/files/*links.csv file")

pytest_assert(len(rdma_ports['tx_ports']) >= tx_port_count,
'MULTIDUT_PORT_INFO doesn\'t have the required Tx ports defined for \
testbed {}, subtype {} in variables.py'.
format(MULTIDUT_TESTBED, testbed_subtype))

pytest_assert(len(rdma_ports['rx_ports']) >= rx_port_count,
'MULTIDUT_PORT_INFO doesn\'t have the required Rx ports defined for \
testbed {}, subtype {} in variables.py'.
format(MULTIDUT_TESTBED, testbed_subtype))
logger.info('Running test for testbed subtype: {}'.format(testbed_subtype))
if is_snappi_multidut(duthosts):
snappi_ports = get_snappi_ports_for_rdma(snappi_port_list, rdma_ports,
tx_port_count, rx_port_count, MULTIDUT_TESTBED)
else:
snappi_ports = get_snappi_ports
testbed_config, port_config_list, snappi_ports = snappi_dut_base_config(duthosts,
snappi_ports,
snappi_api)
testbed_config, port_config_list, snappi_ports = setup_ports_and_dut

all_prio_list = prio_dscp_map.keys()
test_prio_list = lossless_prio_list
Expand Down
Loading

0 comments on commit 0c32259

Please sign in to comment.