PMP Verif Plan and tests #2648

OlivierBetschi · 2024-12-04T11:16:10Z

Verification Plan provided in VP_TOOL for the PMP. The verification plan should be complete, however only a partial set of the tests is available. This is not included in the CI but a bash script is available to run the test. Replacement of #2457

JeanRochCoulon · 2024-12-04T15:38:16Z

@OlivierBetschi can you rebase ?

JeanRochCoulon · 2024-12-04T15:44:54Z

You should create a job in CI to run this test, that's a way to be sure they pass before merging.

JeanRochCoulon · 2024-12-04T15:46:19Z

@valentinThomazic do you approve the DVPlan ? and in case of need, could you help @OlivierBetschi to define a new job in CI.

github-actions · 2024-12-04T16:34:53Z

✔️ successful run, report available here.

valentinThomazic · 2024-12-05T14:42:13Z

Some comments on the test script @OlivierBetschi :

linker script path is wrong (should probably be : ../../config/gen_from_riscv_config/cv32a65x/linker/link.ld )
it would be great to be able to run all the tests simultaneously
it is not needed to install all the test suites
uvm_test arg does not exist on master cva6.py

valentinThomazic · 2024-12-05T14:55:33Z

To get the result of the CI execution of the PMP tests directly on the PR pipeline report, please add the following job to the gitlab-ci @OlivierBetschi :

pmp_tests_cv32a65x:
  extends:
    - .fe_smoke_test
  variables:
    DASHBOARD_JOB_TITLE: "[DEBUG! TO REMOVE FROM LIGHTS TESTS] PMP cv32a65x"
    DASHBOARD_JOB_DESCRIPTION: "PMP regression tests"
    DASHBOARD_SORT_INDEX: 0
    DASHBOARD_JOB_CATEGORY: "Basic"
    DV_SIMULATORS: "vcs-uvm"
    SPIKE_TANDEM: 1
  script: bash verif/regress/pmp_cv32a65x_tests.sh
  after_script: *simu_after_script

OlivierBetschi · 2024-12-05T16:04:00Z

Updated for review, but most only 2 tests are passing. I will have to debug this before the PR is merged. @valentinThomazic could you still have a look at the changes ? Thanks

github-actions · 2024-12-05T16:09:28Z

❌ failed run, report available here.

valentinThomazic · 2024-12-05T16:10:40Z

.gitlab-ci.yml

+    DASHBOARD_JOB_DESCRIPTION: "PMP regression tests"
+    DASHBOARD_SORT_INDEX: 0
+    DASHBOARD_JOB_CATEGORY: "Basic"
+    DV_SIMULATORS: "vcs-uvm"


Suggested change

DV_SIMULATORS: "vcs-uvm"

DV_SIMULATORS: "vcs-uvm"

COLLECT_SIMU_LOGS: 1

to have the logfiles on the dashboard

Can you add "SPIKE_TANDEM: 1" to enable tandem and collect the ci results ?

valentinThomazic · 2024-12-05T16:11:25Z

verif/regress/pmp_cv32a65x_tests.sh

@@ -0,0 +1,296 @@
+# Copyright 2024 Thales DIS design services SAS


You might want to change this

valentinThomazic · 2024-12-05T16:12:18Z

verif/regress/pmp_cv32a65x_tests.sh

+# SPDX-License-Identifier: Apache-2.0 WITH SHL-2.0
+# You may obtain a copy of the License at https://solderpad.org/licenses/
+#
+# Original Author: Jean-Roch COULON - Thales


This should be changed

valentinThomazic · 2024-12-05T16:32:48Z

verif/sim/cva6.py

@@ -954,6 +961,27 @@ def load_config(args, cwd):

  args.spike_params = get_full_spike_param_args(args.spike_params) if args.spike_params else ""

+def edit_Makefile(UVM_TESTNAME="uvmt_cva6_firmware_test_c"):


I think it would be better to pass the uvm test to the Makefile as an env var instead of rewriting it at each simulation. This is how it is done for such things like the test path.
This is how you can do this :

add a uvm_test parameter to the run_test function in cva6.py (I admit that this function has too much parameters...)

also add it in get_iss_cmd function

substitute it in the cmd like it is done for the elf path for instance

add a uvm_test env var for the uvm targets in verif/sim/cva6.yaml

set a default value in verif/sim/Makefile for the environment var uvm_test

replace UVM_TESTNAME=uvmt_qqchose with UVM_TESTNAME=$(uvm_test) in verif/sim/Makefile

Do not hesitate to ask for more details if needed

I guess we could call each others if you think this could speed up things

@OlivierBetschi I wonder why you use a different uvm test than "uvmt_cva6_firmware_test_c" ?
@AnouarZajni, can you help us to understand what is the best solution ?

The different uvm_test came from the need to disable the cache for the PMP testing (that is my understanding at least).

valentinThomazic · 2024-12-05T16:43:07Z

the uvm test is missing from the pr hence the fail

github-actions · 2024-12-06T10:58:26Z

❌ failed run, report available here.

github-actions · 2024-12-11T14:26:42Z

❌ failed run, report available here.

OlivierBetschi · 2024-12-11T15:04:06Z

Hello @JeanRochCoulon @valentinThomazic , I updated the PR : all tests are now passing with vcs-uvm on my side. The change of the uvm_test was not necessary after all. Just let me know what I should update for the gitlab ci.

valentinThomazic · 2024-12-12T22:18:13Z

@OlivierBetschi there isn't any report on the dashboard because on your latest pipeline, there isn't any comparison made between spike and the rtl simulation. I see that you disabled spike tandem, this is what causes the issue.

…ci, update the sh script to run all the tests and add the missing parameter in cva6.py

…ry to switch to a different firmware test

github-actions · 2024-12-13T10:22:03Z

❌ failed run, report available here.

JeanRochCoulon · 2024-12-16T10:29:25Z

Hello @OlivierBetschi
5 PMP tests are failed. As some features (as NAPOT) are not supported on cv32a65x configuration, I wonder whether some of these tests should be pass.

OlivierBetschi requested review from jquevremont, JeanRochCoulon and zarubaf as code owners December 4, 2024 11:16

OlivierBetschi mentioned this pull request Dec 4, 2024

[Pmp] Verif plan TXT #2457

Closed

OlivierBetschi force-pushed the pmp/verif_plan_txt branch from 3e953dc to 2ab9029 Compare December 4, 2024 15:44

valentinThomazic suggested changes Dec 5, 2024

View reviewed changes

dat9val and others added 11 commits December 13, 2024 10:41

draft verif plan for PMP

1d24c02

added PMP Verif Features and Verif Plan Draft

3e4c3d2

added priorities on some high-priority tests

324ec7c

added priorities on some high-priority tests

62b1c3c

Add PMP verif plan integrated in VP tool

6890893

Add dvplan_PMP.md

149db8b

Add PMP tests

4ad5ff2

Remove custom PMP test config

f7ab662

Modify PMP scripts to be included into CI : add the script in gitlab-…

3103472

…ci, update the sh script to run all the tests and add the missing parameter in cva6.py

Make log files available on the dashboard

84c7ed8

Update header file

7e12d6e

OlivierBetschi added 7 commits December 13, 2024 10:41

Add pmp_test into uvm package

dd3eaa3

Only use even addresses for pmpaddr test

ed56480

Remove spike tandem setting for PMP tests as spike is not enabled

a7fbfd0

Rollback of uvm_test parameter : after more testing it is not necessa…

17d77ae

…ry to switch to a different firmware test

Update PMP tests

8bdf4c9

Remove uvm_test option and associated include in uvm package

20c0179

Add SPIKE_TANDEM option

23e25e6

OlivierBetschi force-pushed the pmp/verif_plan_txt branch from 83ee53b to 23e25e6 Compare December 13, 2024 09:41

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

PMP Verif Plan and tests #2648

PMP Verif Plan and tests #2648

OlivierBetschi commented Dec 4, 2024

JeanRochCoulon commented Dec 4, 2024

JeanRochCoulon commented Dec 4, 2024

JeanRochCoulon commented Dec 4, 2024 •

edited

Loading

github-actions bot commented Dec 4, 2024

valentinThomazic commented Dec 5, 2024 •

edited

Loading

valentinThomazic commented Dec 5, 2024 •

edited

Loading

OlivierBetschi commented Dec 5, 2024

github-actions bot commented Dec 5, 2024

valentinThomazic Dec 5, 2024

valentinThomazic Dec 5, 2024

JeanRochCoulon Dec 13, 2024

valentinThomazic Dec 5, 2024

valentinThomazic Dec 5, 2024

valentinThomazic Dec 5, 2024

valentinThomazic Dec 5, 2024

JeanRochCoulon Dec 5, 2024

OlivierBetschi Dec 6, 2024

valentinThomazic commented Dec 5, 2024

github-actions bot commented Dec 6, 2024

github-actions bot commented Dec 11, 2024

OlivierBetschi commented Dec 11, 2024

valentinThomazic commented Dec 12, 2024

github-actions bot commented Dec 13, 2024

JeanRochCoulon commented Dec 16, 2024

	DV_SIMULATORS: "vcs-uvm"
	DV_SIMULATORS: "vcs-uvm"
	COLLECT_SIMU_LOGS: 1

		@@ -0,0 +1,296 @@
		# Copyright 2024 Thales DIS design services SAS

		@@ -954,6 +961,27 @@ def load_config(args, cwd):

		args.spike_params = get_full_spike_param_args(args.spike_params) if args.spike_params else ""

		def edit_Makefile(UVM_TESTNAME="uvmt_cva6_firmware_test_c"):

PMP Verif Plan and tests #2648

Are you sure you want to change the base?

PMP Verif Plan and tests #2648

Conversation

OlivierBetschi commented Dec 4, 2024

JeanRochCoulon commented Dec 4, 2024

JeanRochCoulon commented Dec 4, 2024

JeanRochCoulon commented Dec 4, 2024 • edited Loading

github-actions bot commented Dec 4, 2024

valentinThomazic commented Dec 5, 2024 • edited Loading

valentinThomazic commented Dec 5, 2024 • edited Loading

OlivierBetschi commented Dec 5, 2024

github-actions bot commented Dec 5, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

valentinThomazic commented Dec 5, 2024

github-actions bot commented Dec 6, 2024

github-actions bot commented Dec 11, 2024

OlivierBetschi commented Dec 11, 2024

valentinThomazic commented Dec 12, 2024

github-actions bot commented Dec 13, 2024

JeanRochCoulon commented Dec 16, 2024

JeanRochCoulon commented Dec 4, 2024 •

edited

Loading

valentinThomazic commented Dec 5, 2024 •

edited

Loading

valentinThomazic commented Dec 5, 2024 •

edited

Loading