Skip to content

Commit

Permalink
DOCS: updates focused on plugins.rst, but there were capitalization c…
Browse files Browse the repository at this point in the history
…hanges done to the entire documentation by search and manual replace
  • Loading branch information
jeras committed Jun 29, 2022
1 parent 460eb45 commit 73e842b
Show file tree
Hide file tree
Showing 12 changed files with 331 additions and 312 deletions.
18 changes: 18 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,24 @@

This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [] -
- Documentation consistency cleanup, changes are mostly aesthetic, the content is not changed:
- Expanding use of fixed width fonts for file, function, argument, variable, macro, ... names.
- Fixed missing capitalization and punctuation, mostly for lists.
- Capitalization of acronyms and proper nouns:
- python => Python,
- riscof => RISCOF,
- isa => ISA,
- platform => Platform,
- yaml => YAML,
- verilator => Verilator,
- cli => CLI,
- makefile => Makefile (depending on context),
- ...
- Few spelling and grammar fixes.
- Few instances where names in documentation did not match names in the source code.
This were either copy/paste errors or code changes not propagated to documentation.

## [1.24.0] - 2022-0514
- rename the "master" branch of riscv-arch-test to "main"

Expand Down
10 changes: 5 additions & 5 deletions PLUGINS.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@
List of Reference RISCOF Plugins
================================

This section provides a list of pre-built riscof-plugins which users can refer
to, to build plugins for their own DUT
This section provides a list of pre-built RISCOF plugins which users can refer
to, to build plugins for their own DUT:

- Spike: https://gitlab.com/incoresemi/riscof-plugins/-/tree/master/spike_parallel
- SAIL_cSim: https://gitlab.com/incoresemi/riscof-plugins/-/blob/master/sail_cSim/README.md
- InCore Plugins: https://gitlab.com/incoresemi/riscof-plugins (This is a collection of riscof based plugins for various targets hosted purely for reference.)
- `Spike <https://gitlab.com/incoresemi/riscof-plugins/-/tree/master/spike_parallel>`_,
- `SAIL_cSim <https://gitlab.com/incoresemi/riscof-plugins/-/blob/master/sail_cSim/README.md>`_,
- `InCore Plugins <https://gitlab.com/incoresemi/riscof-plugins>`_ (This is a collection of RISCOF based plugins for various targets hosted purely for reference.)
2 changes: 1 addition & 1 deletion docs/source/TestFormatSpec.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ significant amount of the framework shall depend on the existence of these macro
`RVTEST_ISA(isa_str)`::

- defines the Test Virtual Machine (TVM, the ISA being tested) +
- empty macro to specify the isa required for compilation of the test. +
- empty macro to specify the ISA required for compilation of the test. +
- this is mandated to be present at the start of the test.


Expand Down
6 changes: 3 additions & 3 deletions docs/source/arch-tests.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ Running RISCV-ARCH-TESTS
The following guide provides a walkthrough on how to run the tests available at
the `riscv-arch-test <https://github.com/riscv/riscv-arch-test>`_ repository.

The following assumes you have installed riscof as a cli on your system. If not, then please refer
The following assumes you have installed RISCOF as a CLI on your system. If not, then please refer
to the :ref:`install_riscof` section for the same.


Expand All @@ -17,7 +17,7 @@ Setup all the DUT and Ref Plugins

1. You will first need to install the SAIL C-emulator on your system. You can refer to the
:ref:`plugin_models` section for steps on installing the SAIL C-emulator.
2. You will then need to download/clone the ``sail_cSim`` riscof plugin. You can do this with the
2. You will then need to download/clone the ``sail_cSim`` RISCOF plugin. You can do this with the
following command:

.. code-block:: console
Expand All @@ -28,7 +28,7 @@ Setup all the DUT and Ref Plugins
You will need the path of the `riscof-plugins` directory from the above repo for the next
steps.

3. You will also need to create a riscof-plugin for you own DUT. If you haven't already done so,
3. You will also need to create a RISCOF plugin for you own DUT. If you haven't already done so,
please refer to the :ref:`plugins` section for details on building one.

Create a config.ini file
Expand Down
87 changes: 44 additions & 43 deletions docs/source/commands.rst
Original file line number Diff line number Diff line change
Expand Up @@ -16,36 +16,37 @@ The current list of subcommands includes:
- ``testlist``
- ``run``

arch-tests
----------
``arch-tests``
--------------

This command is used to clone and update the tests from the official `riscv-arch-test <https://github.com/riscv/riscv-arch-test>`_ repository.

This command requires one of the following flags to be specified from the cli.
This command requires one of the following flags to be specified from the CLI.

- ``--show-version``: Display the current version of the official suite present at the specified directory path.
- ``--clone``: Clone the suite from github.
- ``--update``: Update the suite to reflect latest changes from github.

Optional arguments from the cli:
Optional arguments from the CLI:

- ``--get-version``: The specific version of the tests to be fetched. Can be used with both the clone and
update flags. The latest release is fetched if skipped.
- ``--dir``: The path to the directory where the suite is to be cloned to. Defaults to
``./riscv-arch-test`` if skipped.

coverage
--------
``coverage``
------------

This command is used to collect the ISA coverage metrics of a given test-suite and generate a coverage
report in html.

This command will require the following inputs from the cli:
This command will require the following inputs from the CLI:

- ``--suite``: The test suite path on which coverage needs to be run.
- ``--env``: The path to the environment directory containing the suite-specific header files.
- ``--cgf``: The list of covergroup-format files specifying the coverpoints that need to be covered by the suite.

Optional arguments from the cli:
Optional arguments from the CLI:

- ``--config``: The path to the ``config.ini`` file. Defaults to ``./config.ini`` if skipped.
- ``--work-dir``: The path to the working directory where all artifacts need to be dumped. Defaults to
Expand All @@ -54,22 +55,22 @@ Optional arguments from the cli:
browser.

The coverage command simply passes the cgf files to the reference plugin's runTests function. The
Reference plugin is responsible to generating a yaml based coverage report for each test using ``riscv-isac``.
The yaml file should be named ``coverage.rpt``. The ``riscv-isac`` run will also generate a data-propagation
Reference plugin is responsible to generating a YAML based coverage report for each test using ``riscv-isac``.
The YAML file should be named ``coverage.rpt``. The ``riscv-isac`` run will also generate a data-propagation
report which should be named as ``ref.md``.

Once the coverage files for each test has been generated, RISCOF will parse through the working
directories and merge all the ``coverage.rpt`` files to create a single yaml coverage report:
directories and merge all the ``coverage.rpt`` files to create a single YAML coverage report:
``suite_coverage.rpt``. RISCOF then also converts this to an HTML based reports and open it on the
default web-browser.

For a example on using this feature please refer to the :ref:`coverage` section.

gendb
-----
``gendb``
---------

This command is used to generate a database yaml file for all tests available in the test-suite. The
commands requires the following inputs from the cli:
This command is used to generate a database YAML file for all tests available in the test-suite. The
commands requires the following inputs from the CLI:

- ``--suite``: The test suite path for which database needs to be generated.
- ``--work-dir``: The path to the working directory where all artifacts need to be dumped. Defaults to
Expand All @@ -81,7 +82,7 @@ RVTEST_CASE, etc. For each test the utility will create a new entry in a diction
the different parts of the tests, the enabling conditions of each part, the coverage contributions
of each part, any compile macros required for each part and much more.

The generated database yaml will follow the syntax described in section :ref:`database`.
The generated database YAML will follow the syntax described in section :ref:`database`.

The output of this utility is a ``database.yaml`` located in the ``work_dir`` directory. This file is
used by RISCOF to select and filter tests based on input DUT configuration.
Expand All @@ -90,16 +91,16 @@ used by RISCOF to select and filter tests based on input DUT configuration.
<https://github.com/riscv/riscv-arch-test/blob/master/spec/TestFormatSpec.adoc>`_ set forth
by the riscv-arch-test SIG.

setup
-----
``setup``
---------

The setup command is used to generate a series of Template files that are required by RISCOF.
These files are meant to provide ease to users integrating their DUT to RISCOF for the first time
and should be modified by the users.

The setup utility takes in the following optional inputs from the cli:
The setup utility takes in the following optional inputs from the CLI:

- ``--dutname``: The name of the dut for running the tests on. The utility will use this name to create a
- ``--dutname``: The name of the DUT for running the tests on. The utility will use this name to create a
template plugin directory with all the relevant files. These files will have to be modified by
the user. Defaults to ``spike`` when skipped.
- ``--refname``: The name of the reference plugin to be used in RISCOF. The utility will use this name to
Expand All @@ -108,51 +109,51 @@ The setup utility takes in the following optional inputs from the cli:

The setup utility will also create a sample config.ini file using the above inputs.

validateyaml
------------
``validateyaml``
----------------

This command simply performs a validation of the isa spec and the platform pspec yamls of the DUT
as mentioned in the ``config.ini`` using riscv-config. The outputs are checked for the version of the yamls in
This command simply performs a validation of the ISA ``ispec`` and the Platform ``pspec`` YAMLs of the DUT
as mentioned in the ``config.ini`` using riscv-config. The outputs are checked for the version of the YAMLs in
the directory pointed to by ``work_dir``.

testlist
--------
``testlist``
------------

This command is used to filter tests from the database.yaml based on the configuration of DUT
present in the isa and platform yamls as mentioned in the ``config.ini``. This command will require
the following inputs from the cli:
This command is used to filter tests from the ``database.yaml`` based on the configuration of DUT
present in the ISA and Platform YAMLs as mentioned in the ``config.ini``. This command will require
the following inputs from the CLI:

- ``--suite``: The test suite from which the tests need to be filtered.

This command takes the following optional inputs from cli:
This command takes the following optional inputs from CLI:

- ``--config``: The path to the ``config.ini`` file. Defaults to ``./config.ini`` if skipped.
- ``--work-dir``: The path to the working directory where all artifacts need to be dumped. Defaults to
``./riscof_work``.

The utility first creates a ``database.yaml`` for the input suite. For each test in the database yaml,
this utility will check if the conditions of any parts of a test are enabled based on the isa and
platform yaml specs of the DUT. If any part is enabled, then the corresponding test is entered into
The utility first creates a ``database.yaml`` for the input suite. For each test in the database YAML,
this utility will check if the conditions of any parts of a test are enabled based on the ISA and
Platform YAML specs of the DUT. If any part is enabled, then the corresponding test is entered into
the teslist along with the respective coverage labels and compile macros.

The utility will dump the test list in the ``testlist.yaml`` file in the ``work_dir`` directory. This
yaml will follow the same syntax as defined in the :ref:`testlist` section.
YAML will follow the same syntax as defined in the :ref:`testlist` section.

run
---
``run``
-------

This is probably the primary command of RISCOF which is going to be widely used. This command is
currently responsible for first validating the inputs yamls,
currently responsible for first validating the inputs YAMLs,
creating a database of the tests in the ``suite`` directory, generate a
filtered test-list, run the tests on the DUT and then the Reference Plugins, and finally compare the
generated signatures and present an html report of the findings.

The following inputs are required on the cli by this command:
The following inputs are required on the CLI by this command:

- ``--suite``: The test suite path on which coverage needs to be run
- ``--env``: The path to the environment directory containing the suite-specific header files.

Optional arguments from the cli:
Optional arguments from the CLI:

- ``--config``: The path to the ``config.ini`` file. Defaults to ``./config.ini`` if skipped.
- ``--work-dir``: The path to the working directory where all artifacts need to be dumped. Defaults to
Expand All @@ -167,7 +168,7 @@ Optional arguments from the cli:
The ``work_dir`` is cleaned by default. However, if one of ``no-clean``, ``testfile`` or ``dbfile``
are specified, it is preserved as is.

All artifacts of this command are generated in the ``work_dir`` directory. Typicall artifacts will
All artifacts of this command are generated in the ``work_dir`` directory. Typically artifacts will
include:

======================== =============================================================
Expand All @@ -177,7 +178,7 @@ Artifact Description
``Makefile.DUT*`` This is the Makefile generated by the DUT Plugin.
``Makefile.Reference*`` This is the Makefile generated by the Reference Plugin.
``report.html`` The final report generated at the end of the run after signature comparison.
yaml files Verified and checked yaml versions of the input isa and platform yamls.
YAML files Verified and checked YAML versions of the input ISA and Platform YAMLs.
``test_list.yaml`` This list of filtered tests from the ``database.yaml``.
src directory This will include a directory for each test in the ``test_list.yaml``. Each test-directory will include the test, compiled-binaries, signatures from both the DUT and the Reference Plugin.
==================== =============================================================
src directory This will include a directory for each test in the ``test_list.yaml``. Each test-directory will include the test, compiled-binaries, signatures from both the DUT and the Reference Plugin.
======================== =============================================================
26 changes: 13 additions & 13 deletions docs/source/database.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,18 +13,18 @@ in them and constructs a dictionary of sorts, for the framework.
The tests in the directory are identified by their relative path from the repository home.
Each test in the database is defined as follows:

* **file path**: the absolute path of the test on the said system
* **file path**: the absolute path of the test on the said system:

* **commit_id**: Contains the recent commit id of the commit in which the test was modified.
* ``commit_id``: Contains the recent commit id of the commit in which the test was modified.

* **isa**: Contains the isa required for the compilation of the test. This field is extracted from the *RVTEST_ISA* macro.
* ``isa``: Contains the ISA required for the compilation of the test. This field is extracted from the ``RVTEST_ISA`` macro.

* **parts**: Contains the individual parts present in the test and the conditions and macros required by each of them. The parts are identified by unique names as specified in the test. A test must contain at-least one part for it to be included in the database.
* ``parts``: Contains the individual parts present in the test and the conditions and macros required by each of them. The parts are identified by unique names as specified in the test. A test must contain at-least one part for it to be included in the database.

* **part name**: This node is extracted from the *RVTEST_CASE_START* macro in the test.
* **part name**: This node is extracted from the ``RVTEST_CASE_START`` macro in the test.

* **check**: A list of the check statements for the part as specified in the test. These translate to the conditions which need to be satisfied for this part to be included.
* **define**: A list of define statements for the part as specified in the test. These translate to the macros required by this part to run.
* ``check``: A list of the check statements for the part as specified in the test. These translate to the conditions which need to be satisfied for this part to be included.
* ``define``: A list of define statements for the part as specified in the test. These translate to the macros required by this part to run.

Example:

Expand Down Expand Up @@ -52,18 +52,18 @@ Usage
Reasons of Failure
^^^^^^^^^^^^^^^^^^

Possible scenarios where database is not generated
Possible scenarios where database is not generated:

* There does not exist at-least one part in the test.
* Any part which has started does not end before another part starts or the code ends i.e. *RVTEST_CASE_START* exists for that part but *RVTEST_CASE_END* doesn't.
* The part names given in a *RVTEST_CASE_START*-*RVTEST_CASE_END* pair doesn't match.
* *RVTEST_ISA* macro isn't present in the test.
* Any part which has started does not end before another part starts or the code ends i.e. ``RVTEST_CASE_START`` exists for that part but ``RVTEST_CASE_END`` doesn't.
* The part names given in a ``RVTEST_CASE_START``-``RVTEST_CASE_END`` pair doesn't match.
* ``RVTEST_ISA`` macro isn't present in the test.

Notes
^^^^^

1. The database is always alphabetically ordered
1. The database is always alphabetically ordered.
2. The database checks for macro sanity - i.e. certain macros exists and in the correct order.
3. Each time a new test is added to the ``suite`` directory, the database utility has to be run manually and the database.yaml
3. Each time a new test is added to the ``suite`` directory, the database utility has to be run manually and the ``database.yaml``
has to be up-streamed manually to the repository.

Loading

0 comments on commit 73e842b

Please sign in to comment.