Skip to content

Commit

Permalink
consistency fixes (#293)
Browse files Browse the repository at this point in the history
* Changed the example to reflect current line 3 of supplier

The previous example was no longer in line with the current Public Safety Impact values.

* typo fix

* add link to line of CSV file

* Fixing links and add scikit-learn reference

* Fix example to align with current values

* Fix links, markup and made example consistent with current values.

* typo fix in 07_04_tree_customization.md
  • Loading branch information
jeroenh authored Aug 30, 2023
1 parent d18d8d6 commit 22e78d0
Show file tree
Hide file tree
Showing 7 changed files with 27 additions and 15 deletions.
4 changes: 2 additions & 2 deletions doc/md_src_files/03_representing_information.md
Original file line number Diff line number Diff line change
Expand Up @@ -139,9 +139,9 @@ An interoperable CSV format is also flexible enough to support a variety of uses
Every situation in SSVC is defined by the values for each decision point and the priority label (outcome) for that situation (as defined in [Likely Decision Points and Relevant Data](#likely-decision-points-and-relevant-data)).
A CSV will typically be 30-100 rows that each look something like:
```
2,none,slow,diffuse,laborious,partial,minor,defer
2,none,laborious,partial,significant,scheduled
```
Where “2” is the row number, [*none*](#exploitation) through [*minor*](#public-safety-impact) are values for decision points, and *defer* is a priority label or outcome.
Where “2” is the row number, [*none*](#exploitation) through [*significant*](#public-safety-impact) are values for decision points, and *scheduled* is a priority label or outcome.
Different stakeholders will have different decision points (and so different options for values) and different outcomes, but this is the basic shape of a CSV file to define SSVC stakeholder decisions.

The tree visualization options are more diverse.
Expand Down
2 changes: 1 addition & 1 deletion doc/md_src_files/06_04_publication_decision.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,6 @@ Then one might go from [*limited*](#public-value-added) to [*ampliative*](#publi

This decision point accounts for the state of the supplier's work on addressing the vulnerability.

- *Fix Ready*—the supplier has provided a patch or fix
- *Fix Ready*—the supplier has provided a patch or fix.
- *Cooperative*—the supplier is actively generating a patch or fix; they may or may not have provided a mitigation or work-around in the mean time.
- *Uncooperative/Unresponsive*—the supplier has not responded, declined to generate a remediation, or no longer exists.
2 changes: 1 addition & 1 deletion doc/md_src_files/07_00_prioritization.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ The definition of choices can take a logical form, such as:
- ([*Human Impact*](#human-impact) IS [medium](#human-impact))
- THEN priority is *scheduled*.

This logical statement is captured in line 35 of the deployer `.csv` file.
This example logical statement is captured in (line 35 of the deployer `.csv` file)[https://github.com/CERTCC/SSVC/blob/main/data/csvs/deployer-options.csv#L35].

There are different formats for capturing these prioritization decisions depending on how and where they are going to be used.
In this paper, we primarily represent a full set of guidance on how one stakeholder will make a decision as a **decision tree**.
Expand Down
8 changes: 4 additions & 4 deletions doc/md_src_files/07_04_tree_customization.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,9 @@ As long as each team has documented this choice and is consistent in its own app
SSVC enables teams with such different risk appetites to discuss and communicate precisely the circumstances where they differ.

When doing the detailed risk management work of creating or modifying a tree, we recommend working from text files with one line or row for each unique combination of decision values.
For examples, see [SSVC/data](https://github.com/CERTCC/SSVC/tree/main/data).
For examples, see [SSVC/data](https://github.com/CERTCC/SSVC/tree/main/data/csvs).
An important benefit, in our experience, is that it is easier to identify a question by saying “I'm unsure about row 16” than anything else we have thought of so far.
Once the humans agree on the decision tree, it can be converted to a JSON schema for easier machine-readable communication, following the provided [SSVC provision JSON schema](https://github.com/CERTCC/SSVC/blob/main/data/schema/SSVC_Provision_v2.03.schema.json).
Once the humans agree on the decision tree, it can be converted to a JSON schema for easier machine-readable communication, following the provided [SSVC provision JSON schema](https://github.com/CERTCC/SSVC/blob/main/data/schema/SSVC_Provision.schema.json).

Once the decision points are selected and the prioritization labels agreed upon, it is convenient to be able to visually compress the text file by displaying it as a decision tree.
Making the decision process accessible has a lot of benefits.
Expand Down Expand Up @@ -77,7 +77,7 @@ Permutation importance can be calculated on a candidate tree to highlight potent
It works by randomly shuffling the values for each feature individually and comparing a fitness metric on the shuffled tree to the original.
The change in fitness is taken to be the importance of the feature that was shuffled.
Permutation importance is usually given as a number in the interval [0,1].
Python's scikit-learn provides a permutation importance method, which we used to evaluate our trees.
Python's scikit-learn [@pedregosa2011scikit-learn] provides a permutation importance method, which we used to evaluate our trees.

Interpreting the results of a permutation importance computation on a tree involves nuance, but one rule we can state is this:
any feature with a computed permutation importance of zero can be eliminated from the tree without losing any relevant information.
Expand Down Expand Up @@ -162,7 +162,7 @@ For example, an organization with sufficient resources to efficiently deploy fix
2. defining a new decision point and adding it to an existing tree.
Note that adding or removing an option from an existing decision point should be treated as creating a new decision point.
The new decision point should be given a distinct name as well.
3. defining a new tree entirely from existing or new decision points
3. defining a new tree entirely from existing or new decision points.

Because tree customization changes the tree structure and implies the addition or removal of leaf nodes, it will be necessary for the organization to review the decision outputs in light of its risk appetite as well.

Expand Down
2 changes: 1 addition & 1 deletion doc/md_src_files/07_05_evidence_gathering.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,5 +19,5 @@ With a total lack of information, assume [*support crippled*](#mission-impact) a
[*Exploitation*](#exploitation) needs no special default; if adequate searches are made for exploit code and none is found, the answer is [*none*](#exploitation).
If nothing is known about [*Automatable*](#automatable), the safer answer to assume is [*yes*](#automatable).
[*Value Density*](#value-density) should always be answerable; if the product is uncommon, it is probably [*diffuse*](#value-density).
The resulting decision set {*none*, *open*, *efficient*, *medium*} results in a scheduled patch application in our recommended deployer tree.
The resulting decision set {*none*, *open*, *yes*, *medium*} results in a scheduled patch application in our recommended deployer tree.

12 changes: 6 additions & 6 deletions doc/md_src_files/08_communicating_results.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,18 +57,18 @@ The optional parameter `time` is the date and time of the SSVCv2 record creation

Based on this, an example string could be:
```
SSVCv2/Ps:Nm/T:T/U:E/2018-11-13T20:20:00Z/
SSVCv2/Ps:M/T:T/U:E/2018-11-13T20:20:00Z/
```
For a vulnerability with [no or minor](#public-safety-impact) [*Public Safety Impact*](#public-safety-impact), [total](#technical-impact) [*Technical Impact*](#technical-impact), and [efficient](#utility) [*Utility*](#utility), which was evaluated on Nov 13,2018 at 8:20 PM UTC.
For a vulnerability with [minimal](#public-safety-impact) [*Public Safety Impact*](#public-safety-impact), [total](#technical-impact) [*Technical Impact*](#technical-impact), and [efficient](#utility) [*Utility*](#utility), which was evaluated on Nov 13,2018 at 8:20 PM UTC.

While these abbreviated format vectors can be uniquely produced based on a properly formatted JSON object, going from abbreviated form to JSON is not supported.
Therefore, JSON is the preferred storage and transmission method.

### Full JSON format

For a more robust, self-contained, machine-readable, we provide JSON schemas.
The [provision schema](https://github.com/CERTCC/SSVC/blob/main/data/schema/SSVC_Provision_v2.03.schema.json) is equivalent to a decision tree and documents the full set of logical statements that a stakeholder uses to make decisions.
The [computed schema](https://github.com/CERTCC/SSVC/blob/main/data/schema/SSVC_Computed_v2.03.schema.json) expresses a set of information about a work item or vulnerability at a point in time.
The [provision schema](https://github.com/CERTCC/SSVC/blob/main/data/schema/SSVC_Provision.schema.json) is equivalent to a decision tree and documents the full set of logical statements that a stakeholder uses to make decisions.
The [computed schema](https://github.com/CERTCC/SSVC/blob/main/data/schema/SSVC_Computed.schema.json) expresses a set of information about a work item or vulnerability at a point in time.
A computed schema should identify the provision schema used, so the options from which the information was computed are specified.

Each element of `choices` should be an object that is a key-value pair of `decision point`:`value`, where the term `decision point` is a string derived from the name of the decision point as follows:
Expand Down Expand Up @@ -98,7 +98,7 @@ A stakeholder might have information to communicate about some decision points b
If SSVC uses this format to list the values that are in play for a particular vulnerability, there is no need for a special “I don't know” marker.

The merit in this “list all values” approach emerges when the stakeholder knows that the value for a decision point may be A or B, but not C.
For example, say the analyst knows that [*Value Density*](#value-density) is [diffuse](#value-density) but does not know the value for [*Automatability](#automatability).
For example, say the analyst knows that [*Value Density*](#value-density) is [diffuse](#value-density) but does not know the value for [*Automatability*](#automatability).
Then the analyst can usefully restrict [*Utility*](#utility) to one of [laborious](#utility) or [efficient](#utility).
In abbreviated form, write this as `U:LE`.
As discussed below, information can change over time.
Expand Down Expand Up @@ -129,7 +129,7 @@ As an initial heuristic, we suggest the associated polling frequency for each.
These frequencies can be customized, as the update frequency is directly related to the organization's tolerance for the risk that the information is out of date.
As discussed in [Tree Construction and Customization Guidance](#tree-construction-and-customization-guidance), risk tolerance is unique to each organization.
Risk tolerance and risk appetite are primarily reflected in the priority labels (that is, decisions) encoded in the SSVC decision tree, but information polling frequency is also a risk tolerance decision and each organization may choose different time values.
- [*State of Exploitation*](#state-of-exploitation): every 1 day
- [*Exploitation*](#exploitation): every 1 day
- [*Technical Impact*](#technical-impact): never (should be static per vulnerability)
- [*Utility*](#utility): every 6 months
- [*Public Safety Impact*](#public-safety-impact): every 1 year
Expand Down
12 changes: 12 additions & 0 deletions doc/md_src_files/sources_ssvc.bib
Original file line number Diff line number Diff line change
Expand Up @@ -166,6 +166,18 @@ @article{wang2017characterizing
address = {New York, NY, USA}
}

@article{pedregosa2011scikit-learn,
title={Scikit-learn: Machine Learning in {P}ython},
author={Pedregosa, F. and Varoquaux, G. and Gramfort, A. and Michel, V.
and Thirion, B. and Grisel, O. and Blondel, M. and Prettenhofer, P.
and Weiss, R. and Dubourg, V. and Vanderplas, J. and Passos, A. and
Cournapeau, D. and Brucher, M. and Perrot, M. and Duchesnay, E.},
journal={Journal of Machine Learning Research},
volume={12},
pages={2825--2830},
year={2011}
}

%%%% End Miscellaneous articles
%%%% Miscellaneous conference proceedings
Expand Down

0 comments on commit 22e78d0

Please sign in to comment.