Add statement-level HTML to statement results

[sarahmcdougall]

Summary

Adds statement-level HTML as a field on each statement result returned from measure calculation.

See #268 for more details on this feature.

New behavior

The user can now specify a new calculation option: buildStatementLevelHTML. When set to true, the statement results returned from the detailed results will contain a new element statementLevelHTML that maps to a string of the HTML markup for the given statement. Thus, the structure of each statementResult will be as follows:

{
   "libraryName": "library name",
   "statementName": "statement name",
   ...
   "statementLevelHTML": "statement-level HTML"
}

For backwards compatibility, the overall html element and corresponding debug output are still available.

Code changes

• Documentation updates to reflect the new calculation option and updated output
• Calculator type updates to reflect the new calculation option
• HTML Builder changes
  • Removed the statementAnnotations array, which stores all the statement annotations and then loops over each one for generating HTML clauses. Instead generates the statement-level HTML when looping over the relevant statements, and appends the statement-level HTML to an overall HTML structure.
• HTML Builder unit tests

Testing guidance

• Run unit tests and integration tests (npm run test:plus)
• Run measure calculation against a measure with the --debug flag enabled. After calculation finishes, compare the logic highlighting html to the statementResults available on the detailedResults.
  • For statements that are not highlighted in the debug html (that have relevance equal to NA), the statementLevelHTML element should not be present on the statement’s statementResults object.
  • For statements that do not have relevance equal to NA, the statementLevelHTML element should be present on the statement’s statementResults object, and the statement-level HTML should match the HTML markup for the given statement in the debug HTML file. You can search by the data-statement-name/data-library-name in both the statementResults and debug html to compare the markup. The statement-level HTML markup should include the <pre> tag with the appropriate styling.

Note - open to thoughts on integration testing for this feature. A backlog task exists for adding more thorough testing for HTML building since the unit tests are very small in scope, but we could incorporate the beginnings of integration testing into this PR if desired. I did some brainstorming but am not 100% sure of a path forward for the integration testing (ex. Do we want to add HTML building tests for each measure type? Do we want to add tests to just the proportion boolean integration test? Do we want to make a whole new directory of integration tests specifically for HTML building?)

[github-actions]

Coverage report

St.❔	Category	Percentage	Covered / Total
🟢	Statements	86.49% (-0% 🔻)	2349/2716
🟡	Branches	73.77% (+0.08% 🔼)	2180/2955
🟢	Functions	89.05% (-0.02% 🔻)	423/475
🟢	Lines	86.84% (-0.01% 🔻)	2270/2614

Show files with reduced coverage 🔻

St.❔	File	Statements	Branches	Functions	Lines
🟢	... / HTMLBuilder.ts	92.97% (-0.05% 🔻)	79.01% (+2.62% 🔼)	94.44% (-0.15% 🔻)	93.81% (-0.05% 🔻)

Test suite run success

446 tests passing in 31 suites.

Report generated by 🧪jest coverage report action from a5f3f85

[elsaperelli]

Great job with this!! I went to test out implementing it with the prettyResults in fqm-testify and it took me 5 minutes rather than the 1000 it took me when having to parse through the whole HTML :) Some small stuff, but I did some testing and the html looks great and it is backwards compatible.

In terms of more testing, I agree that we should add more HTML testing but in my opinion that is out of scope for this task and that we can add it whenever we do that integration testing task.

I also ran the regression script which was successful since all the tests failed (they should with the added statementLevelHTML but the html object did not change.

[elsaperelli]

Since we mention the .pretty attribute below here, should we mention the new statementLevelHTML attribute in like one sentence here too? Since the two are related (or at least we intend them to be used together). Or maybe just pull the section you wrote about it up here? Maybe?

[elsaperelli]

Maybe add what the pretty result would look like here?

[elsaperelli]

The only other case I can think of for now is maybe one that makes sure there is no statementLevelHTML when the relevance is NA?

[elsaperelli]

Lgtm!

[hossenlopp]

This looks good, one really small request/question, should/is this option be defaulted to false?

Running the CLI with CMS1028 and test-cases generates a 350MB detailed results file. The CLI should minimally default to false to speed up runs.

[sarahmcdougall]

This looks good, one really small request/question, should/is this option be defaulted to false?

Running the CLI with CMS1028 and test-cases generates a 350MB detailed results file. The CLI should minimally default to false to speed up runs.

Ah good point. The option is defaulted to true, but I agree it should default to false to speed up runs. I will make that change this morning 👍

[hossenlopp]

Now we can see a clean regression run for this.