Assertions, Affirmations

[edgar-manukyan]

Let's brainstorm about the ways we will be doing assertions in the functions.

• An interesting standalone script shared by @ddsjoberg https://github.com/r-lib/rlang/blob/main/R/standalone-types-check.R
• admiral developer tools is another option https://github.com/pharmaverse/admiraldev/blob/main/R/assertions.R

[kamilsi]

@edgar-manukyan, Team

Here are my thoughts:

I've always leaned on checkmate for assertions, which has served me well. It's lightweight, robust with a plethora of checks, and the error messages have been instrumental in debugging. Moreover, it's written in C, which means we can expect it to be fast and efficient — a non-trivial advantage when working with large datasets common in our field.

Recently I've stumbled upon a couple of interesting tools that might warrant a closer look:

1. pointblank: Encountered at posit::conf, this package seems to offer robust dataframe-level assertions. It's tailored to handle the kind of data wrangling expected in the sdtm.oak project. While I'm not fully convinced of its advantages over our current tools yet, its specific design for dataframes suggests it could be useful.
2. assertr: This package caught my eye due to its integration with magrittr pipelines, which could potentially allow for more streamlined data checking processes. It's a novel approach that I haven't put to the test in our workflows, but its premise of integrating assertions into data pipelines is interesting.

Both pointblank and assertr present new approaches to assertions in R, and while they seem promising, I believe a thorough evaluation is necessary to determine their practicality for our purposes.

As for the standalone script using rlang for type assertions, while it's a novel approach, I share the concern about the need to fork and maintain our version as these assertions are not exported. I'm curious if we've identified clear advantages to this approach over the established frameworks like checkmate?

On admiral's developer tools: I wasn't aware of them before Edgar's mention. A preliminary look at their webpage didn't clarify why they built a custom solution over open-source alternatives. I haven't worked with admiral's assertion tools yet. If anyone has experience or insights—especially regarding how they differ from other tools and what specific issues they address—I'd be interested to learn more.

Could we benefit from a direct comparison? If admiral offers unique features tailored to clinical trial data that open-source alternatives don't, it might justify their approach.

[edgar-manukyan]

On admiral's developer tools: I wasn't aware of them before Edgar's mention. A preliminary look at their webpage didn't clarify why they built a custom solution over open-source alternatives. I haven't worked with admiral's assertion tools yet. If anyone has experience or insights—especially regarding how they differ from other tools and what specific issues they address—I'd be interested to learn more.

Could we benefit from a direct comparison? If admiral offers unique features tailored to clinical trial data that open-source alternatives don't, it might justify their approach.

@bundfussr, @millerg23, could you please help the oak team to understand the points that @kamilsi made about admiral's assertions https://github.com/pharmaverse/admiraldev/blob/main/R/assertions.R? Thanks so much for your help 🙏

[bundfussr]

At the beginning of admiral we used assertthat for checking arguments. As it is not very powerful, we implemented our own assertions. I think nobody was aware of checkmate. At least we never discussed this option. The admiral assertions are more tailored to admiral than to clinical trial data. Most likely for many of them checkmate functions could be used.
We have only a few data checks in admiral. These provide the data which caused the failure as a dataframe because just printing an error message is not so helpful for finding the reason for the failure. For example, if the check that USUBJID is a unique key of a dataframe fails, the rows where USUBJID is not unique are provided as a dataframe. I am not sure if something similar is offered by other data checking packages.

[kamilsi]

@bundfussr, I really appreciate you taking the time to contribute to our discussion – thank you. Your perspective gives us a clearer picture of the motivations behind admiral's custom assertions.

For example, if the check that USUBJID is a unique key of a dataframe fails, the rows where USUBJID is not unique are provided as a dataframe. I am not sure if something similar is offered by other data checking packages.

Your approach to providing detailed data when checks fail is particularly intriguing; it’s a practical feature for troubleshooting. On that note, I believe pointblank has a strong offering in terms of dataframe-level checks and a comprehensive reporting system that might complement this feature well.

Considering the broader ecosystem, there’s an inherent assurance in adopting tools like checkmate and pointblank which are not only popular but also under active development. It's often less risky to rely on these well-established solutions, as they are typically supported by a larger community and offer a wealth of resources and documentation. This approach could allow us to direct our valuable time and effort towards unique challenges that these tools don't already address.