Meeting Minutes
-Meeting Minutes
Meeting Minutes
Meeting Minutes
Meeting Minutes
Meeting Minutes
Meeting Minutes
Meeting Minutes
Meeting Minutes
Meeting Minutes
Meeting Minutes
Meeting Minutes
Meeting Minutes
Meeting Minutes
Meeting Minutes
Meeting Minutes
Meeting Minutes
Meeting Minutes
Meeting Minutes
Meeting Minutes
Meeting Minutes
Meeting Minutes
Meeting Minutes
Meeting Minutes
Meeting Minutes
Meeting Minutes
Meeting Minutes
Meeting Minutes
Meeting Minutes
Meeting Minutes
Meeting Minutes
Meeting Minutes
Meeting Minutes
Meeting Minutes
Meeting Minutes
Meeting Minutes
-
+
- Eric Nantz (Eli Lilly) +
- Heidi Curinckx (Johnson & Johnson) +
- HyeSoo Cho (FDA) +
- Jizu Zhi (FDA) +
- Joel Laxamana (Roche/Genentech) +
- Joseph Rickert (JBR) (R Consortium) +
- Lovemore Gakava (Novo Nordisk) +
- Nan Xiao (Merck) +
- Nicholas Masel (JRDUS) +
- Ning Leng (Roche/Genentech) +
- Paul Schuette (FDA) +
- Robert Devine (Johnson & Johnson) +
- Saghir Bashir (Argenx) +
- Sam Parmar (Pfizer) +
- Youn Kyeong Chang (FDA) +
The meeting was recorded and the video is available.
+Plot 3 Update
+FDA Recommendation
+HyeSoo stated that the FDA is considering issuing a recommendation that submissions sponsors should use the most recent patch version of the previous minor version of R.
+For example, R versions are organized as V x.y.z where:
+-
+
- x is the major version +
- y is the minor version or subversion +
- z is the patch version +
The WG members affirmed that a policy of using a proven stable version of R and not the latest version is in accordance with the practices of their respective companies.
+After some thought the working group proposed that since patch versions are issued frequently (every couple of months), it might be better to have the FDA guidance suggest the last patch version of the previous subversion.
+Note that the FDA is considering making a recommendation and not considering setting policy.
+Action: JBR will confer with Sam, Paul and HyeSoo about the wording of the recommendation and then draft a blog post. The post will go to Paul, HyeSoo, Eric, and Ning for review before being published on the R Consortium Blog.
+Pilot 3 Update
+HyeSoo and Joel informed the group that they met last month to resolve the discrepancy between the original CDISC version of the ADADAS ADaM data set and the Pilot 3 version of the ADaM data set. Joel noted that there are 818 available records in the QS domain. The CDISC ADADAS only brought in 799 records and imputed the rest, whereas Pilot 3 brought in all available 818 records into ADADAS and then imputed. When the Pilot 3 team adjusted by subsetting to ANL01FL=‘Y’ records first before doing the LOCF imputation the results matched Pilot 1 and the discrepancy was resolved. Look here for details.
+This issue illustrates the level of detail that the FDA must consider in replicating a sponsor’s submission environment. This particular issue was due to a statistical decision and not an error. Ning pointed out that most of the work in resolving discrepancies over the three Pilot submissions are due to insufficient documentation and inadequate communication and not to differences between SAS and R.
+Action: Ning and Joe will work on a blog post about this topic.
+R/Medicine
+Joel has been invited to give a talk at the upcoming R/Medicine virtual conference which will be held from June 10 to 14. He is deciding whether to give a talk or a workshop.
+UseR! 2024
+The group decided to submit an abstract for useR!2024 which will be held from July 8 to 11 in Salzburg, Austria. the deadline for the CFP is Sunday, 10 March at 11:59 PM CET (UTC+1)
+Action:
+Eric and Ning will draft the abstract.
+Pilot 4 update
+At 38:38 into the video of the meeting, Eric previews the Pilot 4 WebAssembly (Wasm) submission. To be compatible with Wasm, the submission is being built with the Rhino framework for shiny which is different from what was used in Pilot 2 but is functionally equivalent. The screen capture below references the compiling the WebR
Wasm submission with shinylive
.
After the build it is necessary to launch a webserver process on your your local setup. The Wasm version of the Shiny app reproduces the teal
filters without using teal itself.
The next step is to test the application on the container approximating the FDA Windows environment.
+Immediate Plans
+Joel’s team will continue working on Pilot 3 re-submission with the goal of picking a submission date at the April meeting.
+The Submission Environmnt Setup
+Setting up the submission environment is a manual process. The WG would like to automate at lest some of this process to ease the burden of setup and improve repeatability. It would be nice to have a GitHub action workflow that uses containers and runs on Azure. This will be a topic for the next meeting.
+Action: JBR will promote this task and try to find a company willing to work on it.
+Swissmedic WG
+JBR asked the WG for help in identifying Europe-based Pharma employees who might be interested in starting a new working group focused on Swissmedic.
+Action: Ning will open an issue on this and tag JBR and Gregory.
+Next Meeting
+The next meeting of the working group will be on Friday, April 5, 2024 at 9AM PST.
+ + +-
-
- Bob Engle (Biogen) -
- Ellis Hughes (GSK) -
- Emily Nguyan (FDA) -
- Eric Nantz (Eli Lilly) -
- Gabriel Becker (Roche/Genentech) -
- Gregory Chen (MSD) -
- Heidi Curinckx (Johnson & Johnson) -
- HyeSoo Cho (FDA) -
- Jizu Zhi (FDA) -
- Joseph Rickert (R Consortium) -
- Kui Schen (Bayer) -
- Miriam Fossati (Merck) -
- Neetu Sangari (Pfizer) -
- Ning Leng (Roche/Genentech) -
- Paul Schuette (FDA) -
- Phanikumar Tata (Syneos Health) -
- Renping Zhang (Johnson & Johnson) -
- Robert Devine (Johnson & Johnson) -
- Sam Parmar (Pfizer) -
- Sean Healey (Pfizer) -
- Steven Hasendinckx (Johnson & Johnson) -
- Xin Qiu (Johnson & Johnson) -
The meeting was recorded and the video with Passcode: NTz3r?uu
-Ning Leng provided an update on the Pilot 2 submission noting that Roche opensourced the teal
package two weeks ago. Currently the team is testing the submission with teal
, fixing some bugs identified in the last review, packaging the Shiny
app into golem
and working on the reviewer guidance documents. Ning expects that the submission team will be ready to release Pilot 2 for a wider review among the entire working group soon after that. The Pilot 2 Shiny
app is available in the submissions-pilot2 repository and the minutes from this technical sub working group are available under the Wiki tab.
Eric Nantz has been developing the Shiny
app in a Linux Docker container but will spin up a Windows container for testing. He and Ellis Hughes suggested that it may be possible to do some automated testing with GitHub actions and the shinytest2
package.
JBR Suggested that we have two stories to tell: the submission itself and the assembly of the submission and the workflows developed to support it.
-Eric pointed out that Roche releasing the teal
framework to open source is a big deal and a success story for the industry. Gabe Becker remarked that he believes that teal
was a joint effort among several companies and theRoche
intends to present it as such. Ning suggested contacting James Black and Tadeusz Lewandowski about Roche’s communication plan for teal
.
Paul Schuette suggested checking with the FDA gateway team to make sure there are no issues and also suggested clarifying the intent of the Shiny
app and asked if it is targeted towards a statistical reviewer or clinical reviewer?
Action items
--
-
- We need volunteers to test the Pilot 2 submission on multiple operating systems and see if they can deploy it. -
- Eric Nantz will spin up a Windows container to test the Submission and open an issue on the repository about testing. -
- JBR will contact James Black and Tad Lewandowski about how Roche wants to tell the
teal
story and take the lead on preparing a blog post. He will open an issue on the repo for this.
- - Ning will reach out to the regulatory team to see if any preparations are required to use the FDA gateway. -
- Ning will communicate Paul’s suggestions about clarifying the intention of the
Shiny
app to the documentation group.
-
Next, JBR asked if there was any update on the effort to work with the Japanese or Chinese regulatory agencies. Ning mentioned that with respect to Japan, Roche is working with the Japanese non-profit organization JPMA. Two main discussion are:
--
-
- The Japanese regulatory agency is understaffed and will not be able to dedicate people to reproducing a submission. The effort may be limited to getting the submission through the Japan gateway. -
- The scope and value of the Pilot. -
JBR if we should invite someone from the Japan team to visit our group. Ning suggested that this might be possible towards the end of the year.
-JBR asked Gregory Chen if he would give an update on his investigations with respect to HTA agencies. Greg reviewed the contents of a recent email on this subject that he sent to JBR. The relevant text of Greg’s email is included as an Addendum below.
-Finally, there was a short discussion about what could be done to make progress on using containers. Paul noted that the FDA NCTR has approved the limited use of ‘podman’, and offered to check with his IT contacts about efforts within the FDA that may be relevant. Eric emphasized that we will need to be open minded in looking for solutions in this area.
-The next meeting of the submissions working group will be at 9AM Pacific Time on Friday, August 3, 2022.
-Addendum - Greg Chen’s email about HTA Submissions
-At Europe level, would be ideal to engage some influential institute in EUnetHTA: Interesting past work stream that we may partner can be the following, considering the nature of our work in R Consortium,
-JA2 WP5 – Applying the HTA Core Model for Rapid Assessment for national adaptation and reporting JA2 WP8 – Maintenance of HTA Core Model infrastructure to support shared production and sharing of HTA information The following partner would be good to engage. They either lead or play crucial role in the creation and maintain of HTA Core Model® Online THL, National Institute for Health and Welfare, Finland Finnish Coordinating Center for Health Technology Assessment (FinCCHTA) Finnish Medicines Agency DEFACTUM, Social & Health Services and Labour Market Corporate Quality, Denmark At country level, the following two countries would be ideal to engage, as they are both reference country for more than 40 other countries. Germany: G-BA and IQWiG. Note, G-BA is the final decision maker, IQWiG is commissioned by G-BA to make independent assessment and recommendation. Of course, both of them play important role also in the formulation of EUnetHTA. G-BA takes a bit more lead in some work streams.
-UK: NICE