Skip to content

Commit

Permalink
v2.0.7.9014
Browse files Browse the repository at this point in the history
* refactor: Update Thresholds tab order
* refactor: Add text to Main - Import tab
  • Loading branch information
leppott committed Sep 8, 2023
1 parent 149dbf4 commit 6158f6d
Show file tree
Hide file tree
Showing 18 changed files with 320 additions and 16,029 deletions.
2 changes: 1 addition & 1 deletion DESCRIPTION
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
Package: ContDataQC
Title: Quality Control (QC) of Continous Monitoring Data
Version: 2.0.7.9013
Version: 2.0.7.9014
Authors@R: c(
person("Erik W", "Leppo", email="Erik.Leppo@tetratech.com",role=c("aut","cre")),
person("Ann","Roseberry Lincoln", role="ctb"),
Expand Down
9 changes: 8 additions & 1 deletion NEWS
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,17 @@ NEWS-ContDataQC

<!-- NEWS.md is generated from NEWS.Rmd. Please edit that file -->

#> Last Update: 2023-09-08 11:07:40.603256
#> Last Update: 2023-09-08 14:58:39.04085

# Version History

## v2.0.7.9014

2023-09-08

- refactor: Update Thresholds tab order
- refactor: Add text to Main - Import tab

## v2.0.7.9013

2023-09-08
Expand Down
9 changes: 8 additions & 1 deletion NEWS.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,17 @@ NEWS-ContDataQC

<!-- NEWS.md is generated from NEWS.Rmd. Please edit that file -->

#> Last Update: 2023-09-08 11:07:40.603256
#> Last Update: 2023-09-08 14:58:39.04085

# Version History

## v2.0.7.9014

2023-09-08

- refactor: Update Thresholds tab order
- refactor: Add text to Main - Import tab

## v2.0.7.9013

2023-09-08
Expand Down
6 changes: 6 additions & 0 deletions NEWS.rmd
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,12 @@ cat(paste0("Last Update: ",Sys.time()))

# Version History

## v2.0.7.9014
2023-09-08

* refactor: Update Thresholds tab order
* refactor: Add text to Main - Import tab

## v2.0.7.9013
2023-09-08

Expand Down
18 changes: 17 additions & 1 deletion inst/extdata/ContDataQC_LibraryCreation.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,23 @@ package directory
myLibrary <- "ContDataQC"
dir_base <- "C:/Users/Erik.Leppo/OneDrive - Tetra Tech, Inc/MyDocs_OneDrive/GitHub"
setwd(file.path(dir_base, myLibrary))
#
# Remove all files in "Results" folder
# Triggered here so can run different files
path_shiny <- file.path("inst", "shiny-examples", "ContDataQC")
path_results <- file.path(path_shiny, "data")
fn_results <- list.files(path_results
, full.names = TRUE
, include.dirs = FALSE
, recursive = TRUE)
file.remove(fn_results) # ok if no files
# Copy file to ensure directory not empty
fn_copy <- "remove.txt"
path_from <- file.path(path_shiny, "external", fn_copy)
path_to <- file.path(path_shiny, "data", fn_copy)
file.copy(path_from, path_to)
# NEWS
# Render then Copy NEWS so picked up in help
rmarkdown::render("NEWS.rmd", "all")
Expand Down
166 changes: 83 additions & 83 deletions inst/extdata/ContDataQC_LibraryCreation.nb.html

Large diffs are not rendered by default.

15,913 changes: 0 additions & 15,913 deletions inst/shiny-examples/ContDataQC/data/7392-354869-base_noQ_noPctSat.csv

This file was deleted.

Empty file.
Empty file.
6 changes: 5 additions & 1 deletion inst/shiny-examples/ContDataQC/external/tab_3_MainFunc.R
Original file line number Diff line number Diff line change
Expand Up @@ -33,9 +33,12 @@ function(){
, tabPanel("Import Files"
, sidebarLayout(
sidebarPanel(
p("1. Upload customized QC test thresholds if desired; otherwise, the default thresholds will be used.")
, p("2. Confirm correct configuration file is being used.")
, p("3. Upload input files.")
#The selected input file
#Tool tip code from https://stackoverflow.com/questions/16449252/tooltip-on-shiny-r
tags$div(title="Select one or more csv files to upload here"
, tags$div(title = "Select one or more csv files to upload here"

#Only allows csv files to be imported
, fileInput("selectedFiles"
Expand All @@ -44,6 +47,7 @@ function(){
, accept = ".csv")
)# tags$div ~ END

, p("4. Choose operation to perform.")
#Operation to be performed on the selected data
,selectInput("Operation"
,label = "Choose operation to perform"
Expand Down
58 changes: 31 additions & 27 deletions inst/shiny-examples/ContDataQC/external/tab_3c_QCThresh.R
Original file line number Diff line number Diff line change
Expand Up @@ -8,17 +8,43 @@ function() {
tabPanel("QC Thresholds"
, mainPanel(
tabsetPanel(type = "tabs"
# 4.1. Default----
# 3c1. Default----
, tabPanel("Default"
, includeHTML("www/RMD_HTML/App_3c1_QCThresh_Defaults.html")
)## tabPanel ~ CURRENT ~ END
# 4.2. Eval----

# 3c2. Upload Custom ----
, tabPanel("Upload Custom Thresholds"
,sidebarLayout(
sidebarPanel(
tags$div(title="Select R configuration file to upload here"
#Only allows R files to be imported
,fileInput("configFile"
, label = "Choose configuration file"
, multiple = FALSE
, accept = ".R")
) # tag$div ~ END
, br()
, br()

#Only shows the "Default configuration" button after a user-selected file has been used
,tags$div(title = "Click to use default configuration data"
,uiOutput('ui.defaultConfig'))
)# sidebarPanel~ END
, mainPanel(
includeHTML("www/RMD_HTML/App_3c2_QCThresh_Upload.html")
) ## mainPanel ~ END
)# sidebarLayout~ END
)## tabPanel ~ Custom Thresh ~ END

# 3c3. Eval----
, tabPanel("Evaluate Thresholds"
, includeHTML("www/RMD_HTML/App_3c2_QCThresh_Eval.html")
, includeHTML("www/RMD_HTML/App_3c3_QCThresh_Eval.html")
# , br()
# , downloadButton("but_thresh_code")
)## tabPanel ~ Eval Thresh
# 4.3. Edit ----

# 3c4. Edit ----
, tabPanel("Edit Thresholds"
,shinyjs::useShinyjs()
# ,h3("Edit your thresholds here by parameter:")
Expand Down Expand Up @@ -63,7 +89,7 @@ function() {

) # sidebarPanel~ END
, mainPanel(width = 7
, includeHTML("www/RMD_HTML/App_3c3_QCThresh_Edit.html")
, includeHTML("www/RMD_HTML/App_3c4_QCThresh_Edit.html")
, tags$hr()
, h4("Gross Thresholds")
, p(paste("Test if data point exceeds a user defind"
Expand Down Expand Up @@ -147,29 +173,7 @@ function() {
)## mainPanel ~ END
)# sidebarLayout~ END
) ## tabPanel ~ Edit Thresh
# 4.4 Upload Custom ----
, tabPanel("Upload Custom Thresholds"
,sidebarLayout(
sidebarPanel(
tags$div(title="Select R configuration file to upload here"
#Only allows R files to be imported
,fileInput("configFile"
, label = "Choose configuration file"
, multiple = FALSE
, accept = ".R")
) # tag$div ~ END
, br()
, br()

#Only shows the "Default configuration" button after a user-selected file has been used
,tags$div(title = "Click to use default configuration data"
,uiOutput('ui.defaultConfig'))
)# sidebarPanel~ END
, mainPanel(
includeHTML("www/RMD_HTML/App_3c4_QCThresh_Upload.html")
) ## mainPanel ~ END
)# sidebarLayout~ END
)## tabPanel ~ Custom Thresh ~ END

)## tabsetPanel ~ END
)## mainPanel ~ END
Expand Down
2 changes: 1 addition & 1 deletion inst/shiny-examples/ContDataQC/ui.R
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ tab_5b_Console <- source("external/tab_5b_Console.R", local = TRUE)$value

shinyUI(
# VERSION, 1, current [non-EPA] ----
navbarPage("Continuous data QC, summary, and statistics - v2.0.7.9012",
navbarPage("Continuous data QC, summary, and statistics - v2.0.7.9014",
theme = shinytheme("spacelab")
,tab_1_About()
#,tab_1a_Overview()
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@



<div id="upload-custom-thresholds" class="section level2">
<h2>Upload Custom Thresholds</h2>
<p>Click the ‘Browse’ button (to the left) and select the desired
customized configuration file. The status bar will tell you when the
file has been uploaded. A new button will appear that gives you the
option of returning to the default configuration file. Do not click this
unless you change your mind and n o longer want to use the customized
file.</p>
<p>Go to the ‘Main Functions’ tab; verify that the correct configuration
file has been uploaded (by checking the file name); select the desired
input file(s); run them through the QC function. Save the output to the
desired location. Open the Word or html QC report and scroll to the
‘Thresholds, Quick Reference’ table (in the middle section of the
report). Verify that the correct thresholds were used.</p>
<p>Be aware that each time you close and reopen the Shiny app, it will
automatically revert back to the default configuration file. Also –
remember the sequencing! If you forget to upload the customized
configuration file prior to running the QC function, it will use the
default thresholds.</p>
<p>Want to try this yourself? Click <a href="Custom_QC_Config_ECO66G20_test1.R">here</a> to download a test
file.</p>
</div>
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@



<div id="evaluate-thresholds" class="section level2">
<h2>Evaluate Thresholds</h2>
<p>If you have one or more years of continuous data for a site, we
strongly encourage you to evaluate the performance of the QC test
thresholds for each parameter at that site and customize the
configuration file if needed.</p>
<p>Click <a href="EvaluateThresholds.zip">here</a> for instructions on
how to generate pivot tables and plots to evaluate thresholds for the
Unrealistic values (‘Gross range’) and Spike tests.</p>
<p>Another option is to use R code written by Tim Martin (Minnesota DNR)
to generate statistical outputs that can help inform thresholds for all
four QC tests. Click <a href="TimMartin_R_ThresholdEval.zip">here</a> to
download the R scripts and instructions.</p>
<p>To aid with documentation of the threshold evaluation process, click
<a href="ThresholdsCheckWorksheet_20220826.xlsx">here</a> to download an
Excel worksheet that lists the default thresholds for each parameter and
has a column where you can enter the customized values for a given site
as well as rationale for making the changes. When doing this, make sure
you consider what units you are using (as well as what the defaults
are), since units have a large effect on thresholds.</p>
</div>
111 changes: 111 additions & 0 deletions inst/shiny-examples/ContDataQC/www/RMD_HTML/App_3c4_QCThresh_Edit.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@



<div id="edit-thresholds" class="section level2">
<h2>Edit Thresholds</h2>
<p>Click <a href="EditingQCtestThresholds_20220117.pdf">here</a> for
instructions on how to edit thresholds and create customized
configuration files. There are two options (see below). For either
option, save the customized configuration file in a folder that is easy
to find so that you can reuse the file each time you QC data for that
site. It is helpful to include the StationID and other relevant
information in the file name (for example, if thresholds are customized
for a particular season).</p>
<p>After you save the file, go to the ‘upload custom thresholds’ tab,
browse to the custom thresholds file, upload the file and then, under
‘Main functions’, generate the QC reports. If you forget the upload
step, the default QC thresholds will be used.</p>
<div id="option-1-download-the-default-configuration-file-open-the-file-in-r-or-notepad-make-edits-and-save." class="section level3">
<h3>Option 1, download the default configuration file, open the file in
R or Notepad++, make edits and save.</h3>
<p>Configuration files can be downloaded here -</p>
<ul>
<li><p><a href="Config_default.zip">Default</a>, with hydrologic
parameters in feet</p></li>
<li><p><a href="Config_Lakes_meters.zip">Hydrologic parameters in
meters</a> Navigate to the ‘QC tests and calculations’ section. Edit
thresholds as desired. Save the file.</p></li>
</ul>
<p>When you download the configuration file(s), you will see two types
of files: an R file (.R) and a text (.txt) file for users who do not
have R installed on their computer. You can download R from this
website:
<a href="https://cran.r-project.org" target="_blank">https://cran.r-project.org</a>.
You may also want to download
<a href="https://www.rstudio.com/products/rstudio/download/" target="_blank">RStudio</a>,
which is a separate program but is a useful code editor and interface
(IDE) for R. If you cannot install R on your computer, we recommend use
Notepad or
<a href="https://notepad-plus-plus.org/downloads" target="_blank">Notepad++</a>
(some people prefer Notepad++ because it has color-coding).</p>
</div>
<div id="option-2-use-the-interactive-functions-shown-left-and-below." class="section level3">
<h3>Option 2, use the interactive functions shown left and below.</h3>
<ul>
<li><p>Select a parameter using the drop-down menu (upper left
box)</p></li>
<li><p>Edit thresholds using the boxes below</p></li>
<li><p>Save changes (for each parameter)</p></li>
<li><p>Download custom thresholds file</p></li>
</ul>
<table>
<caption>Default Units</caption>
<thead>
<tr class="header">
<th align="left">Parameter</th>
<th align="left">Units</th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td align="left">Temperature</td>
<td align="left">Celsius</td>
</tr>
<tr class="even">
<td align="left">Sensor depth and water level</td>
<td align="left">feet</td>
</tr>
<tr class="odd">
<td align="left">Discharge</td>
<td align="left">ft3/s</td>
</tr>
<tr class="even">
<td align="left">Dissolved oxygen</td>
<td align="left">mg/L</td>
</tr>
<tr class="odd">
<td align="left">DO pctsat</td>
<td align="left">%</td>
</tr>
<tr class="even">
<td align="left">Conductivity</td>
<td align="left">uS/cm</td>
</tr>
<tr class="odd">
<td align="left">Pressure</td>
<td align="left">psi</td>
</tr>
<tr class="even">
<td align="left">pH</td>
<td align="left">SU</td>
</tr>
<tr class="odd">
<td align="left">Turbidity</td>
<td align="left">NTU</td>
</tr>
<tr class="even">
<td align="left">Salinity</td>
<td align="left">ppt</td>
</tr>
<tr class="odd">
<td align="left">Chlorophylla</td>
<td align="left">g/cm3</td>
</tr>
</tbody>
</table>
<p>If you do not want to use the default units, you will need to use
Option 1.</p>
<p>Acknowledgment: Tim Martin (MN DNR) wrote the R code that we used for
the interactive functions below.</p>
</div>
</div>

0 comments on commit 6158f6d

Please sign in to comment.