24.3 fb notification revamps (#685)

* - In Blood Draws Today (All, Animal Care, and Vet Staff), I changed 'Assignment Status' column header to 'Unassigned' and 'Completion Status' to 'Incomplete'.  I also sorted all results so Incomplete draws show up first.
- In BloodDrawReviewTriggerNotification, I made it so warnings only send when the draw has a date of today.
- In BloodDrawReviewDailyNotification, I added a check to verify that there are no upcoming blood draws that will overdraw.

- TODO: After testing this on test servers, make sure to remove test dates in BloodDrawsTodayAll and BloodDrawReviewDailyNotification.

* Migrated automated test changes from 23.11 to 24.3.  23.11_notificationRevamps is now ready to be deleted.

* Added blood overdraw trigger notification.

* Registered new notification.

* Updated code so message sends when testing/running in browser.  Updated log for wnprc_triggers.js for when function is run to help debug until issue with webpack generation is fixed.

* Updated overdraw notification.  Notification is now created everytime blood is updated.  Logic is run inside notification to determine if current draw is an overdraw.  If it is an overdraw, the message is sent.

* Fixed issue where 'Animal Replacement Fee' showed incorrectly in the revamped Death Notification.  This occurred when the Type of Death resulted in a fee in the table 'ehr_lookups' > 'death_cause', but the 'prepaid' field was empty in the table 'study' > 'demographics'.  This was due to labkeys table lookup resulting in a string 'null' instead of a real null when using the TableSelector.  My new notifications use a new query function i wrote, but this Death Notification used the old TableSelector method.

* Reordered messages in BloodDrawReviewDailyNotification.java so overdraws are listed first.

* -NotificationToolkit.java: Updated getWeightFromAnimalID() and getSexFromAnimalID() so they use new query function getTableMultiRowMultiColumnWithFieldKeys() instead of old function getTableRowAsList().  Also moved DeathNecropsyObject and DeathDemographicObject to DeathNotificationRevamp.java.
- DeathNotificationRevamp.java: Moved DeathNecropsyObject and DeathDemographicObject here from NotificationToolkit.java.  Also cleaned up these objects and updated these them to use the new query getTableMultiRowMultiColumnWithFieldKeys.

* Removed automated tests that don't aren't finished.

* Added 2 new functions to NotificationToolkit for creating URL's.  This is to avoid hardcoding URL's per new LabKey policy.  Implimented these 2 new functions in DeathNotificationRevamp.java for getting the necropsy and animal abstract URL's.

* Adding automated tests for notifications.

* - WNPRC_EHRTest.java: Added notification setup function, multiple notification check functions, and a toolkit object containing reusable test functions that are commonly used when writing different tests.
- BloodOverdrawTriggerNotification.java & BloodDrawReviewTriggerNotification.java: Added a reset function to clear the data after notification is triggered.  This is because there are null checks that don't work when artifacts are left over from previous instances.
- NotificationToolkit.java: Added null check to checkIfBloodDrawIsOverdraw() because empty data was crashing the function.

* - DeathNotificationRevamp.java: Added a null check to make sure class doesn't cause a failure if there's no taskID in the row returned from the query.
- WNPRC_EHRTest.java: Added a check for death notification and prenatal death notification.

* Added null check before sending blood overdraw trigger manually.  The built in labkey notification setup just doesn't send a notification if there's a null body, but for my trigger notifications where I send it manually, I should check for null.

* Moved the automated tests to the end of the list and removed the @test flag.

* Disabled site-wide notifications.  Also disable individual notifications.

* Re-added 'test' tag before function.  Originally removed this because Marty said it was not necessary if I called the function in doSetup(), but it's not showing up in the console log anymore.  Trying to re-add this.

* Added try/catch for insertValueIntoBloodBilledByDataset due to duplicate data being uploaded.  Looks like labkey's loadBloodBilledByLookup() already uploads the same values.

* Added try/catch for remaining functions that insert into datasets.

* Changed a query function in checkIfAnimalIsAlive() so it no longer references my functions that use QviewObject.  Need to evenutally phase out all these functions as my new query functions work much better.

* Removing migrated alerts from ehrcron to Java based notifications

* Added comment to leave commented-out code block alone.  This will be used for future notifications, and it's very tough coding this correctly in the right order.  Please do not delete.

* Removed function call for notification tests in doSetup().  This was recommended by Binal via a LabKey ticket; it caused my test to run twice since the @test annotation already exists.

* - TriggerScriptHelper.java: Added trigger call for new AnimalRequestUpdateNotificationRevamp notification.  Also moved call for old notification inside if/else statement so 'on/off' status is checked before sending.  Will delete the old call after vefifying new version works well.

- WNPRC_EHRModule.java: Registered new EmptyNotificationRevamp and AnimalRequestUpdateNotificationRevamp notifications.

- AdminAlertsNotificationRevamp.java: Updated this notification so the days of the week are displayed in the correct order.  Also fixed an error with the wrong results being queried (needed to update the filter).

- AnimalRequestNotificationRevamp.java: Added test data to be set when notification is triggered from 'Run Report in Browser'.  Also added a resetClass function.

- AnimalRequestUpdateNotificationRevamp.java:  Created this new revamped notification.

- ColonyAlertsLiteNotificationRevamp.java: Updated this notification to use the new EmptyNotificationRevamp notification.  This is sent instead of the original notification if there is no data to be sent.

- EmptyNotificationRevamp.java: Created this new revamped notification.  This is sent when certain notifications have no data to send.  This is to prevent users from receiving empty emails, but allows Daniel and I to see that the notifications are still being sent.

- NotificationToolkit.java: Added function to send the new EmptyNotificationRevamp.java notification.

* - TriggerScriptHelper.java: Added 'sendManually' function call that I had forgotten.
- BloodDrawReviewDailyNotification.java: Added 2 extra checks requested by blood draw team.
- BloodDrawsTodayAnimalCare.java: Updated this to use the new dummy notification when there is no data to be sent.

* - animal_requests.js: Added fix for fatal issue with qcstatus being different in dataset and form.
- ColonyAlertsNotificationRevamp.java: Added extra query requested by Kim.  This query checks all animals in the 'Assignments' dataset and returns any where the project has expired or the protocol has deactivated.

* - WNPRC_EHRModule.java: Registered new notification.
- TreatmentAlertsNotificationRevamp.java: Created new notification.
- BloodDrawsTodayAll.java: Added 'incomplete count' to the notification (as requested by blood draw team).
- BloodDrawsTodayVetStaff.java: Added functionality so message does not send when there is no data (as requested by blood draw team).

* Removed joda time import statement because it was unused.

---------

Co-authored-by: F. Daniel Nicolalde <dnicolalde@gmail.com>

WNPRC_EHR/resources/queries/wnprc/animal_requests.js

@@ -90,6 +90,24 @@ function onAfterInsert(helper,errors,row){
 function onAfterUpdate(helper,errors,row,oldRow){
     var rowid = row.rowId;
     var hostName = 'https://' + LABKEY.serverName;
-    console.log ("animal_requests.js: New request updated, rowid: "+ rowid);
+    console.log("animal_requests.js: New request updated, rowid: "+ rowid);
+
+    if ("QCStateLabel" in row) {
+        delete row.QCState;
+        row["qcstate"] = row["QCStateLabel"];
+        delete row.QCStateLabel;
+    }
+    if ("QCStateLabel" in oldRow) {
+        oldRow["qcstate"] = oldRow["QCStateLabel"];
+        delete oldRow.QCStateLabel;
+    }
+
+    if ("_publicData" in row) {
+        delete row._publicData;
+    }
+    if ("_publicData" in oldRow) {
+        delete oldRow._publicData;
+    }
+
     WNPRC.Utils.getJavaHelper().sendAnimalRequestNotificationUpdate(rowid, row, oldRow, hostName);
 }

WNPRC_EHR/src/org/labkey/wnprc_ehr/WNPRC_EHRModule.java

@@ -392,7 +392,8 @@ public void registerNotifications() {
                 new ColonyAlertsLiteNotificationRevamp(this),
                 new BloodOverdrawTriggerNotification(this),
                 new EmptyNotificationRevamp(this),
-                new AnimalRequestUpdateNotificationRevamp(this)
+                new AnimalRequestUpdateNotificationRevamp(this),
+                new TreatmentAlertsNotificationRevamp(this)
                 );
 
         for (Notification notification : notifications)

WNPRC_EHR/src/org/labkey/wnprc_ehr/notification/BloodDrawsTodayAll.java

@@ -81,18 +81,17 @@ public String getMessageBodyHTML(Container c, User u) {
 
         // Prints all tables.
         if (myBloodDrawNotificationObject.resultsByArea.isEmpty()) {
-            messageBody.append("There are no scheduled blood draws for this group today.");
+            messageBody.append("There are no scheduled blood draws today.");
         }
         else {
+            messageBody.append("<p><b>REMAINING INCOMPLETE TOTALS:</b><br>\n");
+            messageBody.append("Animal Care: " + myBloodDrawNotificationObject.numIncompleteAnimalCare + "</br>\n");
+            messageBody.append("Research Staff: " + myBloodDrawNotificationObject.numIncompleteResearchStaff + "</br>\n");
+            messageBody.append("SPI: " + myBloodDrawNotificationObject.numIncompleteSPI + "</br>\n");
+            messageBody.append("Vet Staff: " + myBloodDrawNotificationObject.numIncompleteVetStaff + "</p>");
             messageBody.append(myBloodDrawNotificationObject.printTablesAsHTML());
         }
 
-//        // Creates table.
-//        String[] myTableColumns = new String[]{"Id", "Blood Remaining", "Project Assignment", "Completion Status", "Group", "Other Groups Drawing Blood Today"};
-//        NotificationToolkit.NotificationRevampTable myTable = new NotificationToolkit.NotificationRevampTable(myTableColumns, myBloodDrawNotificationObject.myTableData);
-//        myTable.rowColors = myBloodDrawNotificationObject.myTableRowColors;
-//        messageBody.append(myTable.createBasicHTMLTable());
-
         return messageBody.toString();
     }
 
@@ -105,8 +104,11 @@ public static class BloodDrawsTodayObject {
         NotificationToolkit.DateToolkit dateToolkit = new NotificationToolkit.DateToolkit();
 
         ArrayList<String[]> myTableData = new ArrayList<>();        // List of all blood draws as [[id, blood remaining, project assignment, completion status, assigned to]]
-        //        ArrayList<String> myTableRowColors = new ArrayList<>();     // List of all row colors (same length as myTableData).
         HashMap<String, HashMap<String, ArrayList<String[]>>> resultsByArea = new HashMap<>();  // Area(Room(List of draws))
+        Integer numIncompleteSPI = 0;
+        Integer numIncompleteAnimalCare = 0;
+        Integer numIncompleteVetStaff = 0;
+        Integer numIncompleteResearchStaff = 0;
 
 
         //Gets all info for the BloodDrawNotificationObject.
@@ -145,31 +147,37 @@ public static class BloodDrawsTodayObject {
 
                 // Updates id.
                 myCurrentRow[0] = result.get("id");
+
                 // Updates blood remaining.
                 myCurrentRow[1] = result.get("BloodRemaining/AvailBlood");
+
                 // Updates project status (this checks if animal is assigned to a project).
                 if (!result.get("qcstate/label").equals("Request: Denied") && !result.get("projectStatus").isEmpty()) {
                     myCurrentRow[2] = "UNASSIGNED";
                 }
                 else {
                     myCurrentRow[2] = "";
                 }
+
                 // Updates completion status (this checks if blood draw has been completed).
                 if (!result.get("qcstate/label").equals("Completed")) {
                     myCurrentRow[3] = "INCOMPLETE";
                 }
                 else {
                     myCurrentRow[3] = "";
                 }
+
                 // Updates the current group assigned to this animal.
                 myCurrentRow[4] = result.get("billedby/title");
+
                 // Updates the current area.
                 if (!result.get("Id/curLocation/area").isEmpty()) {
                     myCurrentRow[6] = result.get("Id/curLocation/area");
                 }
                 else {
                     myCurrentRow[6] = "Unknown Area";
                 }
+
                 // Updates the current room.
                 if (!result.get("Id/curLocation/room").isEmpty()) {
                     myCurrentRow[7] = result.get("Id/curLocation/room");
@@ -191,18 +199,6 @@ else if (availBlood <= bloodThreshold) {
                         myCurrentRow[8] = "orange";
                     }
                 }
-//                String currentRowColor = "white";
-//                if (!result.get("BloodRemaining/AvailBlood").isEmpty()) {
-//                    Float availBlood = Float.parseFloat(result.get("BloodRemaining/AvailBlood"));
-//                    if (availBlood <= 0) {
-//                        // If blood draw is over limit, color it red.
-//                        currentRowColor = "red";
-//                    }
-//                    else if (availBlood <= bloodThreshold) {
-//                        // If blood draw is over threshold limit, color it orange.
-//                        currentRowColor = "orange";
-//                    }
-//                }
 
                 // Adds the current row to myTableData (based on group being queried).
                 if (assignmentGroup.equals("animalCare")) {
@@ -234,6 +230,24 @@ else if (assignmentGroup.equals("vetStaff")) {
                         myTableData.add(myCurrentRow);
                     }
                 }
+
+                // Updates number of incomplete draws.
+                if (assignmentGroup.equals("all")) {
+                    if (myCurrentRow[3].equals("INCOMPLETE")) {
+                        if (result.get("billedby/title").equals("SPI")) {
+                            numIncompleteSPI++;
+                        }
+                        else if (result.get("billedby/title").equals("Animal Care")) {
+                            numIncompleteAnimalCare++;
+                        }
+                        else if (result.get("billedby/title").equals("Research Staff")) {
+                            numIncompleteResearchStaff++;
+                        }
+                        else if (result.get("billedby/title").equals("Vet Staff")) {
+                            numIncompleteVetStaff++;
+                        }
+                    }
+                }
             }
 
             //Goes through each draw to find draws scheduled for more than one group, then updates myTableData with information.

WNPRC_EHR/src/org/labkey/wnprc_ehr/notification/BloodDrawsTodayVetStaff.java

@@ -69,17 +69,13 @@ public String getMessageBodyHTML(Container c, User u) {
 
         // Creates table.
         if (myBloodDrawNotificationObject.resultsByArea.isEmpty()) {
-            messageBody.append("There are no scheduled blood draws for this group today.");
+            notificationToolkit.sendEmptyNotificationRevamp(c, u, "Blood Draws Today (Vet Staff)");
+            return null;
         }
         else {
             messageBody.append(myBloodDrawNotificationObject.printTablesAsHTML());
+            return messageBody.toString();
         }
 
-//        String[] myTableColumns = new String[]{"Id", "Blood Remaining", "Project Assignment", "Completion Status", "Group", "Other Groups Drawing Blood Today"};
-//        NotificationToolkit.NotificationRevampTable myTable = new NotificationToolkit.NotificationRevampTable(myTableColumns, myBloodDrawNotificationObject.myTableData);
-//        myTable.rowColors = myBloodDrawNotificationObject.myTableRowColors;
-//        messageBody.append(myTable.createBasicHTMLTable());
-
-        return messageBody.toString();
     }
 }

WNPRC_EHR/src/org/labkey/wnprc_ehr/notification/ColonyAlertsNotificationRevamp.java

@@ -27,6 +27,8 @@
 import java.sql.ResultSet;
 
 import java.sql.SQLException;
+import java.text.ParseException;
+import java.text.SimpleDateFormat;
 import java.util.ArrayList;
 import java.util.Calendar;
 import java.util.Collections;
@@ -41,6 +43,8 @@
 
 import javax.script.ScriptEngine;
 
+import static org.labkey.api.search.SearchService._log;
+
 
 /**
  * Created by Alex Schmidt on 12/27/23.
@@ -97,6 +101,12 @@ public String getMessageBodyHTML(Container c, User u) {
         final StringBuilder messageBody = new StringBuilder();
         ColonyInformationObject myColonyAlertObject = new ColonyInformationObject(c, u, "colonyAlert");
 
+        // Creates CSS.
+        messageBody.append(styleToolkit.beginStyle());
+        messageBody.append(styleToolkit.setBasicTableStyle());
+        messageBody.append(styleToolkit.setHeaderRowBackgroundColor("#d9d9d9"));
+        messageBody.append(styleToolkit.endStyle());
+
         //Begins message info.
         messageBody.append("<p> This email contains a series of automatic alerts about the colony.  It was run on: " + dateToolkit.getCurrentTime() + ".</p>");
 
@@ -373,6 +383,19 @@ public String getMessageBodyHTML(Container c, User u) {
             messageBody.append(notificationToolkit.createHyperlink("<p>Click here to view them<br>\n", myColonyAlertObject.totalFinalizedRecordsWithFutureDatesURLView));
             messageBody.append("<hr>\n");
         }
+        //38. Find any animals assigned to an inactivated project or deactivated protocol.
+        if (!myColonyAlertObject.animalsWithInvalidProjectOrProtocol.isEmpty()) {
+            // Creates HTML table to return.
+            String[] myTableColumns = new String[]{"Id", "Project", "Project End Date", "Protocol", "Protocol End Date"};
+            NotificationToolkit.NotificationRevampTable myTable = new NotificationToolkit.NotificationRevampTable(myTableColumns, myColonyAlertObject.animalsWithInvalidProjectOrProtocol);
+
+            // Displays message.
+            messageBody.append("<b>WARNING: There are " + myColonyAlertObject.animalsWithInvalidProjectOrProtocol.size() + " living animals with inactivated projects or deactivated protocols.</b><br>");
+            messageBody.append(myTable.createBasicHTMLTable());
+            messageBody.append(notificationToolkit.createHyperlink("<p>Click here to view all Assignments.<br>\n", myColonyAlertObject.animalsWithInvalidProjectOrProtocolURLView));
+            messageBody.append("<hr>\n");
+
+        }
 
         //Returns string.
         return messageBody.toString();
@@ -469,6 +492,8 @@ public ColonyInformationObject(Container currentContainer, User currentUser, Str
                 getPrenatalDeathsInLastFiveDays();
                 //37. Find the total finalized records with future dates.
                 getTotalFinalizedRecordsWithFutureDates();
+                //38. Find any animals assigned to an inactivated project or deactivated protocol.
+                getAnimalsWithInvalidProjectOrProtocol();
             }
             else if (alertType.equals("colonyManagement")) {
                 // 1. Find all living animals without a weight.
@@ -1356,6 +1381,83 @@ private void getProtocolsNearingAnimalLimitPercentage() {
             this.protocolsNearingAnimalLimitPercentage = returnArray;
             this.protocolsNearingAnimalLimitPercentageURLView = viewQueryURL;
         }
+
+        // Find any animals assigned to an inactivated project or deactivated protocol.
+        ArrayList<String[]> animalsWithInvalidProjectOrProtocol = new ArrayList<>();   //
+        String animalsWithInvalidProjectOrProtocolURLView;
+        private void getAnimalsWithInvalidProjectOrProtocol() {
+            // Creates filter.
+            SimpleFilter myFilter = new SimpleFilter("Id/Dataset/Demographics/calculated_status", "Alive", CompareType.EQUAL);
+//            myFilter.addCondition("project/protocol", "", CompareType.NONBLANK);
+            // Gets columns to retrieve.
+            String[] targetColumns = new String[]{"id", "project", "project/protocol", "project/enddate", "project/protocol/enddate"};
+            // Runs query.
+            ArrayList<HashMap<String, String>> returnArray = notificationToolkit.getTableMultiRowMultiColumnWithFieldKeys(c, u, "study", "Assignment", myFilter, null, targetColumns);
+
+            // Sets up a Try/Catch block to catch date parsing errors.
+            try {
+                // Sets variables.
+                SimpleDateFormat myFormat = new SimpleDateFormat("yyyy-MM-dd");
+                Date formattedCurrentDate = myFormat.parse(dateToolkit.getCurrentTime());
+
+                for (HashMap<String, String> result : returnArray) {
+                    //  0: ID
+                    //  1: Project
+                    //  2: Project End Date
+                    //  3: Protocol
+                    //  4: Protocol End Date
+                    String[] myCurrentRow = new String[5];
+                    myCurrentRow[1] = "";
+                    myCurrentRow[2] = "";
+                    myCurrentRow[3] = "";
+                    myCurrentRow[4] = "";
+
+                    // Retrieves row data.
+                    String currentID = result.get("id");
+                    String currentProject = result.get("project");
+                    String currentProtocol = result.get("project/protocol");
+                    String currentProjectEnd = result.get("project/enddate");
+                    String currentProtocolEnd = result.get("project/protocol/enddate");
+                    Boolean projectOrProtocolExpired = false;
+
+                    // Adds id.
+                    myCurrentRow[0] = currentID;
+                    // Checks project.
+                    if (currentProject != null && currentProjectEnd != null) {
+                        if (!currentProject.isEmpty() && !currentProjectEnd.isEmpty()) {
+                            Date formattedProjectEnd = myFormat.parse(currentProjectEnd);
+                            if (formattedCurrentDate.compareTo(formattedProjectEnd) > 0) {
+                                myCurrentRow[1] = currentProject;
+                                myCurrentRow[2] = currentProjectEnd;
+                                projectOrProtocolExpired = true;
+                            }
+                        }
+                    }
+                    // Checks protocol.
+                    if (currentProtocol != null && currentProtocolEnd != null) {
+                        if (!currentProtocol.isEmpty() && !currentProtocolEnd.isEmpty()) {
+                            Date formattedProtocolEnd = myFormat.parse(currentProtocolEnd);
+                            if (formattedCurrentDate.compareTo(formattedProtocolEnd) > 0) {
+                                myCurrentRow[3] = currentProtocol;
+                                myCurrentRow[4] = currentProtocolEnd;
+                                projectOrProtocolExpired = true;
+                            }
+                        }
+                    }
+
+                    // Adds row to return list if there is an expired project or protocol.
+                    if (projectOrProtocolExpired) {
+                        animalsWithInvalidProjectOrProtocol.add(myCurrentRow);
+                    }
+                }
+            }
+            catch (ParseException e) {
+                _log.error("There was a parsing exception for: ColonyAlertsNotificationRevamp->getAnimalsWithInvalidProjectOrProtocol", e);
+            }
+
+            // Creates url link to the assignments table.
+            this.animalsWithInvalidProjectOrProtocolURLView = notificationToolkit.createQueryURL(c, "execute", "study", "Assignment", null);
+        }
     }
 }