[PERF-274] Include additional randomness in tests to avoid StaleObjec…

…tExceptions (#53)

* Added additional randomness to tests

* Added additional randomness and removed primary key updates that result in 409

* Cautionary note for anyone stumbling on these docs

* Better type handling

* Descriptors need to use instance var instead of class because of inheritance

Co-authored-by: Stephen A. Fuqua <stephen@safnet.com>

docs/old/how-to-create-tests.md

@@ -1,5 +1,8 @@
 # How to Create Tests
 
+| ❗❗❗ These instructions are out of date, applicable to the original version of the performance testing. ❗❗❗ |
+| ---------------------------------------------------------------------------------------------------------- |
+
 ## Resource Tests
 
 When creating a test for a new resource, you will be creating 5 different classes:
@@ -41,7 +44,7 @@ Perform the following steps to create a factory (let's call our example resource
    from .. import APIFactory
    from ..descriptors.utils import build_descriptor
    from ..utils import RandomSuffixAttribute
-   
+
    class CourseFactory(APIFactory):
        courseTitle = 'Algebra I'
        educationOrganizationReference = factory.Dict(
@@ -242,7 +245,7 @@ Perform the following steps to create a simple volume test:
     * The method name must be decorated with `@task` so locust knows that it's a locust task
     * The first two parameters for `run_scenario()` include the name of the attribute to be updated and its corresponding value
     * Each additional argument must be formatted as shown above in the second scenario, since those are the values that differ from the first
-    * Note that if there were no values to update, you wouldn't pass any arguments 
+    * Note that if there were no values to update, you wouldn't pass any arguments
     * Normally, your first scenario would just take in the update values since the factory will have all of its attributes
     * The second scenario would take in the update values, along with any key-value pairs
 
@@ -353,4 +356,4 @@ Perform the following steps to create a pipeclean test for a composite:
     * The remaining pipeclean tests contain the list of the enrollment composite resources and inherit from `EdFiCompositePipecleanTestBase`, as expected.
     * The list of composite resources is located near the top since it is shared by most of the pipeclean tests
 
-3. After adding this, you should be finished with these tests and they should run. Make sure the client and pipeclean test class names follow the same naming convention and are on the correct file path. Remember to replace 'enrollment' with the name of your new composite.
+3. After adding this, you should be finished with these tests and they should run. Make sure the client and pipeclean test class names follow the same naming convention and are on the correct file path. Remember to replace 'enrollment' with the name of your new composite.

docs/old/lab-extension-testing.md

@@ -1,5 +1,8 @@
 # Lab Exercise - Testing Extensions
 
+| ❗❗❗ These instructions are out of date, applicable to the original version of the performance testing. ❗❗❗ |
+| ---------------------------------------------------------------------------------------------------------- |
+
 This lab exercise, developed for the 2018 bootcamp, walks through the process of
 adding test cases for extensions to the API.
 
@@ -188,7 +191,7 @@ Some highlights:
 * Because Locust runs test clients in parallel, the total requests per second is
   not the average of the individual values. In this case we managed
   approximately 42 requests per second.
-  
+
 (!) These data are from a small-ish VM installed in single-server mode, which is far
 from ideal. In other words, these numbers are not reflective of what a good
 production setup can achieve.
@@ -377,4 +380,4 @@ Re-run the performance tests to see what the impact of this index is.
 
 Conclusion: adding this index, at this volume of data, does not have a clear
 negative impact, and is therefore the risk of introducing a performance problem
-in production is very low.
+in production is very low.

src/edfi-performance-test/edfi_performance_test/api/client/calendar_date.py

@@ -1,4 +1,4 @@
-# SPDX-License-Identifier: Apache-2.0
+# SPDX-License-Identifier: Apache-2.0
 # Licensed to the Ed-Fi Alliance under one or more agreements.
 # The Ed-Fi Alliance licenses this file to you under the Apache License, Version 2.0.
 # See the LICENSE and NOTICES files in the project root for more information.
@@ -33,6 +33,8 @@ def create_with_dependencies(self, **kwargs):
                 "calendarCode"
             ],
             calendarReference__schoolId=school_id,
-            calendarReference__schoolYear=2014,
+            calendarReference__schoolYear=calendar_reference["attributes"][
+                "schoolYearTypeReference"
+            ]["schoolYear"],
             **kwargs
         )

src/edfi-performance-test/edfi_performance_test/api/client/course_offering.py

@@ -15,15 +15,15 @@ class CourseOfferingClient(EdFiAPIClient):
 
     def create_with_dependencies(self, **kwargs):
         school_id = kwargs.pop("schoolId", SchoolClient.shared_elementary_school_id())
-
+        school_year = kwargs.get("schoolYear", 2014)
         session_reference = self.session_client.create_with_dependencies(
-            schoolId=school_id
+            schoolId=school_id, schoolYear=school_year
         )
 
         return self.create_using_dependencies(
             session_reference,
             sessionReference__schoolId=school_id,
-            sessionReference__schoolYear=2014,
+            sessionReference__schoolYear=school_year,
             sessionReference__sessionName=session_reference["attributes"][
                 "sessionName"
             ],

src/edfi-performance-test/edfi_performance_test/api/client/section.py

@@ -23,12 +23,14 @@ class SectionClient(EdFiAPIClient):
 
     def create_with_dependencies(self, **kwargs):
         school_id = kwargs.get("schoolId", SchoolClient.shared_elementary_school_id())
+        school_year = kwargs.get("schoolYear", 2014)
         custom_course_code = kwargs.pop("courseCode", "ELA-01")
 
         # Create a course offering and its dependencies
         course_offering_reference = (
             self.course_offering_client.create_with_dependencies(
                 schoolId=school_id,
+                schoolYear=school_year,
                 courseReference__courseCode=custom_course_code,
                 courseReference__educationOrganizationId=school_id,
                 schoolReference__schoolId=school_id,
@@ -57,6 +59,7 @@ def create_with_dependencies(self, **kwargs):
                 "localCourseCode"
             ],
             courseOfferingReference__schoolId=school_id,
+            courseOfferingReference__schoolYear=school_year,
             courseOfferingReference__sessionName=course_offering_attrs[
                 "sessionReference"
             ]["sessionName"],
@@ -111,9 +114,10 @@ def create_with_dependencies(self, **kwargs):
             schoolId=school_id,
         )
         calendar_date_attrs = calendar_date_reference["attributes"]
+        calendarSchoolYear = calendar_date_attrs["calendarReference"]["schoolYear"]
 
         section_reference = self.section_client.create_with_dependencies(
-            schoolId=school_id,
+            schoolId=school_id, schoolYear=calendarSchoolYear
         )
         section_attrs = section_reference["attributes"]
 
@@ -123,9 +127,11 @@ def create_with_dependencies(self, **kwargs):
                 {"section_client": section_reference},
             ],
             calendarDateReference__schoolId=school_id,
+            calendarDateReference__schoolYear=calendarSchoolYear,
             calendarDateReference__calendarCode=calendar_date_attrs[
                 "calendarReference"
             ]["calendarCode"],
+            calendarDateReference__date=calendar_date_attrs["date"],
             sectionReference__sectionIdentifier=section_attrs["sectionIdentifier"],
             sectionReference__localCourseCode=section_attrs["courseOfferingReference"][
                 "localCourseCode"

src/edfi-performance-test/edfi_performance_test/api/client/session.py

@@ -17,13 +17,15 @@ class SessionClient(EdFiAPIClient):
 
     def create_with_dependencies(self, **kwargs):
         school_id = kwargs.pop("schoolId", SchoolClient.shared_elementary_school_id())
-
+        school_year = kwargs.get("schoolYear", 2014)
         # Create two grading periods
         period_1_reference = self.grading_period_client.create_with_dependencies(
             schoolReference__schoolId=school_id,
+            schoolYearTypeReference__schoolYear=school_year,
         )
         period_2_reference = self.grading_period_client.create_with_dependencies(
             schoolReference__schoolId=school_id,
+            schoolYearTypeReference__schoolYear=school_year,
             beginDate="2014-10-06",
             endDate="2014-12-15",
             totalInstructionalDays=30,
@@ -39,6 +41,7 @@ def create_with_dependencies(self, **kwargs):
                 {"period_2_reference": period_2_reference},
             ],
             schoolReference__schoolId=school_id,
+            schoolYearTypeReference__schoolYear=school_year,
             gradingPeriods__0__gradingPeriodReference__periodSequence=period_1_reference[
                 "attributes"
             ][

src/edfi-performance-test/edfi_performance_test/factories/resources/calendar.py

@@ -8,13 +8,16 @@
 from edfi_performance_test.api.client.school import SchoolClient
 from edfi_performance_test.factories.resources.api_factory import APIFactory
 from edfi_performance_test.factories.descriptors.utils import build_descriptor
-from edfi_performance_test.factories.utils import RandomSuffixAttribute
+from edfi_performance_test.factories.utils import (
+    RandomSuffixAttribute,
+    RandomSchoolYearAttribute,
+)
 
 
 class CalendarFactory(APIFactory):
     schoolYearTypeReference = factory.Dict(
         {
-            "schoolYear": 2014,
+            "schoolYear": RandomSchoolYearAttribute(),
         }
     )
     calendarTypeDescriptor = build_descriptor("CalendarType", "IEP")

src/edfi-performance-test/edfi_performance_test/factories/resources/calendar_date.py

@@ -6,7 +6,9 @@
 import factory
 
 from edfi_performance_test.api.client.school import SchoolClient
-from edfi_performance_test.factories.utils import current_year
+from edfi_performance_test.factories.utils import (
+    RandomDateAttribute,
+)
 from edfi_performance_test.factories.resources.api_factory import APIFactory
 from edfi_performance_test.factories.descriptors.utils import build_descriptor_dicts
 
@@ -16,8 +18,8 @@ class CalendarDateFactory(APIFactory):
         dict(
             calendarCode="107SS111111",
             schoolId=SchoolClient.shared_elementary_school_id(),
-            schoolYear=current_year(),
+            schoolYear=2014,
         )
     )
     calendarEvents = build_descriptor_dicts("CalendarEvent", ["Holiday"])
-    date = "2014-09-16"
+    date = RandomDateAttribute()

src/edfi-performance-test/edfi_performance_test/factories/resources/cohort.py

@@ -7,12 +7,12 @@
 
 from edfi_performance_test.api.client.school import SchoolClient
 from edfi_performance_test.factories.resources.api_factory import APIFactory
-from edfi_performance_test.factories.utils import RandomSuffixAttribute
+from edfi_performance_test.factories.utils import UniqueIdAttribute
 from edfi_performance_test.factories.descriptors.utils import build_descriptor
 
 
 class CohortFactory(APIFactory):
-    cohortIdentifier = RandomSuffixAttribute("1")
+    cohortIdentifier = UniqueIdAttribute(num_chars=20)
     educationOrganizationReference = factory.Dict(
         {"educationOrganizationId": SchoolClient.shared_elementary_school_id()}
     )

src/edfi-performance-test/edfi_performance_test/factories/resources/discipline.py

@@ -8,12 +8,12 @@
 from edfi_performance_test.api.client.school import SchoolClient
 from edfi_performance_test.factories.resources.api_factory import APIFactory
 from edfi_performance_test.factories.descriptors.utils import build_descriptor
-from edfi_performance_test.factories.utils import formatted_date, UniqueIdAttribute
+from edfi_performance_test.factories.utils import RandomDateAttribute, UniqueIdAttribute
 
 
 class DisciplineActionFactory(APIFactory):
     disciplineActionIdentifier = UniqueIdAttribute(num_chars=20)
-    disciplineDate = formatted_date(9, 20)
+    disciplineDate = RandomDateAttribute()
     disciplines = factory.List(
         [
             factory.Dict(
@@ -50,7 +50,7 @@ class DisciplineActionFactory(APIFactory):
 
 
 class DisciplineIncidentFactory(APIFactory):
-    incidentDate = formatted_date(9, 25)
+    incidentDate = RandomDateAttribute()
     incidentIdentifier = UniqueIdAttribute(num_chars=20)
     schoolReference = factory.Dict(
         dict(schoolId=SchoolClient.shared_elementary_school_id())

src/edfi-performance-test/edfi_performance_test/factories/resources/graduation_plan.py

@@ -4,17 +4,21 @@
 # See the LICENSE and NOTICES files in the project root for more information.
 
 import factory
-
+import random
 from edfi_performance_test.factories.resources.api_factory import APIFactory
 from edfi_performance_test.factories.descriptors.utils import build_descriptor
+from edfi_performance_test.factories.utils import RandomSchoolYearAttribute
 
 
 class GraduationPlanFactory(APIFactory):
     totalRequiredCredits = 28
-    graduationPlanTypeDescriptor = build_descriptor("GraduationPlanType", "Recommended")
+    graduationPlanTypeDescriptor = build_descriptor(
+        "GraduationPlanType",
+        random.choice(["Recommended", "Distinguished", "Minimum", "Standard"]),
+    )
     educationOrganizationReference = factory.Dict(
         dict(educationOrganizationId=None),
     )
     graduationSchoolYearTypeReference = factory.Dict(
-        dict(schoolYear=2014),
+        dict(schoolYear=RandomSchoolYearAttribute())
     )

src/edfi-performance-test/edfi_performance_test/factories/resources/staff.py

@@ -1,4 +1,4 @@
-# SPDX-License-Identifier: Apache-2.0
+# SPDX-License-Identifier: Apache-2.0
 # Licensed to the Ed-Fi Alliance under one or more agreements.
 # The Ed-Fi Alliance licenses this file to you under the Apache License, Version 2.0.
 # See the LICENSE and NOTICES files in the project root for more information.
@@ -96,7 +96,7 @@ class StaffAbsenceEventFactory(APIFactory):
 
 
 class StaffCohortAssociationFactory(APIFactory):
-    beginDate = "2014-09-14"
+    beginDate = RandomDateAttribute()
     staffReference = factory.Dict(dict(staffUniqueId=None))  # Must be entered by user
     cohortReference = factory.Dict(
         dict(

src/edfi-performance-test/edfi_performance_test/factories/resources/student.py

@@ -107,7 +107,7 @@ class StudentCohortAssociationFactory(APIFactory):
             educationOrganizationId=SchoolClient.shared_elementary_school_id(),  # Prepopulated school
         )
     )
-    beginDate = formatted_date(9, 14)
+    beginDate = RandomDateAttribute()
 
 
 class StudentDisciplineIncidentAssociationFactory(APIFactory):

src/edfi-performance-test/edfi_performance_test/factories/utils.py

@@ -82,6 +82,15 @@ def evaluate(self, instance, step, extra):
         )
 
 
+class RandomSchoolYearAttribute(declarations.BaseDeclaration):
+    """
+    Returns a random date between 1991 & 2050.
+    """
+
+    def evaluate(self, instance, step, extra):
+        return random.randint(1991, 2050)
+
+
 class RandomSuffixAttribute(LazyAttribute):
     """
     Subclasses `factory.LazyAttribute` to append a random string of characters.

src/edfi-performance-test/edfi_performance_test/helpers/argparser.py

@@ -82,7 +82,7 @@ def parse_main_arguments() -> MainArguments:
         help="Deliberately introduce requests that result in failure",
         action="store_true",  # default false
         env_var="PERF_FAIL_DELIBERATELY",
-        default=False
+        default=False,
     )
     parser.add(  # type: ignore
         "-c",

src/edfi-performance-test/edfi_performance_test/tasks/pipeclean/descriptors.py

@@ -12,16 +12,17 @@
 
 
 class DescriptorPipecleanTestBase(EdFiPipecleanTestBase):
-    update_attribute_name = "codeValue"
-    update_attribute_value = random_chars(15)
 
     def __init__(self, descriptor: str, parent, *args, **kwargs):
         super(DescriptorPipecleanTestBase, self).__init__(parent, *args, **kwargs)
-        self.namespace = "{}{}".format(descriptor[0].upper(), descriptor[1:])
-        self._api_client.factory.namespace = "uri://ed-fi.org/{}Descriptor".format(
-            descriptor.title()
-        )
-        self._api_client.endpoint = "{}Descriptors".format(descriptor)
+
+        self.update_attribute_name = "codeValue"
+        self.update_attribute_value = random_chars(15)
+
+        self.namespace = f"{descriptor[0].upper()}{descriptor[1:]}"
+
+        self._api_client.factory.namespace = f"uri://ed-fi.org/{descriptor.title()}Descriptor"
+        self._api_client.endpoint = f"{descriptor}Descriptors"
 
     def generate_client_class(self) -> Any:
         class_path = (

src/edfi-performance-test/edfi_performance_test/tasks/pipeclean/ed_fi_pipeclean_test_base.py

@@ -131,7 +131,7 @@ class EdFiPipecleanTaskSequence(SequentialTaskSet):
     def __init__(self, *args, **kwargs):
         super(EdFiPipecleanTaskSequence, self).__init__(*args, **kwargs)
         EdFiAPIClient.client = self.client
-        EdFiAPIClient.token = None
+        EdFiAPIClient.token = ""
 
 
 class EdFiPipecleanTestTerminator(TaskSet):
@@ -149,4 +149,5 @@ class EdFiPipecleanTestTerminator(TaskSet):
 
     @task
     def finish_pipeclean_test_run(self):
-        self.user.environment.runner.quit()
+        if self.user.environment.runner is not None:
+            self.user.environment.runner.quit()

src/edfi-performance-test/edfi_performance_test/tasks/volume/calendar.py

@@ -5,10 +5,8 @@
 
 from locust import task
 
-from edfi_performance_test.api.client.school import SchoolClient
 from edfi_performance_test.factories.descriptors.utils import build_descriptor
 from edfi_performance_test.factories.descriptors.utils import build_descriptor_dicts
-from edfi_performance_test.factories.utils import RandomSuffixAttribute
 from edfi_performance_test.tasks.volume.ed_fi_volume_test_base import EdFiVolumeTestBase
 
 
@@ -22,8 +20,6 @@ def run_calendar_scenarios(self):
         self.run_scenario(
             "gradeLevels",
             build_descriptor_dicts("GradeLevel", ["Ninth grade", "Tenth grade"]),
-            calendarCode=RandomSuffixAttribute("IEP001"),
-            schoolReference__schoolId=SchoolClient.shared_high_school_id(),
             gradeLevels=build_descriptor_dicts("GradeLevel", ["Ninth grade"]),
         )
 

src/edfi-performance-test/edfi_performance_test/tasks/volume/calendar_date.py

@@ -7,7 +7,6 @@
 
 from edfi_performance_test.api.client.school import SchoolClient
 from edfi_performance_test.factories.descriptors.utils import build_descriptor_dicts
-from edfi_performance_test.factories.utils import RandomSuffixAttribute
 from edfi_performance_test.tasks.volume.ed_fi_volume_test_base import EdFiVolumeTestBase
 
 
@@ -26,5 +25,4 @@ def run_calendar_scenarios(self):
             build_descriptor_dicts("CalendarEvent", ["Instructional day"]),
             calendarEvents=build_descriptor_dicts("CalendarEvent", ["Holiday"]),
             schoolId=SchoolClient.shared_high_school_id(),
-            calendarCode=RandomSuffixAttribute("IEP001"),
         )