API Sec Lab update

.gitignore

@@ -1,4 +1,5 @@
 /docs/_build/*
 #!/docs/_build/html/
 #/docs/_build/html/*
-#!/docs/_build/html/assets/
+#!/docs/_build/html/assets/
+docs/__pycache__/custom_roles.cpython-311.pyc

docs/class4/class4.rst

@@ -12,9 +12,6 @@ Lab Maintainers:
 
 |
 
-
-.. note:: For the TechXchange 2024 - Please use this `Teams chat <https://teams.microsoft.com/l/channel/19%3Aca3de856a85c47b1809f9803723c45d2%40thread.tacv2/XC%20API%20Security%20Lab%20Support?groupId=100b8a10-f3d0-4d73-bc24-c463f941d064&tenantId=dd3dfd2f-6a3b-40d1-9be0-bf8327d81c50>`_ if you have any issues, questions or feedback for the XC API security lab
-
 * Please start this `UDF <https://udf.f5.com/b/1cb733ca-3e77-4783-a91d-98d469eb220b#documentation>`_ to get an invite via F5 email to reset your password for the "f5-emea-workshop" tenant. 
 *	You are going to run a script to generate traffic at the end of “Class 4 - Dynamic API Protection – Enable API discovery” and it can take up to 2 hours maximum to see all results in the Dashboard for the following “API Discovery outcomes” lab section. There is also a note about this in the specific lab section, maybe you combine it with a break 😉.
 * Please make sure you complete the XC API Security Lab before the UDF shuts down because your XC account and configuration will be removed automatically when the UDF deployment stops.

docs/class4/module2/lab1/lab1.rst

@@ -1,4 +1,4 @@
-Enable API Validation
+Enable API validation
 =====================
 
 In the previous section, we enabled API Protection. API Protection is based on rules (allow, deny), but API Validation goes deeper into the validation.

docs/class4/module2/lab2/lab2.rst

@@ -1,79 +1,72 @@
-Enable API discovery
-====================
+Enable API code scanning discovery (under construction)
+=======================================================
 
-In the previous section, we enabled ``API Validation`` in order to enforce protection on ``what we know`` from the OpenAPI Spec file.
-But we kept the ``Fall Through Mode`` to ``Allow`` so that we do not break the application or impact business down when DevOps push a new version of the API, but SecOps are not ready or up to date.
+F5 solutions can detect and protect APIs during the full API develoment lifecycle which includes also to learn API endpoints and further information to e.g. build the schema from the source code the developers create and maintain on the code repository.
 
-The ``API Discovery`` will provide visility for SecOps in order to see this ``Drift``. This Drift is the difference between ``what we know`` and ``what we see / what is consumed``
+We use the "Sentence application" source code for this lab: https://github.com/ca-scans/sentence-source-code-v2
 
-.. image:: ../pictures/slide-api-discovery.png
-   :align: center
-   :scale: 40%
 
-Enable Endpoint Discovery
--------------------------
+.. note:: There is also a video and a FAQ available for F5 employees
 
-* Edit your Load Balancer again, go to API Protection and enable ``API Discovery`` (keep the default settings)
+ * FAQ -  XC API code scan `FAQ <https://f5.sharepoint.com/sites/SalesCoP/SitePages/XC-API-code-scan-FAQ.aspx>`_
+ * Video – API discovery from `code <https://f5.sharepoint.com/sites/SalesCoP/SitePages/API-discovery-from-code---introduction-video.aspx>`_
 
-.. image:: ../pictures/enable-api-discovery.png
-   :align: left
-   :scale: 40%
+|
 
-Enable PII Discovery
---------------------
+Enable Code Base Integration
+----------------------------
 
-OWASP Top10 API requires to detect and discover sensitive datas in Requests and Responses. F5 Distributed Cloud supports this and provides a predefined list of known PII (Personal Identifiable Information), such as:
+* Goto Web App & API Protection > API Management > Code Base Ingration
+* Add a new "Code Base Integration" with the following values
 
-* email
-* credit card number
-* US Social Security Number
-* IP address
+  * Name: ``github-sentence``
+  * Code base: ``Github Integration``
+  * Github UserName: ``please check the "internal" tab in the UDF deployment for the username``
+  * GitHub Personal Access Token: ``please check the "internal" tab in the UDF deployment for the token``
 
-But you want to detect your own PII, such as:
+
+.. image:: ../pictures/code-base-integration-username.png
+   :align: left
 
-* Country Social Security Number
-* Mobile Phone Number
-* Etc ...
+|
 
-Create custom PII
-^^^^^^^^^^^^^^^^^
+* Click on Configure under "GitHub Personal Access Token" to enter the token into "Secret to Blindfold". 
 
-* In Sensitive Data Detection, click on ``configure``.
-* Add two new ``Defined Custom Sensitive Data Types``, enable detection for ``All Endpoint, Request and Response, Value Pattern``
+.. image:: ../pictures/code-base-integration-token-blindfold.png
+      :align: left
 
-  * For France/French SSN, use this regex ``[12][0-9]{2}(0[1-9]|1[0-2])(2[AB]|[0-9]{2})[0-9]{3}[0-9]{3}([0-9]{2})``
-
-  * For France/French Mobile Phone, use this regex ``^(?:(?:\+|00)33|0)\s*[1-9](?:[\s.-]*\d{2}){4}$``
 
-.. image:: ../pictures/pii.png
+* Click Apply and Save and Exit
+* Go to settings and change it as shown in the screenshot below to display further information. The Health status should change from "INITIALIZING" to "CONNECTED".
+
+.. image:: ../pictures/code-base-integration-initializing-and-show-settings.png
    :align: left
-   :scale: 50%
 
 |
 
-.. image:: ../pictures/pii2.png
-   :align: left
-   :scale: 50%
-|
+* XC is going to scan the code, so give it some time until you see further updates on e.g. the "Number of API Repositories"  
 
-.. image:: ../pictures/pii-both.png
+.. image:: ../pictures/code-base-integration-connected.png
    :align: left
-   :scale: 50%
-|
 
-* SAVE your Load Balancer changes
+|
 
+Assign the Code Base Integration to the Load Balancer
+-----------------------------------------------------
 
-Run the traffic generator script
---------------------------------
+* Edit your Load Balancer, go to the API Protection and enable API Discovery (keep the default settings)
 
-It is time to run a traffic generator script to populate the logs and the AI/ML engines.
 
-* SSH or WEBSSH to the Jumphost
-* Run this script into /home/ubuntu/api-protection-lab folder
+.. image:: ../pictures/API-discovery-enable.png
+   :align: left
 
-.. code-block:: none
+1. In the same configuration screen, look for API repositories and click on Configure
+2. In "Select Code Base Integrations" click on "Add Item"
+3. Select the previously created Code Base "github-sentence"
+4. Select "Selected API Repositories"
+5. Select ca-scans/sentence-source-code-v2
 
-   cd /home/ubuntu/api-protection-lab
-   bash api-all.sh sentence-re-$$makeId$$.workshop.emea.f5se.com
+.. image:: ../pictures/select-api-repo-code.png
+   :align: left
 
+Select Apply twice and Save and Exit

docs/class4/module2/lab3/lab3.rst

@@ -1,111 +1,105 @@
-API Discovery outcomes
-======================
+Enable API traffic discovery
+============================
 
-.. note:: The "discovery" scheduler runs on a random interval within a two hours time window and therefore it can take up to 2 hours (maximum) to see all results in the Dashboard for the "API Discovery outcomes" lab section. You can also continue with the next lab "Advanced Protection - "JWT validation and access control" (module 3) and continue here later.
+In the previous section, we enabled ``API Validation`` in order to enforce protection on ``what we know`` from the OpenAPI Spec file.
+But we kept the ``Fall Through Mode`` to ``Allow`` so that we do not break the application or impact business down when DevOps push a new version of the API, but SecOps are not ready or up to date.
 
-Endpoint Discovery
-------------------
+The ``API Discovery`` will provide visility for SecOps in order to see this ``Drift``. This Drift is the difference between ``what we know`` and ``what we see / what is consumed``
 
-* Goto Web App & API Protection > Overview > Security > Dashboard
-* Click on your Application Load Balancer
-* Click on ``API Endpoints`` to see the endpoints in the the "Table" view.
+.. image:: ../pictures/slide-api-discovery.png
+   :align: center
+   :scale: 40%
 
-.. image:: ../pictures/api-endpoints-table.png
-   :align: left
-   :scale: 50%
-
-Understand the API Discovery elements
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-On the top left corner, there are 3 important elements:
-
-* **Inventory** : Endpoints known from the OpenAPI Spec file
-
-  * In our lab, there are 3 endpoints know (adjectives, animals, locations)
+But OWASP Top10 requires also to provide visibility on PII (Personal Identifiable Information) in order to avoid Data Leakage. To do so, we will enable ``Sensitive Data Disvovery``
 
-* **Discovered** : Endpoints that the XC platform has discovered/learned from live traffic (known and unknown endpoints)
-* **Shadow** : Endpoints that have been ``Discovered`` but are **NOT PART** of the ``Inventory``
+Enable API Endpoint Discovery
+-----------------------------
 
-You can filter on ``Shadow`` only to show the ``/colors`` endpoint as a Shadow API.
+* If you skipped the lab2 "Enable Code Scanning Discovery, edit your Load Balancer again, go to API Protection and enable ``API Discovery`` (keep the default settings)
 
-.. image:: ../pictures/shadow.png
+.. image:: ../pictures/enable-traffic-discovery.png
    :align: left
-   :scale: 50%
+   :scale: 40%
 
-Go deeper into the discovery
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Enable Sensitive Data Discovery
+-------------------------------
 
-* Click on the ``/colors`` shadow API endpoint. A pop-up will appear on the right side of the screen.
-* You can see on the top right corner, 2 actions
+OWASP Top10 API requires to detect and discover sensitive datas in Requests and Responses. F5 Distributed Cloud supports this and provides a predefined list (+400) of known PII (Personal Identifiable Information), such as:
 
-  * **API Protection rule** : if you want to block this endpoint. Let's say SecOps have this power to block unknown endpoints.
+* email
+* credit card number
+* US Social Security Number
+* IP address
 
-  * **Rate Limiting** : if you want to Rate Limit this endpoint because SecOps don't have the full power and don't want to break the app.
+.. note:: By default, a list is already assigned to the Load Balancer
 
-* Click on the ``Discovered`` tab and navigate into the sub-menus. You will see all the details discovered by the platform.
+  .. image:: ../pictures/default-pii-setting.png
+     :align: left
+     :scale: 50%
 
-.. image:: ../pictures/discovered.png
-   :align: left
-   :scale: 50%
 
+But if you want to detect your own PII, such as:
 
-PII Discovery
--------------
+* Country Social Security Number
+* Mobile Phone Number
+* Etc ...
 
-* Click on the ``/animals`` API endpoint. A pop-up will appear on the right side of the screen.
+You must create your own patterns.
 
-  .. image:: ../pictures/pii-1.png
-     :align: left
-     :scale: 50%
+Create custom Sensitive Data Discovery
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-* Click on the ``Discovered`` tab to show discovered sensitive data for requests and responses.
+* In Sensitive Data Discovery, select ``Custom``
+* Add a new item
 
-  .. image:: ../pictures/pii-2.png
-     :align: left
-     :scale: 50%
+  * Give a name: custom-frenchies
+  * Select the Compliance Frameworks required for this API Application. We select ``PCI-DSS`` ``GDPR`` 
 
-.. warning:: Dataguard can obfuscate sensitive PII data in the response but currently not for custom created PII configurations. This feature is in the roadmap. OWASP Top 10 does not require to ``hide`` sensitive data.
+.. note:: By selecting PCI-DSS and GDPR, all data patterns classified as PCI-DSS and GDPR will be added.
 
+* But now, we want to add custom patterns to detect frenchy sensitive datas
+* Configure ``Defined Custom Sensitive Data Types``, and add 2 items
 
-Click on the ``Graph`` tab to show the API endpoints in a different view.
+  * Name: ``france-ssn``
+
+    * Data Type Rules: 
+
+      * Value Pattern
+      * Regex Value : ``[12][0-9]{2}(0[1-9]|1[0-2])(2[AB]|[0-9]{2})[0-9]{3}[0-9]{3}([0-9]{2})``
 
-.. image:: ../pictures/octopus.png
-   :align: left
-   :scale: 50%
+    * Mark as Sensitive Data
+    * Mark as PII
+    * Relevant Compliance: ``GDPR``
 
-
-Authentication Discovery
-------------------------
+    .. image:: ../pictures/pii-ssn.png
+       :align: left
+       :scale: 50%
 
-* Click on an endpoint with an ``Authenticated`` state, like **/api/locations**
 
-  .. image:: ../pictures/authenticated-endpoint.png
-     :align: left
-     :scale: 50%
+  * Name: ``france-mobile-phone``
+
+    * Data Type Rules: 
+
+      * Value Pattern
+      * Regex Value : ``^(?:(?:\+|00)33|0)\s*[1-9](?:[\s.-]*\d{2}){4}$``
 
-* Click on ``Discovered`` tab and check the Authentication details
+    * Mark as Sensitive Data
+    * Mark as PII
+    * Relevant Compliance: ``GDPR``
 
-  .. image:: ../pictures/auth-discovery-new.png
-     :align: left
-     :scale: 50%
+* Apply and Save your LB config
 
-* Notice that the auth information collected from the OpenAPI Spec file differs from the discovered auth information. If both don't match, a "Security Posture" is raised.
 
-  .. image:: ../pictures/basic-auth.png
-     :align: left
-     :scale: 50%
+Run the traffic generator script
+--------------------------------
 
-AI/ML Security Posture
-----------------------
+It is time to run a traffic generator script to populate the logs and the AI/ML engines.
 
-* Click on an endpoint with the highest ``Risk Score``
-* And click on the ``Security Posture`` tab
-* Review the recommandations done by the AI/ML engine
+* SSH or WEBSSH to the Jumphost
+* Run this script into /home/ubuntu/api-protection-lab folder
 
-.. image:: ../pictures/security-posture.png
-   :align: left
-   :scale: 50%
+.. code-block:: none
 
-* Click on the ``Evidence`` link to get more details about the logs who generated this security posture.
+   cd /home/ubuntu/api-protection-lab
+   bash api-all.sh sentence-re-$$makeId$$.workshop.emea.f5se.com
 
-.. note:: Congratulation, your application is now protected by a modern engine enforcing (validating) what is provided by the developers, but also providing visibility for unkown traffic.