diff --git a/manageability-and-operations/observability-and-manageability/oci-monitoring/custom-metrics/README.md b/manageability-and-operations/observability-and-manageability/oci-monitoring/custom-metrics/README.md new file mode 100644 index 000000000..ea821fe4c --- /dev/null +++ b/manageability-and-operations/observability-and-manageability/oci-monitoring/custom-metrics/README.md @@ -0,0 +1,16 @@ + +# Custom Metrics + +In this section will bring some examples about how to create OCI Monitoring custom metric namespaces to extend the default, out-of-the-box, OCI Monitoring metrics for OCI resources. + +# Team Publications + +- [Using python SDK to create OCI Monitoring custom metric namespace: Services Limit monitoring example use case](./custom-metric-python-SDK-services-limit-monitoring/README.md) + +# License + +Copyright (c) 2023 Oracle and/or its affiliates. + +Licensed under the Universal Permissive License (UPL), Version 1.0. + +See [LICENSE](https://github.com/oracle-devrel/technology-engineering/blob/folder-structure/LICENSE) for more details. \ No newline at end of file diff --git a/manageability-and-operations/observability-and-manageability/oci-monitoring/custom-metrics/custom-metric-python-SDK-services-limit-monitoring/LICENSE b/manageability-and-operations/observability-and-manageability/oci-monitoring/custom-metrics/custom-metric-python-SDK-services-limit-monitoring/LICENSE new file mode 100644 index 000000000..fe1109d44 --- /dev/null +++ b/manageability-and-operations/observability-and-manageability/oci-monitoring/custom-metrics/custom-metric-python-SDK-services-limit-monitoring/LICENSE @@ -0,0 +1,35 @@ +Copyright (c) 2023 Oracle and/or its affiliates. + +The Universal Permissive License (UPL), Version 1.0 + +Subject to the condition set forth below, permission is hereby granted to any +person obtaining a copy of this software, associated documentation and/or data +(collectively the "Software"), free of charge and under any and all copyright +rights in the Software, and any and all patent rights owned or freely +licensable by each licensor hereunder covering either (i) the unmodified +Software as contributed to or provided by such licensor, or (ii) the Larger +Works (as defined below), to deal in both + +(a) the Software, and +(b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +one is included with the Software (each a "Larger Work" to which the Software +is contributed by such licensors), + +without restriction, including without limitation the rights to copy, create +derivative works of, display, perform, and distribute the Software and make, +use, sell, offer for sale, import, export, have made, and have sold the +Software and the Larger Work(s), and to sublicense the foregoing rights on +either these or other terms. + +This license is subject to the following condition: +The above copyright notice and either this complete permission notice or at +a minimum a reference to the UPL must be included in all copies or +substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. \ No newline at end of file diff --git a/manageability-and-operations/observability-and-manageability/oci-monitoring/custom-metrics/custom-metric-python-SDK-services-limit-monitoring/README.md b/manageability-and-operations/observability-and-manageability/oci-monitoring/custom-metrics/custom-metric-python-SDK-services-limit-monitoring/README.md new file mode 100644 index 000000000..ffd988251 --- /dev/null +++ b/manageability-and-operations/observability-and-manageability/oci-monitoring/custom-metrics/custom-metric-python-SDK-services-limit-monitoring/README.md @@ -0,0 +1,129 @@ + +# Using python SDK to create OCI Monitoring custom metric namespace: Services Limit monitoring example use case + +## 1. INTRODUCTION + +Describes how any user can create an OCI Monitoring ***custom metric namespace*** to being able to extend the default services metric namespaces. For that, we'll support on python SDK to create an script that can be run in an OCI VM (using instance principals authentication), or any other external system (using OCI IAM principals). To cover this educational example, we'll use as an example the creation of a custom metric namespace to monitor the OCI Services Limits usage. With this custom metric namespace, OCI alarms can be created and OCI Notification Service can be used to send the alarm information by different means to allow to create proactively a Service Limit Service Request to increase the limit before causing any disruption in the running services or services to be provisioned. + +## 2. SOLUTION +We can see the overall architecture in the following logical diagram: + +![Logical diagram](./files/Diagrams/custom-metrics-python-SDK-services-limit_solution1.png) + +The script will take care of gathering the OCI Services Limits information and post as a custom metric namespace. The metrics where to post the data will be: + +1. **used**: This is the amount of the Service Limit that it is being used +2. **max_limit**: This is the amount of the Service Limit that we can use (and increase with a SR) +3. **available**: This is the remaining amount of the Service Limit that we can use until get the max_limit + +Not all the Services Limits are equal as they depend of the scope they have (Global, Regional or Availability Domain). We'll introduce the characteristics of a Service Limit as **Dimensions** of the metric, so we can select the Service Limit, the limit name and the scope to filter the specific limit. Thus, we'll have as **Dimensions**: + +1. **service_name**: The name of the OCI Service that the Service Limit belongs to (e.g.: Compute) +2. **limit_name**: The name of the Service Limit (e.g.: bm-standard2-52-count) +3. **AD**: If the Service Limit has an AD availability, we can filter for the AD where we would like to filter the metric + +With this raw data, we could be able to build Alarm Definitions on the specific Services Limits that we would like to monitor, as usually customers do not use all the possible OCI services. + +Optionally, they could use any OCI Notification Service to being notified when the alarm fires receiving the notification message in any of the supported options. Some of them will enable the integration with 3rd party services. + +## 3. SCRIPT'S LOGIC + +Here we'll focus in reviewing the logic behind using the OCI Python SDK to get the objectives to monitor the Services Limits with OCI Monitoring. + +We use 3 different OCI services API calls to gather the needed information to create the custom metric namespace for the Services Limit monitoring, these are: + +1. [Identity and Access Management Service API](https://docs.oracle.com/en-us/iaas/api/#/en/identity/20160918/): To use the [ListAvailabilityDomains](https://docs.oracle.com/en-us/iaas/api/#/en/identity/20160918/AvailabilityDomain/ListAvailabilityDomains) API Call to get the list of the availability domains existing in the input tenancy. +2. [Monitoring API](https://docs.oracle.com/en-us/iaas/api/#/en/monitoring/20180401/): To use the [PostMetricData](https://docs.oracle.com/en-us/iaas/api/#/en/monitoring/20180401/MetricData/PostMetricData), to post the metrics information in the existing (or non yet existing) custom service metric namespace. If it doesn't exist yet, it just creates it so we don't need to explicitly create it before. +3. [Service Limits API](https://docs.oracle.com/en-us/iaas/api/#/en/limits/20181025/): To use the [ListLimitsDefinition](https://docs.oracle.com/en-us/iaas/api/#/en/limits/20181025/LimitDefinitionSummary/ListLimitDefinitions) to get the full list of Service Limits in a given compartment and the [GetResouceAvailability](https://docs.oracle.com/en-us/iaas/api/#/en/limits/20181025/ResourceAvailability/GetResourceAvailability), to gather the used and available limits depending in the scope (AD, regional, whole tenancy), of the known Service Limits from the previous list. + +Basically the **logic** is: + +```` +Start +Parse input arguments (compartment_ocid, region) +Setup the OCI connection authentication (OCI IAM Config or Instance Principals) +Initialise the clients for the different API calls (IAM, Monitoring, Service Limits) +Gather the full list of Service Limits Definitions sorted by Service Limit name +For the list of Service Limit names + If the scope is Availability Domain + For every AD + Get the limits and usage for the Service Limit name within this AD + Post in the custom metric namespace the metric with the dimension of the Service Limit, name and AD with the resources used, available and the service limit maximum + For the non-AD Service Limit names (Global or Regional) + Get the limits and usage for the Service Limit name + Post in the custom metric namespace the metric with the dimension of the Service Limit and name with the resources used, available and the service limit maximum +End +```` + +## 4. REQUIREMENTS + +We have different requirements depending on the variant of this asset that we would use: + +1. **IAM User** + * An existing OCI IAM user with an API signing key + * An .oci/config profile for the tenancy, with the previous IAM user details + * A policy granting the user to: + * inspect resource-availability in tenancy + * inspect limits in tenancy + * use metrics in tenancy + +2. **Instance Principal** + * An existing dynamic group identifying the VM where to run the script as member of the group + * A policy granting the dynamic group to: + * inspect resource-availability in tenancy + * inspect limits in tenancy + * use metrics in tenancy + +3. ***Common requirements*** +The VM where to run the script must have installed python3 with the following required packages installed with pip: + * **oci** + +## 5. INPUT + +The required input arguments are: + +* **compartment_ocid** with the OCID of your tenancy root compartment.  +* **region** where you want to get the Services Limits with regional scope and where to publish metrics + +## 6. OUTPUT + +Every time the script is run, it will feed a custom metric namespace called "**limits_metrics**" in the tenancy's root compartment with the information of the Services Limits usage. + +You can check the custom metric extension from the OCI Metrics Explorer, where you will be able also to create an alarms from an specific metric query. + +## 7. GETTING STARTED + +To execute the script: + +1. Ensure that the requirements are met with your desired variant (using IAM user or Instance Principals) +2. Upload the script into your administration VM inside OCI (IAM user or Instance Principals), or outside OCI (IAM user only) +3. To execute the script: + * For the IAM User principals authentication method, execute: + ```` + $ python3 serviceLimitsMetricsIAM.py -c/--compartment_ocid -r/--region + ```` + * The script is available [here](./files/Scripts/postServiceLimitsMetricsIAM.py) + * For the Instance principal authentication method, execute: + ```` + $ python3 serviceLimitsMetricsIP.py -c/--compartment_ocid -r/--region + ```` + * The script is available [here](./files/Scripts/postServiceLimitsMetricsIP.py) + +## 8.KNOWN PROBLEMS + +None at this point. + +## 9.RELEASE NOTES + +There 2 scripts with same code except for the OCI authentication. Both scripts will be maintained in parallel with same versions: + +2023-07-25 (version 0.2). compartment_ocid & region are now input arguments. Fixed region wide limits publishing. +2023-07-18 (version 0.1). Initial public release. + +# LICENSE + +Copyright (c) 2023 Oracle and/or its affiliates. + +Licensed under the Universal Permissive License (UPL), Version 1.0. + +See [LICENSE](https://github.com/oracle-devrel/technology-engineering/blob/folder-structure/LICENSE) for more details. diff --git a/manageability-and-operations/observability-and-manageability/oci-monitoring/custom-metrics/custom-metric-python-SDK-services-limit-monitoring/files/Diagrams/custom-metrics-python-SDK-services-limit_solution1.png b/manageability-and-operations/observability-and-manageability/oci-monitoring/custom-metrics/custom-metric-python-SDK-services-limit-monitoring/files/Diagrams/custom-metrics-python-SDK-services-limit_solution1.png new file mode 100644 index 000000000..a3162ede8 Binary files /dev/null and b/manageability-and-operations/observability-and-manageability/oci-monitoring/custom-metrics/custom-metric-python-SDK-services-limit-monitoring/files/Diagrams/custom-metrics-python-SDK-services-limit_solution1.png differ diff --git a/manageability-and-operations/observability-and-manageability/oci-monitoring/custom-metrics/custom-metric-python-SDK-services-limit-monitoring/files/Scripts/postServiceLimitsMetricsIAM.py b/manageability-and-operations/observability-and-manageability/oci-monitoring/custom-metrics/custom-metric-python-SDK-services-limit-monitoring/files/Scripts/postServiceLimitsMetricsIAM.py new file mode 100644 index 000000000..eea861675 --- /dev/null +++ b/manageability-and-operations/observability-and-manageability/oci-monitoring/custom-metrics/custom-metric-python-SDK-services-limit-monitoring/files/Scripts/postServiceLimitsMetricsIAM.py @@ -0,0 +1,235 @@ +# Copyright (c) 2023 Oracle and/or its affiliates. + +# The Universal Permissive License (UPL), Version 1.0 + +# Subject to the condition set forth below, permission is hereby granted to any +# person obtaining a copy of this software, associated documentation and/or data +# (collectively the "Software"), free of charge and under any and all copyright +# rights in the Software, and any and all patent rights owned or freely +# licensable by each licensor hereunder covering either (i) the unmodified +# Software as contributed to or provided by such licensor, or (ii) the Larger +# Works (as defined below), to deal in both + +# (a) the Software, and +# (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +# one is included with the Software (each a "Larger Work" to which the Software +# is contributed by such licensors), + +# without restriction, including without limitation the rights to copy, create +# derivative works of, display, perform, and distribute the Software and make, +# use, sell, offer for sale, import, export, have made, and have sold the +# Software and the Larger Work(s), and to sublicense the foregoing rights on +# either these or other terms. + +# This license is subject to the following condition: +# The above copyright notice and either this complete permission notice or at +# a minimum a reference to the UPL must be included in all copies or +# substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +### +# This is a sample python script that post a custom metric (service_limits) to oci monitoring based on Tenancy Service Limits information. +# Run this script on any host with python with access to your tenancy. +# Command: python3 postServiceLimitsMetricsIAM.py +# Version: 0.2 +### + +import oci, datetime, json, argparse + +from oci.config import from_file +from pytz import timezone + +# Functions definition + +# postMetric: posts custom monitoring metric information for compartment, metric(used,available,max_limit) and with its dimensions (service, limit name and availability domain(if AD specific)) +def postMetric(compartment_ocid, m_name, s_name,l_name, value, monitoring_client, a_domain=None): + # Get the timestamp for setup the monitoring metric post information + times_stamp = datetime.datetime.now(timezone('UTC')) + if a_domain is None: + post_metric_data_response = monitoring_client.post_metric_data( + post_metric_data_details=oci.monitoring.models.PostMetricDataDetails( + metric_data=[ + oci.monitoring.models.MetricDataDetails( + namespace = "limits_metrics", + compartment_id = compartment_ocid, + name = m_name, + dimensions={ + 'service_name': s_name, + 'limit_name': l_name, + }, + datapoints=[ + oci.monitoring.models.Datapoint( + timestamp=datetime.datetime.strftime( + times_stamp,"%Y-%m-%dT%H:%M:%S.%fZ"), + value = value)] + )] + ) + ) + else: + post_metric_data_response = monitoring_client.post_metric_data( + post_metric_data_details=oci.monitoring.models.PostMetricDataDetails( + metric_data=[ + oci.monitoring.models.MetricDataDetails( + namespace = "limits_metrics", + compartment_id = compartment_ocid, + name = m_name, + dimensions={ + 'service_name': s_name, + 'limit_name': l_name, + 'availability_domain': a_domain + }, + datapoints=[ + oci.monitoring.models.Datapoint( + timestamp=datetime.datetime.strftime( + times_stamp,"%Y-%m-%dT%H:%M:%S.%fZ"), + value = value)] + )] + ) + ) + return post_metric_data_response + +# getServiceLimitsUsage: gets the existing limits for a service and limit name in a compartment and, if AD specific, for its AD +def getServiceLimitsUsage(s_name, l_name, compartment_ocid, limits_client, a_domain=None): + if a_domain is None: + # We gather the service limit usage + get_resource_availability_response = limits_client.get_resource_availability(service_name = s_name, limit_name = l_name, compartment_id = compartment_ocid) + + # We need to gather the service limit limit + list_limit_values_response = limits_client.list_limit_values(compartment_id = compartment_ocid, service_name = s_name, limit = 1) + else: + # We gather the service limit usage + get_resource_availability_response = limits_client.get_resource_availability(service_name = s_name, limit_name = l_name, compartment_id = compartment_ocid, availability_domain = a_domain) + + # We need to gather the service limit limit + list_limit_values_response = limits_client.list_limit_values(compartment_id = compartment_ocid, service_name = s_name, availability_domain = a_domain, limit = 1) + + usage = json.loads(str(get_resource_availability_response.data)) + used = usage["used"] + available = usage["available"] + limit_limit = json.loads(str(list_limit_values_response.data[0])) + max_limit = limit_limit["value"] + + # We create the return type data dictionary + values = { + "used": used, + "available": available, + "max_limit": max_limit + } + return json.dumps(values) + + +# Parse the input arguments +argParser = argparse.ArgumentParser() +argParser.add_argument("-c", "--compartment_ocid", help="Your root's compartment OCID, typically tenancy OCID",required=True) +argParser.add_argument("-r", "--region", help="The region where you want to get the Services Limits. E.g.: eu-frankfurt-1", required=True) +args = argParser.parse_args() +compartment_ocid = args.compartment_ocid +region = args.region + +# Start: +now = datetime.datetime.now().strftime("%d/%m/%Y %H:%M:%S") +print("[", now,"] Starting OCI Service Metrics limits gathering and customer metrics post...") + +# Using IAM user credentials and default profile (~/.oci/config) +config = from_file() +# We change the default profile region for the given region which we want to get services limits information and publish the custom metric +config["region"] = region + +# We gather the list of availability domains in the region +identity_client = oci.identity.IdentityClient(config) + +# Let's check if the provided region is valid and it is among the subscribed regions +list_regions_response = identity_client.list_regions() + +for e in list_regions_response.data: + reg = json.loads(str(e)) + if reg["name"] == region: + break +else: + print("ERROR: Wrong or non-subscribed OCI region, exiting...") + exit() + +try: + list_availability_domains_response = identity_client.list_availability_domains(compartment_id = compartment_ocid) +except Exception as ex: + print("ERROR: The given compartment is wrong or you aren't authorized to list the availability domains, exiting...") + exit(1) + +service_endpoint = "https://telemetry-ingestion." + region + ".oraclecloud.com" + +# Initialize service client with default config file +monitoring_client = oci.monitoring.MonitoringClient(config,service_endpoint=service_endpoint) + +# Get Service Limits + +# Initialize service client with default config file +limits_client = oci.limits.LimitsClient(config) + +# Send the request to service, some parameters are not required, see API doc for more info +list_limit_definitions_response = limits_client.list_limit_definitions( + compartment_id = compartment_ocid, + sort_by="name", + sort_order="ASC") + +# We iterate the list of all the service limits +for x in list_limit_definitions_response.data: + limit = json.loads(str(x)) + s_name = limit["service_name"] + l_name = limit["name"] + l_scope = limit["scope_type"] + + # If the resource limit has an AD scope, we have to specify the AD or we'll get an API 400 response + if l_scope == "AD" : + # We have AD scope, we've to gather the limit for all the ADs in the region and include the availabilityDomain + for AD in list_availability_domains_response.data: + + a_domain = json.loads(str(AD)) + + limit_usage = json.loads(getServiceLimitsUsage(s_name, l_name, compartment_ocid, limits_client, a_domain["name"])) + + # Posting custom metrics to oci monitoring for each of the metrics (max, used, available) + + # Max limit + postMetric(compartment_ocid, "max_limit", s_name, l_name, limit_usage["max_limit"], monitoring_client, a_domain["name"]) + + # Used + postMetric(compartment_ocid, "used", s_name, l_name, limit_usage["used"], monitoring_client, a_domain = a_domain["name"]) + + # Available + postMetric(compartment_ocid, "available", s_name, l_name, limit_usage["available"], monitoring_client, a_domain = a_domain["name"]) + + else: + # We are in GLOBAL or REGION case + + limit_usage = json.loads(getServiceLimitsUsage(s_name, l_name, compartment_ocid, limits_client)) + + max_limit = limit_usage["max_limit"] + used = limit_usage["used"] + + if max_limit == "null" : + continue + + # Posting custom metrics to oci monitoring for each of the metrics (max, used, available) + + # Max limit + postMetric(compartment_ocid, "max_limit", s_name, l_name, max_limit, monitoring_client) + + if used is None : + continue + + # Used + postMetric(compartment_ocid, "used", s_name, l_name, used, monitoring_client) + + # Available + postMetric(compartment_ocid, "used", s_name, l_name, limit_usage["available"], monitoring_client) + +# Finish: +now = datetime.datetime.now().strftime("%d/%m/%Y %H:%M:%S") +print("[", now,"] Finish OCI Service Metrics limits gathering and customer metrics post.") diff --git a/manageability-and-operations/observability-and-manageability/oci-monitoring/custom-metrics/custom-metric-python-SDK-services-limit-monitoring/files/Scripts/postServiceLimitsMetricsIP.py b/manageability-and-operations/observability-and-manageability/oci-monitoring/custom-metrics/custom-metric-python-SDK-services-limit-monitoring/files/Scripts/postServiceLimitsMetricsIP.py new file mode 100644 index 000000000..b7140cfdf --- /dev/null +++ b/manageability-and-operations/observability-and-manageability/oci-monitoring/custom-metrics/custom-metric-python-SDK-services-limit-monitoring/files/Scripts/postServiceLimitsMetricsIP.py @@ -0,0 +1,234 @@ +# Copyright (c) 2023 Oracle and/or its affiliates. + +# The Universal Permissive License (UPL), Version 1.0 + +# Subject to the condition set forth below, permission is hereby granted to any +# person obtaining a copy of this software, associated documentation and/or data +# (collectively the "Software"), free of charge and under any and all copyright +# rights in the Software, and any and all patent rights owned or freely +# licensable by each licensor hereunder covering either (i) the unmodified +# Software as contributed to or provided by such licensor, or (ii) the Larger +# Works (as defined below), to deal in both + +# (a) the Software, and +# (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +# one is included with the Software (each a "Larger Work" to which the Software +# is contributed by such licensors), + +# without restriction, including without limitation the rights to copy, create +# derivative works of, display, perform, and distribute the Software and make, +# use, sell, offer for sale, import, export, have made, and have sold the +# Software and the Larger Work(s), and to sublicense the foregoing rights on +# either these or other terms. + +# This license is subject to the following condition: +# The above copyright notice and either this complete permission notice or at +# a minimum a reference to the UPL must be included in all copies or +# substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +### +# This is a sample python script that post a custom metric (service_limits) to oci monitoring based on Tenancy Service Limits information. +# Run this script on any host with python with access to your tenancy. +# Command: python3 postServiceLimitsMetricsIP.py +# Version: 0.2 +### + +import oci, datetime, json, argparse + +from pytz import timezone + +# Functions definition + +# postMetric: posts custom monitoring metric information for compartment, metric(used,available,max_limit) and with its dimensions (service, limit name and availability domain(if AD specific)) +def postMetric(compartment_ocid, m_name, s_name,l_name, value, monitoring_client, a_domain=None): + # Get the timestamp for setup the monitoring metric post information + times_stamp = datetime.datetime.now(timezone('UTC')) + if a_domain is None: + post_metric_data_response = monitoring_client.post_metric_data( + post_metric_data_details=oci.monitoring.models.PostMetricDataDetails( + metric_data=[ + oci.monitoring.models.MetricDataDetails( + namespace = "limits_metrics", + compartment_id = compartment_ocid, + name = m_name, + dimensions={ + 'service_name': s_name, + 'limit_name': l_name, + }, + datapoints=[ + oci.monitoring.models.Datapoint( + timestamp=datetime.datetime.strftime( + times_stamp,"%Y-%m-%dT%H:%M:%S.%fZ"), + value = value)] + )] + ) + ) + else: + post_metric_data_response = monitoring_client.post_metric_data( + post_metric_data_details=oci.monitoring.models.PostMetricDataDetails( + metric_data=[ + oci.monitoring.models.MetricDataDetails( + namespace = "limits_metrics", + compartment_id = compartment_ocid, + name = m_name, + dimensions={ + 'service_name': s_name, + 'limit_name': l_name, + 'availability_domain': a_domain + }, + datapoints=[ + oci.monitoring.models.Datapoint( + timestamp=datetime.datetime.strftime( + times_stamp,"%Y-%m-%dT%H:%M:%S.%fZ"), + value = value)] + )] + ) + ) + return post_metric_data_response + +# getServiceLimitsUsage: gets the existing limits for a service and limit name in a compartment and, if AD specific, for its AD +def getServiceLimitsUsage(s_name, l_name, compartment_ocid, limits_client, a_domain=None): + if a_domain is None: + # We gather the service limit usage + get_resource_availability_response = limits_client.get_resource_availability(service_name = s_name, limit_name = l_name, compartment_id = compartment_ocid) + + # We need to gather the service limit limit + list_limit_values_response = limits_client.list_limit_values(compartment_id = compartment_ocid, service_name = s_name, limit = 1) + else: + # We gather the service limit usage + get_resource_availability_response = limits_client.get_resource_availability(service_name = s_name, limit_name = l_name, compartment_id = compartment_ocid, availability_domain = a_domain) + + # We need to gather the service limit limit + list_limit_values_response = limits_client.list_limit_values(compartment_id = compartment_ocid, service_name = s_name, availability_domain = a_domain, limit = 1) + + usage = json.loads(str(get_resource_availability_response.data)) + used = usage["used"] + available = usage["available"] + limit_limit = json.loads(str(list_limit_values_response.data[0])) + max_limit = limit_limit["value"] + + # We create the return type data dictionary + values = { + "used": used, + "available": available, + "max_limit": max_limit + } + return json.dumps(values) + + +# Parse the input arguments +argParser = argparse.ArgumentParser() +argParser.add_argument("-c", "--compartment_ocid", help="Your root's compartment OCID, typically tenancy OCID",required=True) +argParser.add_argument("-r", "--region", help="The region where you want to get the Services Limits. E.g.: eu-frankfurt-1", required=True) +args = argParser.parse_args() +compartment_ocid = args.compartment_ocid +region = args.region + + +# Start: +now = datetime.datetime.now().strftime("%d/%m/%Y %H:%M:%S") +print("[", now,"] Starting OCI Service Metrics limits gathering and customer metrics post...") + +# Setup the signer for the instance principal auth method +signer = oci.auth.signers.InstancePrincipalsSecurityTokenSigner() + +# We gather the list of availability domains in the region +identity_client = oci.identity.IdentityClient(config={'region': region}, signer=signer) + +# Let's check if the provided region is valid and it is among the subscribed regions +list_regions_response = identity_client.list_regions() + +for e in list_regions_response.data: + reg = json.loads(str(e)) + if reg["name"] == region: + break +else: + print("ERROR: Wrong or non-subscribed OCI region, exiting...") + exit() + +try: + list_availability_domains_response = identity_client.list_availability_domains(compartment_id = compartment_ocid) +except Exception as ex: + print("ERROR: The given compartment is wrong or you aren't authorized to list the availability domains, exiting...") + exit(1) + +service_endpoint = "https://telemetry-ingestion." + region + ".oraclecloud.com" + +# Initialize service client with default config file +monitoring_client = oci.monitoring.MonitoringClient(config={'region': region},service_endpoint=service_endpoint,signer=signer) + +# Get Service Limits + +# Initialize service client with default config file +limits_client = oci.limits.LimitsClient(config={'region': region}, signer=signer) + +# Send the request to service, some parameters are not required, see API +# doc for more info +list_limit_definitions_response = limits_client.list_limit_definitions( + compartment_id = compartment_ocid, + sort_by="name", + sort_order="ASC") + +# We iterate the list of all the service limits +for x in list_limit_definitions_response.data: + limit = json.loads(str(x)) + s_name = limit["service_name"] + l_name = limit["name"] + l_scope = limit["scope_type"] + + # If the resource limit has an AD scope, we have to specify the AD or we'll get an API 400 response + if l_scope == "AD" : + # We have AD scope, we've to gather the limit for all the ADs in the region and include the availabilityDomain + for AD in list_availability_domains_response.data: + + a_domain = json.loads(str(AD)) + + limit_usage = json.loads(getServiceLimitsUsage(s_name, l_name, compartment_ocid, limits_client, a_domain["name"])) + + # Posting custom metrics to oci monitoring for each of the metrics (max, used, available) + + # Max limit + postMetric(compartment_ocid, "max_limit", s_name, l_name, limit_usage["max_limit"], monitoring_client, a_domain = a_domain["name"]) + + # Used + postMetric(compartment_ocid, "used", s_name, l_name, limit_usage["used"], monitoring_client, a_domain = a_domain["name"]) + + # Available + postMetric(compartment_ocid, "available", s_name, l_name, limit_usage["available"], monitoring_client, a_domain = a_domain["name"]) + + else: + # We are in GLOBAL or REGION case + + limit_usage = json.loads(getServiceLimitsUsage(s_name, l_name, compartment_ocid, limits_client)) + + max_limit = limit_usage["max_limit"] + used = limit_usage["used"] + + if max_limit == "null" : + continue + + # Posting custom metrics to oci monitoring for each of the metrics (max, used, available) + + # Max limit + postMetric(compartment_ocid, "max_limit", s_name, l_name, max_limit, monitoring_client) + + if used is None : + continue + + # Used + postMetric(compartment_ocid, "used", s_name, l_name, used, monitoring_client) + + # Available + postMetric(compartment_ocid, "used", s_name, l_name, limit_usage["available"], monitoring_client) + +# Finish: +now = datetime.datetime.now().strftime("%d/%m/%Y %H:%M:%S") +print("[", now,"] Finish OCI Service Metrics limits gathering and customer metrics post.")