Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding config option to set the timeout for the Elastic request #91

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Adding config option to set the timeout for the Elastic request #91

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
218c47a
Adding the option 'es_timeout' to the App 'fn_elasticsearch'
smoreau Sep 23, 2023
09a9b68
Adding the option 'es_timeout' to the App 'fn_elasticsearch'
smoreau Sep 23, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions fn_elasticsearch/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@
## Release Notes
| Version | Date | Notes |
| ------- | ---- | ----- |
| 1.0.9 | 09/2023 | Added support for setting Timeout request |
| 1.0.9 | 06/2022 | Added support for disabling SSL Certificate verification |
| 1.0.8 | | Pinned dependency ``elasticsearch~=7.17`` |
| 1.0.7 | | Add support for AppHost |
Expand Down Expand Up @@ -150,6 +151,7 @@ The following table provides the settings you need to configure the app. These s
| **es_datastore_scheme** | Yes | `<HTTPS OR HTTP>` | *If HTTPS is provided, an SSL Context is configured for the connection* |
| **es_datastore_url** | Yes | `<ELASTICSEARCH_URL>` | *URL of the elastic instance* |
| **es_use_http** | Yes | `<True OR False>` | *If true, connection to the elastic instance uses HTTP. Set to False if the es_veryify_certs is True.* |
| **es_timeout** | No | `10` | *The number of seconds to timeout after when making a request to the elastic instance.* |


## Function - ElasticSearch Utilities: Query
Expand Down
7 changes: 4 additions & 3 deletions fn_elasticsearch/fn_elasticsearch/components/fn_elasticsearch_query.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@ def _fn_elasticsearch_query_function(self, event, *args, **kwargs):
"es_auth_password", True)
ELASTICSEARCH_VERIFY_CERTS = str_to_bool(value=helper.get_config_option(
"es_verify_certs", True))
ELASTICSEARCH_TIMEOUT = kwargs.get("es_timeout", None)
# Get the function parameters:
es_index = kwargs.get("es_index") # text
es_doc_type = kwargs.get("es_doc_type") # text
Expand Down Expand Up @@ -105,15 +106,15 @@ def _fn_elasticsearch_query_function(self, event, *args, **kwargs):
# Connect to the ElasticSearch instance
es = Elasticsearch(ELASTICSEARCH_SCHEME.lower() + "://" + ELASTICSEARCH_URL, verify_certs=ELASTICSEARCH_VERIFY_CERTS,
ssl_context=context, http_auth=(ELASTICSEARCH_USERNAME, ELASTICSEARCH_PASSWORD),
connection_class=ProxiedConnection, proxies=self.requestscommon.get_proxies())
connection_class=ProxiedConnection, proxies=self.requestscommon.get_proxies(), timeout=ELASTICSEARCH_TIMEOUT)
else:
# This handles for HTTP statements
if ELASTICSEARCH_BOOL_HTTP_AUTH:
es = Elasticsearch([ELASTICSEARCH_URL], verify_certs=False, cafile=ELASTICSEARCH_CERT, http_auth=(
ELASTICSEARCH_USERNAME, ELASTICSEARCH_PASSWORD), connection_class=ProxiedConnection, proxies=self.requestscommon.get_proxies())
ELASTICSEARCH_USERNAME, ELASTICSEARCH_PASSWORD), connection_class=ProxiedConnection, proxies=self.requestscommon.get_proxies(), timeout=ELASTICSEARCH_TIMEOUT)
else:
es = Elasticsearch(
[ELASTICSEARCH_URL], verify_certs=False, cafile=ELASTICSEARCH_CERT, connection_class=ProxiedConnection, proxies=self.requestscommon.get_proxies())
[ELASTICSEARCH_URL], verify_certs=False, cafile=ELASTICSEARCH_CERT, connection_class=ProxiedConnection, proxies=self.requestscommon.get_proxies(), timeout=ELASTICSEARCH_TIMEOUT)
except Exception as e:
raise FunctionError(
"Encountered error while connecting to ElasticSearch {0}".format(e))
Expand Down
7 changes: 4 additions & 3 deletions fn_elasticsearch/fn_elasticsearch/util/selftest.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ def selftest_function(opts):
ELASTICSEARCH_PASSWORD = helper.get_config_option("es_auth_password", True)
ELASTICSEARCH_VERIFY_CERTS = str_to_bool(value=helper.get_config_option(
"es_verify_certs", True))
ELASTICSEARCH_TIMEOUT = kwargs.get("es_timeout", None)

log = logging.getLogger(__name__)

Expand All @@ -49,15 +50,15 @@ def selftest_function(opts):
context = create_default_context(cafile=ELASTICSEARCH_CERT)
# Connect to the ElasticSearch instance
es = Elasticsearch(ELASTICSEARCH_SCHEME.lower() + "://" + ELASTICSEARCH_URL, verify_certs=ELASTICSEARCH_VERIFY_CERTS,
ssl_context=context, http_auth=(ELASTICSEARCH_USERNAME, ELASTICSEARCH_PASSWORD))
ssl_context=context, http_auth=(ELASTICSEARCH_USERNAME, ELASTICSEARCH_PASSWORD), timeout=ELASTICSEARCH_TIMEOUT)
else:
# Connect without to Elastic without HTTPS
if ELASTICSEARCH_BOOL_HTTP_AUTH:
es = Elasticsearch([ELASTICSEARCH_URL], verify_certs=ELASTICSEARCH_VERIFY_CERTS, cafile=ELASTICSEARCH_CERT, http_auth=(
ELASTICSEARCH_USERNAME, ELASTICSEARCH_PASSWORD))
ELASTICSEARCH_USERNAME, ELASTICSEARCH_PASSWORD), timeout=ELASTICSEARCH_TIMEOUT)
else:
es = Elasticsearch(
[ELASTICSEARCH_URL], verify_certs=ELASTICSEARCH_VERIFY_CERTS, cafile=ELASTICSEARCH_CERT)
[ELASTICSEARCH_URL], verify_certs=ELASTICSEARCH_VERIFY_CERTS, cafile=ELASTICSEARCH_CERT, timeout=ELASTICSEARCH_TIMEOUT)
try:
# If we cant ping the ES instance we can't query it
if not es.ping():
Expand Down