Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Harvest should handle HTTP 414 responses from REST endpoints #3029

Closed
cgrinds opened this issue Jul 2, 2024 · 0 comments · Fixed by #3186
Closed

Harvest should handle HTTP 414 responses from REST endpoints #3029

cgrinds opened this issue Jul 2, 2024 · 0 comments · Fixed by #3186
Assignees
@Hardikl
Labels
24.11 feature New feature or request status/testme

Comments

@cgrinds
Copy link
Collaborator

cgrinds commented Jul 2, 2024

With the recent change (#2740) to send the exact list of counters, there is the possibility that a template with a large number of counters could hit ONTAP's request URI limit.

When that happens, the Harvest REST collector should revert to the fields=* logic.

Here's a Python script that makes a request that is the maximum request URI length - 1, followed by a request that is too large. This script will cause the following error to be printed.

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>414 Request-URI Too Long</title>
</head><body>
<h1>Request-URI Too Long</h1>
<p>The requested URL's length exceeds the capacity
limit for this server.<br />
</p>
</body></html>

import base64
import http.client

host = "10.193.48.154"
user = "admin"
password = "password"

apache_max = 8 * 1024 # 8,192

conn = http.client.HTTPConnection(host)

# Create basic auth token
auth = user + ":" + password
auth = auth.encode("utf-8")
auth = base64.b64encode(auth).decode("utf-8")

headers = {
    "Authorization": f"Basic {auth}",
}

big = "a" * apache_max
url = "/api/cluster?fields=" + big

# The host + url can be at most apache_max characters
too_big = apache_max - len(host)
just_right = apache_max - len(host) - 1

# Send the just_right request.
# This will return a 400 since it is a bogus request, but that's OK
conn.request("GET", url[:just_right], headers=headers)
response = conn.getresponse()
data = response.read()

print("Status:", response.status)
print("Reason:", response.reason)
print("Data:", data.decode("utf-8"))

# Send the too_big request.
# This will fail with a 414
conn.request("GET", url[:too_big], headers=headers)
response = conn.getresponse()
data = response.read()

print("Status:", response.status)
print("Reason:", response.reason)
print("Data:", data.decode("utf-8"))

conn.close()
@cgrinds cgrinds added the feature New feature or request label Jul 2, 2024
@cgrinds cgrinds added the 24.11 label Aug 12, 2024
@Hardikl Hardikl self-assigned this Sep 30, 2024
@Hardikl Hardikl linked a pull request Oct 1, 2024 that will close this issue
feat: handle url limit in rest #3186
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Hardikl Hardikl

Labels
24.11 feature New feature or request status/testme
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat: handle url limit in rest NetApp/harvest
3 participants
@cgrinds @rahulguptajss @Hardikl