-
Notifications
You must be signed in to change notification settings - Fork 480
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Bug]: Importing OTLP "File Exporter" JSON fails #2225
Comments
Two issues with your data:
{
"resourceSpans": ...
} You have multiple objects (one per line) which do not have an equivalent representation in OTLP data types (it would be an array of
|
The implementation expects it to be a single JSON object, but the specification linked to by the origin feature request (#4949) states that the file is supposed to be in "JSON lines" format.
When I use the current file-exporter (https://github.com/open-telemetry/opentelemetry-collector-contrib/releases/tag/v0.96.0), it generates a file in "JSON lines" format.
In order to avoid this as a distraction, I'm including a trace that I've generated using opentelemetry's js SDK. The opentelemetry collector contrib configuration: receivers:
otlp:
protocols:
http:
endpoint: 0.0.0.0:4318
processors:
batch:
# The following ensures that one span is written per line.
send_batch_size: 1
send_batch_max_size: 1
exporters:
file:
path: /output/otel.json
service:
pipelines:
traces:
receivers: [otlp]
processors: [batch]
exporters: [file] The generated trace otel.json. Note that there are 4 liness, one JSON object per line. I can use the following jq to process the file into a single JSON object. The following concatenates all of the
Which gives me otel-out.json. My expectation is that if Jaeger says that it will import OpenTelemetry JSON generated by the opentelemetry collector's file-export, then it should be able to handle a multi-line JSON file. |
A initial proposal: what if we get the frontend to add a |
@yurishkuro Any guidance on fixing it will be more helpful |
I will move to UI |
The current code that checks for OTLP format is here:
It effectively checks for JSON like
|
|
I need this too. For now, I'm using a script like this that allows me to send all traces from a local file to Jagear. import requests
import json
import argparse
from pathlib import Path
def send_post_request(url, json_data):
headers = {'Content-Type': 'application/json'}
response = requests.post(url, data=json.dumps(json_data), headers=headers)
response.raise_for_status()
return response
def process_file(input_file: Path, server_url: str):
lines = input_file.read_text().splitlines()
api_url = server_url.rstrip("/") + "/v1/traces"
for line in lines:
json_data = json.loads(line.strip())
send_post_request(api_url, json_data)
print("Finished")
def main():
parser = argparse.ArgumentParser(description="Send saves traces to OLTP collector")
parser.add_argument(
'--server-url',
type=str,
default='http://localhost:4318',
help="The server URL to send the POST requests to."
)
parser.add_argument('input_file', type=Path, help="The JSONL file to be processed.")
args = parser.parse_args()
process_file(args.input_file, args.server_url)
if __name__ == "__main__":
main() |
@sfc-gh-kbregula if you're proficient in JS consider if you can bring #2254 over the finish line. |
@yurishkuro I will take this up. Lately I have searching this issue in Jaeger. But moved here 😅 |
OpenTelemetry specifies that JSON line files can contain valid JSON objects which are seperated with a newline character. The existing readJsonFile function couldn't handle those instances, this PR fixes that, along with test cases to verify. Signed-off-by: Muthukumar <muthuku37@gmail.com>
OpenTelemetry specifies that JSON line files can contain valid JSON objects which are seperated with a newline character. The existing readJsonFile function couldn't handle those instances, this PR fixes that, along with test cases to verify. Signed-off-by: Muthukumar <muthuku37@gmail.com>
Signed-off-by: Muthukumar <muthuku37@gmail.com>
What happened?
jaegertracing/all-in-one 1.55 fails to import JSON generated by OTEL File Exporter.
Steps to reproduce
test.json
test.json
onto Jaeger'sClick or drag files to this area.
Expected behavior
I expect Jaeger to have loaded successfully loaded the Open Telemetry traces, per jaegertracing/jaeger#4949
Relevant log output
No response
Screenshot
Additional context
The existing code only expects there to be a single JSON object entry in the OTEL File Exporter JSON output, but the file exporter specification says that the is in JSON lines format (file contains multiple JSON serialized objects, with a new-line separating each).
This is evident when looking at the test data in the original PR, as it is only a single JSON object entry.
https://github.com/jaegertracing/jaeger/pull/5155/files
Jaeger backend version
1.55
SDK
No response
Pipeline
Example of the opentelemetry collector configuration I use to capture OTEL traces to OTEL file-exporter format:
Stogage backend
No response
Operating system
No response
Deployment model
No response
Deployment configs
No response
The text was updated successfully, but these errors were encountered: