traceviewer: how to handle large traces?

[milianw]

Hello!

I'm trying to convert LTTng trace points to the Chrome Trace Format and then load the data into chrome. The LTTng trace data is roughly ~100MB in its binary CTF (common trace format) representation. The JSON version of that easily takes four times as much and doesn't yet contain all information - mostly just the events names and timestamps without any additional optional metadata. That already is impossible to load, the tab gets killed once it reaches ~3GB of memory or so.

I saw that you seem to be using catapult for Android ftrace data, which should easily generate as much data as I'm seeing here with LTTng. How do you handle this?

[milianw]

a ~160MB binary LTTng trace gets converted to ~2.68GB of JSON text which then also completely fails to load:

While importing:
RangeError: Invalid typed array length: 2682325620
    at new Uint8Array (<anonymous>)
    at Object.flattenChunks (chrome://tracing/tracing.js:4696:6572)
    at i.onEnd (chrome://tracing/tracing.js:4696:5526)
    at i.push (chrome://tracing/tracing.js:4696:5263)
    at n (chrome://tracing/tracing.js:4696:3781)
    at Object.a.inflateRaw (chrome://tracing/tracing.js:4696:5679)
    at Function.GzipImporter.inflateGzipData_ (chrome://tracing/tracing.js:4710:137)
    at GzipImporter.extractSubtraces (chrome://tracing/tracing.js:4719:240)
    at chrome://tracing/tracing.js:1323:66
    at Task.run (chrome://tracing/tracing.js:2313:95)

could one use something more efficient than JSON for the trace format here? best would be something that's streamable and doesn't have a tree structure like JSON