perfGenie

perfGenie is a continuous low overhead contextual profiling solution that can parse and visualize Java flight recorder(JFR) format profile and Jstacks. JFR profiles can be created using Java machine control(JMC), Java flight recorder, Async-profiler, or jcmd. It provides the ability to view profiles in context trees, samples explorer, flame Graph, thread state, river, and hotspot surface views. It helps to compare two profiles using context tree diff or flame graph diff views. It also provides functionality to filter profiles for a given custom event context, tid or thread name, etc. The request timeline view helps look at samples of a particular request context. It also can visualize thread context requests and context metric timeline views. The aggregation feature allows for combining profiles for a more extended period. Thread dumps can also be converted into supported profile views.

Continuous low overhead contextual profiling ✨

perfGenie can be deployed as a continuous profiling solution. It uses Cantor (https://github.com/salesforce/cantor) as a data layer. Cantor can be configured to store data in H2, MySQL, or S3. This project is set up with a simple cron job to monitor a directory for any JFR files, parse and store in H2 (default configuration).

Development

Create a new Fork and Clone the repository

$ git clone git@github.com:salesforce-misc/perfGenie.git

Build

$ export JAVA_HOME=<jdk home path>
$ mvn clean install

Config

add profiles and custom events to be parsed in config.properties file

ex:
customevents=LogContext;MqFrm;CPUEvent;MemoryEvent;Search
profiles=ExecutionS;Socket;NewTLAB;OutsideTLAB
jfrdir=/tmp/jfrs
tenant=dev

#h2,grpc,mySQL
storageType=h2

#h2 config
h2dir=/tmp/h2.db

#mySQL config
mySQL.host=xxxx
mySQL.port=3306
mySQL.user=xxxx
mySQL.pwd=xxxx

#grpc, cantor server
grpc.target=localhost:7443

#filtering
threshold=0.05
filterDepth=4
maxStackDepth=10

#experimental to enable river view and surface views
isExperimental=false

Start application server

$ java -jar perfgenie/target/perfgenie.jar

Access URL http://localhost:8080

How to add JFR's for testing

The application server has a cron job to monitor and parse JFR (*.jfr or .jfr.gz), and Jstack (.jstack) files available at /tmp/jfrs/

Note: example files provided in the examples directory can be copied into jfrdir for testing

Note: refer simulator module to generate a sample jfr with custom events added

Agent mode

$ java -jar agent/target/agent.jar

Agent to monitor and parse JFR (*.jfr or .jfr.gz), and Jstack (.jstack) files available at /tmp/agent/jfrs/

Features

• Tree view (backtrace, call tree, and compare view)
• Samples explorer view (grouping samples by tid, thread name, and custom event context dimensions)
• Flame graph view (backtrace, call tree, and compare view)
• Thread state view (This view is supported for thread dumps)
• River view (Stack trace timeline view)
• Hotspot surface (3D Stacktrace timeline view)
• Custom event table view
• Custom event context filters on the profile
• Thread request view (based on custom events)
• Metric timeline view (based on custom event metrics)
• Request timeline view (to see samples of particular request)

User interface:

Context Filters

Context table view

Request samples view

Thread request view

Metric timeline view

Profile views

Calling context tree

Calling context tree compare view

Samples explorer

Flame graph (Brendan Gregg, https://www.brendangregg.com/flamegraphs.html)

Flame graph compare view

Thread state view

River view (Experimental stack trace timeline view)

Hotspot surface view (Experimental stack trace timeline view)