Details | Benchmarking with Graal

To benchmark run the miniboxing benchmarks Graal, you will need to perform 5 steps:

• Install miniboxing on your machine
• Update the Graal and benchmarks to handle miniboxing
• Run the benchmarks

The last two steps will be presented in detail.

Installing Miniboxing on Your Machine

To install miniboxing on your machine, see the instructions in the installing page.

Updating the Graal Instalation and the Benchmarks

To install Graal, first follow the instructions on its wiki. Once Graal is installed, you will need simulate the java command:

$ cd graal
$ mkdir bin
$ cd bin
$ echo '#!/bin/bash' >> java
$ echo 'cd '`pwd`'/../' >> java
$ echo './mx.sh vm $@' >> java
$ chmod u+x java
$ cd ../..

Okay, almost there. Now we need to set the path to Graal in the miniboxing plugin:

$ cd miniboxing-plugin
$ nano tests/benchmarks/src/miniboxing/benchmarks/launch/BenchmarkingTest.scala

There, add your machine's Graal parent directory to baseDir:

  lazy val baseDir =
    java.net.InetAddress.getLocalHost().getHostName() match {
      case "sun-laptop"   => "/home/sun/workspace/dev/"
      case "lamppc47"     => "/localhome/ureche/software/"
      case <your machine> => <path to graal>
      case _              => "./" // uh-oh, I don't know
    }

And now comment the lines following HotSpot, uncomment the lines following Graal. You may also need to tune the flags passed to Graal. The code should look like:

  // the command used to start the JVM
  // HotSpot:
  //  lazy val javaCommand = "java -server"
  //  lazy val javaPreJDK7 = false
  // JRockit:
  //  lazy val javaCommand = baseDir + "jrockit/bin/java -jrockit -d64 -Xms4g -Xmx4g -Xss4m"
  //  lazy val javaPreJDK7 = true
  // Graal:
  lazy val javaCommand = baseDir + "graal/bin/java -graal -dsa -Xmx8g -Xms8g -Xss10m -d64 -XX:+BootstrapGraal"
  lazy val javaPreJDK7 = true

This should do it.

Running the Graal Benchmarks

To run the benchmarks do:

$ sbt miniboxing-benchmarks/run
Detected sbt version 0.12.2
[info] Loading project definition from /home/mbtest/miniboxing-plugin/project
[info] Set current project to miniboxing (in build file:/home/mbtest/miniboxing-plugin/)
[warn] Potentially incompatible versions of dependencies of {file:/home/mbtest/miniboxing-plugin/}miniboxing-benchmarks:
[warn]    org.scala-lang: 2.10.3-SNAPSHOT, 2.10.1, 2.10.0
[info] Compiling 1 Scala source to /home/mbtest/miniboxing-plugin/tests/benchmarks/target/scala-2.10/classes...
[info] Running miniboxing.benchmarks.launch.BenchmarkingTest 
Bootstrapping Graal................................ in 61972 ms
Bootstrapping Graal................................ in 75502 ms
                     ideal.array.insert :  Parameters(size -> 1000000):   10.08445  Parameters(size -> 2000000):   12.55962  Parameters(size -> 3000000):   23.48103  
...

Don't worry about the time Graal takes to bootstrap, that's expected and does not count towards the benchmark times. The labels and details about the benchmarking are described in the benchmarks page.

You can make the Graal compiler crash by enabling a single line:

$ cd miniboxing-plugin
$ nano tests/benchmarks/src/miniboxing/benchmarks/launch/tests/HardcodedMiniboxingSimpleTestFS.scala
uncomment line 218:
          // Doing this will crash Graal with the following stack trace:
          array_find(array_reverse(array_insert()))
          //  scope: Compiling.GraalCompiler.FrontEnd.HighTier.Lowering (BEFORE_GUARDS).IncrementalCanonicalizer.Lowering iteration 0.Schedule.InterceptException
          //  Exception occurred in scope: Compiling.GraalCompiler.FrontEnd.HighTier.Lowering (BEFORE_GUARDS).IncrementalCanonicalizer.Lowering iteration 0.Schedule.InterceptException

The stack trace should be:

          //  scope: Compiling.GraalCompiler.FrontEnd.HighTier.Lowering (BEFORE_GUARDS).IncrementalCanonicalizer.Lowering iteration 0.Schedule.InterceptException
          //  Exception occurred in scope: Compiling.GraalCompiler.FrontEnd.HighTier.Lowering (BEFORE_GUARDS).IncrementalCanonicalizer.Lowering iteration 0.Schedule.InterceptException
          //  Context obj java.lang.NullPointerException
          //  Context obj com.oracle.graal.phases.schedule.SchedulePhase@18f01d41
          //  Context obj com.oracle.graal.phases.common.LoweringPhase$Round@1f64f1b4
          //  Context obj com.oracle.graal.phases.common.IncrementalCanonicalizerPhase@faa7860
          //  Context obj com.oracle.graal.phases.common.LoweringPhase@b0ba210
          //  Context obj com.oracle.graal.compiler.phases.HighTier@1309bc25
          //  Context obj StructuredGraph:36901{HotSpotMethod<MBResizableArray_class_J.contains_J(long, byte)>}
          //  Use -G:+DumpOnError to enable dumping of graphs on this error
          //  Context obj com.oracle.graal.hotspot.amd64.AMD64HotSpotRuntime@2bd816a1
          //  Context obj DebugDumpScope[3387]
          //  java.lang.NullPointerException
          //    at com.oracle.graal.nodes.cfg.ControlFlowGraph.commonDominator(ControlFlowGraph.java:338)
          //    at com.oracle.graal.phases.schedule.SchedulePhase$CommonDominatorBlockClosure.apply(SchedulePhase.java:658)
          //    at com.oracle.graal.phases.schedule.SchedulePhase.blocksForUsage(SchedulePhase.java:781)
          //    at com.oracle.graal.phases.schedule.SchedulePhase.latestBlock(SchedulePhase.java:628)
          //    at com.oracle.graal.phases.schedule.SchedulePhase.assignBlockToNode(SchedulePhase.java:477)
          //    at com.oracle.graal.phases.schedule.SchedulePhase.assignBlockToNodes(SchedulePhase.java:445)
          //    at com.oracle.graal.phases.schedule.SchedulePhase.run(SchedulePhase.java:345)

Btw, the entire benchmark should take around 8 hours.