JUnit Extension and Java Agent for energy consumption measurement.
The power consumption of Java applications should become measurable, and thus visible, with the help of jPowerMonitor. This library includes an extension for measuring unit tests, as well as a Java agent for measuring any Java application. The Java agent collects the activity of the application to be measured at regular, configurable intervals. The agent takes into account the power consumption provided by the configured measurement tool. The CPU usage of the program and the current power consumption are aggregated to energy consumption per method over runtime and written into a CSV file. The result of the measurement is the energy consumption in watt-hours or joule.
- Install and configure Tool Libre Hardware Monitor: https://github.com/LibreHardwareMonitor/LibreHardwareMonitor
- Configure Libre Hardware Monitor to start a web server:
- If necessary, an alternative port (default port is 8085) can be also activated there.
- IMPORTANT: To start the web server, Libre Hardware Monitor may have to be started as administrator.
- After that the tool is also accessible in your browser: http://localhost:8085/.
- The jPowerMonitor JUnit extension and the Java Agent internally read the json document, which can be retrieved at http://localhost:8085/data.json.
- Configure Libre Hardware Monitor to start a web server:
- The tool HWiNFO or another tool that writes sensor values to a CSV file could be used alternatively, measurement -> method must be set to 'csv' and Logging to CSV in HWiNFO must be active: https://www.hwinfo.com/
- Configure HWiNFO to log the values of the power sensors to the CSV file.
- Start the logging in HWiNFO to a file e.g. in your project directory.
- If no measurement tool available or no measurement interfaces accessible (e.g. in virtualized cloud scenarios) configure measurement -> method to 'est' (@since jpowermonitor-1.2.0) and configure appropriate 'cpuMinWatts' / 'cpuMaxWatts' according to your platform (@see https://github.com/cloud-carbon-footprint/cloud-carbon-coefficients/tree/main/data).
- To start the Java agent, the "fat jar" (incl. dependencies, with name
jpowermonitor-<version>-all.jar) must be downloaded from mvn central here or must first be built with the Gradle taskshadowJar. - Copy
src/main/resources/jpowermonitor-template.yamlto the execution directory and rename it to./jpowermonitor.yaml. - Configure (at least) measurement -> lhm -> paths -> path to match your machine for using Libre Hardware Monitor, or the CSV section to use HWiNFO or another tool that writes sensor values to a CSV file or estimation 'est' (@since jpowermonitor:1.2.0).
- For using the JUnit Extension see below section JUnit Tests
- For more configuration settings see below section Configuration.
The configuration is done via a YAML file. Normally this would be jpowermonitor.yaml. The agent
has multiple ways of getting this configuration:
- You can pass it as a programm argument, see section above. If not, it will simply assume the name
and path is
jpowermonitor.yaml. - It first tries to read this path as an external file.
- If that fails, it tries to load this path from the JAR's resources
For the configuration of the JUnit extension a yaml file with the name of the executed test is searched.
| configuration path | description | optional | default value |
|---|---|---|---|
| initCycles | The number of cycles to initialize for measuring the base load on the system. (Parameter is only used in JUnitExtension, not in JavaAgent) |
X | 10 |
| samplingIntervalForInitInMs | Polling interval for initialization phase. (Parameter is only used in JUnitExtension, not in JavaAgent) |
X | 1000 |
| calmDownIntervalInMs | After initialization, the system waits until the test starts. (Parameter is only used in JUnitExtension, not in JavaAgent) |
X | 1000 |
| percentageOfSamplesAtBeginningToDiscard | What percentage of samples should be discarded at the beginning of the measurement to get more meaningful results. Meaningful: 5-20%. (Parameter is only used in JUnitExtension, not in JavaAgent) |
X | 15 |
| samplingIntervalInMs | Polling interval for test phase. (Parameter is only used in JUnitExtension, not in JavaAgent) |
X | 300 |
| carbonDioxideEmissionFactor | Conversion factor to calculate approximated CO2 consumption in grams from energy consumption per kWh. Depends on the energy mix of your location, for Germany compare e.g. https://www.umweltbundesamt.de/themen/klima-energie/energieversorgung/strom-waermeversorgung-in-zahlen#Strommix | X | 485 |
| measurement -> method | Specify which measurement method to use. Possible values: lhm, csv, est (@since jpowermonitor:1.2.0) | 'lhm' | |
| measurement -> csv | Measure the power using a CSV File Source. The source could be supplied from e.g. HWiNFO (see https://www.hwinfo.com) | X | |
| measurement -> csv -> inputFile | Path to csv file to read measure values from | 'hwinfo.csv' | |
| measurement -> csv -> lineToRead | Which line in the csv input file contains the current measured values? The first or the last? This depends on the measurement tool. Possible value: first, last | X | 'last' |
| measurement -> csv -> columns | Columns to read, index starts at 0. | ||
| measurement -> csv -> columns -> index | Index from a sensor, index starts at 0. | 95 | |
| measurement -> csv -> columns -> name | Name of a sensor | 'CPU Package Power [W]' | |
| measurement -> csv -> columns -> energyInIdleMode | For the current measuring sensors the base load per sensor path can be configured (self-measured). If nothing is specified, then a base load measurement is performed in @BeforeAll (see also initCycles and samplingIntervalForInitInMs) and this value is used. |
X | |
| measurement -> csv -> encoding | Encoding to use for reading the csv input file | X | 'UTF-8' |
| measurement -> csv -> delimiter | Delimiter to use for separating the columns in the csv input file | X | ',' |
| measurement -> lhm | Measure the Power using Libre Hardware Monitor (see https://github.com/LibreHardwareMonitor/LibreHardwareMonitor) Libre Hardware monitor must be started in administrator mode | X | |
| measurement -> lhm -> url | Url of the Libre Hardware Monitor incl. port | ||
| measurement -> lhm -> paths | Multiple paths to the sensors can be specified. This depends on the machine and must be viewed in the Libre Hardware Monitor. | ||
| measurement -> lhm -> paths -> path | Path to a sensor | ||
| measurement -> lhm -> paths -> energyInIdleMode | For the current measuring sensors the base load per sensor path can be configured (self-measured). If nothing is specified, then a base load measurement is performed in @BeforeAll (see also initCycles and samplingIntervalForInitInMs) and this value is used. |
X | |
| measurement -> est | Use estimated values according to Etsy's “Cloud Jewels” method (https://www.etsy.com/codeascraft/cloud-jewels-estimating-kwh-in-the-cloud) to estimate energy coefficients (kWh) for cloud service compute, storage, networking, and memory usage. See https://www.cloudcarbonfootprint.org/docs/methodology/#energy-estimate-watt-hours. Find the cloud-platform values here https://github.com/cloud-carbon-footprint/cloud-carbon-coefficients/tree/main/data or here https://github.com/re-cinq/emissions-data/tree/main/data/v2 @since jpowermonitor:1.2.0 | ||
| measurement -> est -> cpuMinWatts | Minimum CPU watts (idle). | 0.74 (AWS EC2 avg.) | |
| measurement -> est -> cpuMaxWatts | Maximum CPU watts (100% load). | 3.5 (AWS EC2 avg.) | |
| csvRecording -> resultCsv | Only affects the JUnit extension: Result CSV Name (specify paths with slash, they will be created automatically) | X | |
| csvRecording -> measurementCsv | Only affects the JUnit extension: Measurement CSV Name (specify paths with slash, they will be created automatically) | X | |
| javaAgentCfg -> packageFilter | Filter power and energy for methods starting with this packageFilter names, write results of filtered methods to separate CSV files. | X | 'group.msg', 'com.msgforbanking' |
| javaAgentCfg -> measurementIntervalInMs | Energy measurement interval in milliseconds for the Java Agent. This is the interval the data source for the sensor values is questioned for new values. | X | 1000 |
| javaAgentCfg -> gatherStatisticsIntervalInMs | Gather statistics interval in milliseconds. This is the interval the stacktrace of each active thread is questioned for active methods. Should be smaller than measurementIntervalInMs. |
X | 10 |
| javaAgentCfg -> writeEnergyMeasurementsToCsvIntervalInS | Write energy measurement results to CSV files interval in seconds. Leave empty to write energy measurement results only at program exit (be sure your application to measure exits "gracefully", thus by calling System.exit(..), else results might be lost!). | X | 30 |
| javaAgentCfg -> monitoring | Section for configuration of monitoring interfaces. @since jpowermonitor:1.2.0 | ||
| javaAgentCfg -> monitoring -> prometheus | Section for configuration of Prometheus monitoring interface. @since jpowermonitor:1.2.0 | ||
| javaAgentCfg -> monitoring -> prometheus -> enabled | Set to true, if prometheus monitoring should be enabled. This will start a HttpServer on the configured port. | X | false |
| javaAgentCfg -> monitoring -> prometheus -> httpPort | Prometheus Http Server Port. Prometheus will ask the jPowerMonitored application for metrics on this port. | X | 1234 |
| javaAgentCfg -> monitoring -> prometheus -> writeEnergyIntervalInS | Write energy measurement results to Prometheus interval in seconds. | X | 30 |
| javaAgentCfg -> monitoring -> prometheus -> publishJvmMetrics | Should the default JVM Metric be published to Prometheus? This includes information about GC, memory etc. | X | false |
If no base load (energyInIdleMode) is specified for a path, this is measured before each test. So a mixed operation between configuration of the base load and measurement is also possible and the results can be compared (some sensors provide very similar values).
For non-current measuring sensors (e.g. temperature) the base load is not calculated extra and also not subtracted from the measured value! It is only output if a base load must also be calculated for a current-measuring sensor because this is not specified in the configuration.
- You may build the jpowermonitor fat jar using the build target
shadowJarand the copy the-all.jarinto a folder of your project. - You can use this jar for the java agent. Alternatively you may download the fat jar from mvn central here.
- For testing call with
java -javaagent:.\build\libs\jpowermonitor-<version>-all.jar[=path-to-jpowermonitor.yaml] -jar .\build\libs\jpowermonitor-demo-<version>.jar [runtimeSeconds] [cpuThreads] - The
.\build\libs\jpowermonitor-demo-<version>.jaris just an example and can be replaced by anyjarof your choice - The configuration can be passed as a parameter using the
=sign! If nothing is passed, the default configuration filejpowermonitor.yamlis searched. - For starting the agent with Spring Boot, Servlet-Container etc. please consult the respective documentation for adding a java agent.
- e.g. for use with gradle and Spring Boot you may use
in your gradle script.bootRun { jvmArgs += ["-javaagent:./lib/jpowermonitor-1.2.0-all.jar=./lib/jpowermonitor.yaml"] }
The add the test dependency to your gradle build (analogue for maven builds):
testImplementation(
[group: 'io.github.msg-systems', name: 'jpowermonitor', version: jPowerMonitorVersion],
)
The extension is designed for JUnit 5 (jupiter) tests and is included into your test class as follows:
@ExtendWith({JPowerMonitorExtension.class})
jPowerMonitor then searches for a configuration file with the name of the executed test + .yaml extension.
For example if your test is named MyAlgorithmText.java then the configuration for jPowerMonitor is expected to be found in MyAlgorithmText.yaml somewhere in the classpath.
The tests are best executed as @RepeatedTests(...) in order to obtain several measurements. For example always 10 times.
In the result CSV (configuration: resultCsv) you can then calculate an average of the results in Excel.
@SensorValues annotated fields in the test class of the type List<SensorValue> can be retrieved after each test in the @AfterEach method.
A SensorValue contains the resulting measured value with name of the sensor and the following additional information:
Unit unit: The unit of the value (e.g. W, J, Wh)BigDecimal powerInIdleMode: The power consumption that has been measured in idle mode.LocalDateTime executionTime: The execution time of measure.long durationOfTestInNanoSeconds: The duration of the whole test.BigDecimal valueWithoutIdlePowerPerHour: The value without the idle power value.BigDecimal valueWithIdlePowerPerHour: The value together with the idle power value.
The sensor values could be used to create your own output file in your own format.
The file for the measuring points can be configured under csvRecording.measurementCsv. jPowerMonitor outputs all considered measurement points (also those of the base load measurement, if it takes place) into this file.
Please note: the first percentageOfSamplesAtBeginningToDiscard% measurement points are always discarded.
The result file can be configured under csvRecording.resultCsv. The summary of energy consumption is written to this file.
The headers of the csv output file are configured in the csvExport.properties (default language English).
The file currently is translated into German and French and is locale dependent.
You may add your own locale and add the translated file to the classpath. You may set the language via JVM option e.g. -Duser.language=es.
Find information about connecting grafana and prometheus e.g. here: https://prometheus.io/docs/visualization/grafana
{__name__=~"jPowerMonitor_.*"}
Show average power per method as multiple threads may execute same method and usually one wants to see the average:
topk(5, sort_desc(avg by(method) (jPowerMonitor_power_per_method_filtered{job=~"jPowerMonitor"})))
topk(5, sort_desc(sum by(method) (jPowerMonitor_energy_per_method_filtered{job=~"jPowerMonitor"})))
topk(5, sort_desc(sum by(method) (jPowerMonitor_co2_per_method_filtered{job=~"jPowerMonitor"})))
You may specify the following additional attributes:
- pid - get only data for one process id:
topk(5, sort_desc(sum by(method) (jPowerMonitor_energy_per_method_filtered{pid="27712", job=~"jPowerMonitor"}))) - thread - get only data for one thread:
topk(5, sort_desc(sum by(method) (jPowerMonitor_energy_per_method_filtered{thread="main", job=~"jPowerMonitor"})))
- If built-in benchmark is run with all available CPU-threads (default) parallel measurement result will be too low since PowerMeasurement-Thread does not get enough time in 100% CPU load scenario -> Try with 2-4 threads less for proper results.
This markdown can be converted to html with
pandoc --self-contained -t slidy -c docs/slidy.css -o Readme.html README.md to html and from there to pdf via print function of the browser.
Copyright © 2022-2024 msg for banking ag
Licensed under Apache License 2.0