Skip to content

Commit

Permalink
Squashed commit of the following:
Browse files Browse the repository at this point in the history 
commit 96fd9be
Merge: 6c9f313 52e42eb
Author: Arne Tarara <arne@datafuse.de>
Date:   Sat Aug 27 17:32:24 2022 +0200

    Merge branch 'dev' of github.com:green-coding-berlin/green-metrics-tool into dev

    * 'dev' of github.com:green-coding-berlin/green-metrics-tool:
      AC measurement entry fixed and deactived by default

commit 6c9f313
Author: Arne Tarara <arne@datafuse.de>
Date:   Sat Aug 27 17:32:06 2022 +0200

    GUDE Readme updated

commit 52e42eb
Author: Arne Tarara <arne@datafuse.de>
Date:   Fri Aug 26 17:46:45 2022 +0200

    AC measurement entry fixed and deactived by default

commit 856301d
Author: Arne Tarara <arne@datafuse.de>
Date:   Fri Aug 19 17:33:28 2022 +0200

    Squashed commit of the following:

    commit c0e4e8924b12f4a9923041b92eeae18f32fc17b8
    Author: Arne Tarara <arne@datafuse.de>
    Date:   Fri Aug 19 17:32:01 2022 +0200

        AC measurement with GUDE added

    commit 0372acd
    Author: Arne Tarara <arne@datafuse.de>
    Date:   Wed Aug 17 11:01:41 2022 +0200

        Version 0.5 release - Squashed commit of the following:

        commit b94d1cb
        Author: Arne Tarara <arne@datafuse.de>
        Date:   Tue Aug 16 16:47:45 2022 +0200

            Differential moved to AVG. and disabled unused channels for now

        commit 5fcb537
        Author: Arne Tarara <arne@datafuse.de>
        Date:   Tue Aug 16 14:57:09 2022 +0200

            Introduced measurement time as s in overview

        commit c4916d1
        Author: Arne Tarara <arne@datafuse.de>
        Date:   Tue Aug 16 10:02:57 2022 +0200

            setuid bit now used, so we can execute still with root rights but without calling sudo. chown for root in make. removed sudoers file usage; small fixes to printf error calls which were not on Stderr

        commit 16ef1de
        Author: Arne Tarara <arne@datafuse.de>
        Date:   Mon Aug 15 19:43:09 2022 +0200

            Added cpu_utilization_procfs_system_provider

        commit d44950f
        Author: Arne Tarara <arne@datafuse.de>
        Date:   Mon Aug 15 19:41:08 2022 +0200

            Renamed cpu_time_system provider to cpu_time_procfs_system

        commit 2533083
        Merge: 4676f71 805016e
        Author: Arne Tarara <arne@datafuse.de>
        Date:   Mon Aug 15 15:40:23 2022 +0200

            Merge branch 'main' into dev

            * main:
              Adding missing time and atx metrics to lookup
              Release 1.0: Squashed commit of the following:

        commit 4676f71
        Author: dan-mm <33732895+dan-mm@users.noreply.github.com>
        Date:   Mon Aug 15 15:26:51 2022 +0200

            Bugfixes rapl c improvements (#19)

            * small refactor of code; use enum for measurement modes; moved all setup call outside of main measurement loop

            * get userid dynamically; makefile now chmod +s on static-binary

            * changed enum to #define for measurement mode

            * missed a variable during refactor

            * implemented user_id fix to the rest of the metric providers which use cgroups

            Co-authored-by: Dan M <dan@green-coding.org>

        commit 79bb522
        Author: Arne Tarara <arne@datafuse.de>
        Date:   Sat Aug 13 14:37:22 2022 +0200

            Metrics have now dimension added in their chart displays

        commit ab2dc17
        Author: Arne Tarara <arne@datafuse.de>
        Date:   Sat Aug 13 14:03:36 2022 +0200

            RAM reporter renamed to Memory

        commit 66c6830
        Author: Arne Tarara <arne@datafuse.de>
        Date:   Sat Aug 13 13:55:40 2022 +0200

            Explanations added for network formulas and accounting

        commit ae2e615
        Author: Arne Tarara <arne@datafuse.de>
        Date:   Sat Aug 13 13:55:23 2022 +0200

            Time reporters inactive by default

        commit f590ec4
        Author: Arne Tarara <arne@datafuse.de>
        Date:   Sat Aug 13 12:37:00 2022 +0200

            Network IO was not correctly added up, but only last container was accounted for

        commit 6a30388
        Author: Arne Tarara <arne@datafuse.de>
        Date:   Sat Aug 13 11:08:18 2022 +0200

            CLI is now very pretty

        commit f483dd9
        Author: Arne Tarara <arne@datafuse.de>
        Date:   Fri Aug 12 20:23:57 2022 +0200

            Added all channels to importer

        commit ab486a4
        Author: Arne Tarara <arne@datafuse.de>
        Date:   Fri Aug 12 13:35:08 2022 +0200

            DC converter now operates with mV and new measurement resistor of 0.005 W

        commit e455ce1
        Author: Arne Tarara <arne@datafuse.de>
        Date:   Fri Aug 12 12:47:38 2022 +0200

            Added Watt power reading and renamed atx channel with DC added

        commit ed9752e
        Author: Arne Tarara <arne@datafuse.de>
        Date:   Fri Aug 12 12:31:51 2022 +0200

            Moved dc_converter to mV

        commit 448ea1e
        Author: Arne Tarara <arne@datafuse.de>
        Date:   Fri Aug 12 12:30:09 2022 +0200

            Clarifying comment

        commit e25c273
        Author: Arne Tarara <arne@datafuse.de>
        Date:   Sun Aug 7 22:42:44 2022 +0200

            Removed obsolte TODOs and comments

        commit a895f73
        Author: Arne Tarara <arne@datafuse.de>
        Date:   Sun Aug 7 22:41:22 2022 +0200

            Clarifying comment

        commit a58c3eb
        Author: Arne Tarara <arne@datafuse.de>
        Date:   Sun Aug 7 15:12:47 2022 +0200

            Introduced typo

        commit 4100461
        Author: Arne Tarara <arne@datafuse.de>
        Date:   Sun Aug 7 15:10:47 2022 +0200

            also renamed field to MB

        commit 65a242c
        Author: Arne Tarara <arne@datafuse.de>
        Date:   Sun Aug 7 15:09:51 2022 +0200

            Network IO moved to MB

        commit f7a5068
        Author: Arne Tarara <arne@datafuse.de>
        Date:   Sun Aug 7 15:09:01 2022 +0200

            ATX is now using channel as identifier

        commit 573d199
        Author: Arne Tarara <arne@datafuse.de>
        Date:   Sun Aug 7 14:48:33 2022 +0200

            DC measurement converter and display added

commit b94d1cb
Author: Arne Tarara <arne@datafuse.de>
Date:   Tue Aug 16 16:47:45 2022 +0200

    Differential moved to AVG. and disabled unused channels for now

commit 5fcb537
Author: Arne Tarara <arne@datafuse.de>
Date:   Tue Aug 16 14:57:09 2022 +0200

    Introduced measurement time as s in overview

commit c4916d1
Author: Arne Tarara <arne@datafuse.de>
Date:   Tue Aug 16 10:02:57 2022 +0200

    setuid bit now used, so we can execute still with root rights but without calling sudo. chown for root in make. removed sudoers file usage; small fixes to printf error calls which were not on Stderr

commit 16ef1de
Author: Arne Tarara <arne@datafuse.de>
Date:   Mon Aug 15 19:43:09 2022 +0200

    Added cpu_utilization_procfs_system_provider

commit d44950f
Author: Arne Tarara <arne@datafuse.de>
Date:   Mon Aug 15 19:41:08 2022 +0200

    Renamed cpu_time_system provider to cpu_time_procfs_system

commit 2533083
Merge: 4676f71 805016e
Author: Arne Tarara <arne@datafuse.de>
Date:   Mon Aug 15 15:40:23 2022 +0200

    Merge branch 'main' into dev

    * main:
      Adding missing time and atx metrics to lookup
      Release 1.0: Squashed commit of the following:

commit 4676f71
Author: dan-mm <33732895+dan-mm@users.noreply.github.com>
Date:   Mon Aug 15 15:26:51 2022 +0200

    Bugfixes rapl c improvements (#19)

    * small refactor of code; use enum for measurement modes; moved all setup call outside of main measurement loop

    * get userid dynamically; makefile now chmod +s on static-binary

    * changed enum to #define for measurement mode

    * missed a variable during refactor

    * implemented user_id fix to the rest of the metric providers which use cgroups

    Co-authored-by: Dan M <dan@green-coding.org>

commit 79bb522
Author: Arne Tarara <arne@datafuse.de>
Date:   Sat Aug 13 14:37:22 2022 +0200

    Metrics have now dimension added in their chart displays

commit ab2dc17
Author: Arne Tarara <arne@datafuse.de>
Date:   Sat Aug 13 14:03:36 2022 +0200

    RAM reporter renamed to Memory

commit 66c6830
Author: Arne Tarara <arne@datafuse.de>
Date:   Sat Aug 13 13:55:40 2022 +0200

    Explanations added for network formulas and accounting

commit ae2e615
Author: Arne Tarara <arne@datafuse.de>
Date:   Sat Aug 13 13:55:23 2022 +0200

    Time reporters inactive by default

commit f590ec4
Author: Arne Tarara <arne@datafuse.de>
Date:   Sat Aug 13 12:37:00 2022 +0200

    Network IO was not correctly added up, but only last container was accounted for

commit 6a30388
Author: Arne Tarara <arne@datafuse.de>
Date:   Sat Aug 13 11:08:18 2022 +0200

    CLI is now very pretty

commit f483dd9
Author: Arne Tarara <arne@datafuse.de>
Date:   Fri Aug 12 20:23:57 2022 +0200

    Added all channels to importer

commit ab486a4
Author: Arne Tarara <arne@datafuse.de>
Date:   Fri Aug 12 13:35:08 2022 +0200

    DC converter now operates with mV and new measurement resistor of 0.005 W

commit e455ce1
Author: Arne Tarara <arne@datafuse.de>
Date:   Fri Aug 12 12:47:38 2022 +0200

    Added Watt power reading and renamed atx channel with DC added

commit ed9752e
Author: Arne Tarara <arne@datafuse.de>
Date:   Fri Aug 12 12:31:51 2022 +0200

    Moved dc_converter to mV

commit 448ea1e
Author: Arne Tarara <arne@datafuse.de>
Date:   Fri Aug 12 12:30:09 2022 +0200

    Clarifying comment

commit e25c273
Author: Arne Tarara <arne@datafuse.de>
Date:   Sun Aug 7 22:42:44 2022 +0200

    Removed obsolte TODOs and comments

commit a895f73
Author: Arne Tarara <arne@datafuse.de>
Date:   Sun Aug 7 22:41:22 2022 +0200

    Clarifying comment

commit a58c3eb
Author: Arne Tarara <arne@datafuse.de>
Date:   Sun Aug 7 15:12:47 2022 +0200

    Introduced typo

commit 4100461
Author: Arne Tarara <arne@datafuse.de>
Date:   Sun Aug 7 15:10:47 2022 +0200

    also renamed field to MB

commit 65a242c
Author: Arne Tarara <arne@datafuse.de>
Date:   Sun Aug 7 15:09:51 2022 +0200

    Network IO moved to MB

commit f7a5068
Author: Arne Tarara <arne@datafuse.de>
Date:   Sun Aug 7 15:09:01 2022 +0200

    ATX is now using channel as identifier

commit 573d199
Author: Arne Tarara <arne@datafuse.de>
Date:   Sun Aug 7 14:48:33 2022 +0200

    DC measurement converter and display added
  • Loading branch information
@ArneTR
ArneTR committed Sep 30, 2022
1 parent 0372acd commit 77870b6
Show file tree
Hide file tree
Showing 6 changed files with 147 additions and 1 deletion.
1 change: 1 addition & 0 deletions config.yml.example
View file
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ measurement:
memory.total.cgroup.container.provider.MemoryTotalCgroupContainerProvider: 100
memory.energy.RAPL.MSR.system.provider.MemoryEnergyRaplMsrSystemProvider: 100
network.io.cgroup.container.provider.NetworkIoCgroupContainerProvider: 100
# psu.energy.ac.system.provider.PsuEnergyAcSystemProvider: 100

admin:
# This address will get an email, when a new project was added through the frontend
Expand Down
19 changes: 18 additions & 1 deletion frontend/js/stats.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,11 @@ const metrics_info = {
SI_conversion_factor: 1000,
unit_after_conversion: 'J'
},
psu_energy_ac_system: {
unit: 'mJ',
SI_conversion_factor: 1000,
unit_after_conversion: 'J'
},
memory_energy_rapl_msr_system: {
unit: 'mJ',
SI_conversion_factor: 1000,
Expand Down Expand Up @@ -153,7 +158,7 @@ const fillProjectTab = (selector, data) => {
}

const getMetrics = (stats_data, style='apex') => {
const metrics = {cpu_utilization_containers: [], cpu_utilization_system: [], mem_total: [], network_io: {}, series: {}, atx_energy: 0, cpu_energy: 0, memory_energy: 0}
const metrics = {cpu_utilization_containers: [], cpu_utilization_system: [], mem_total: [], network_io: {}, series: {}, atx_energy: 0, psu_ac_energy: 0, cpu_energy: 0, memory_energy: 0}

let accumulate = 0;

Expand Down Expand Up @@ -184,6 +189,8 @@ const getMetrics = (stats_data, style='apex') => {
if (accumulate === 1) metrics.cpu_energy += value;
} else if (metric_name == 'atx_energy_dc_channel') {
if (accumulate === 1) metrics.atx_energy += value;
} else if (metric_name == 'psu_energy_ac_system') {
if (accumulate === 1) metrics.psu_ac_energy += value;
} else if (metric_name == 'memory_energy_rapl_msr_system') {
if (accumulate === 1) metrics.memory_energy += value;
} else if (metric_name == 'memory_total_cgroup_container') {
Expand Down Expand Up @@ -345,6 +352,10 @@ const createGraph = (element, data, labels, title) => {

const fillAvgContainers = (stats_data, metrics) => {

<<<<<<< HEAD
=======
const psu_ac_energy_in_mWh = ((metrics.psu_ac_energy) / 3600) * 1000;
>>>>>>> dev
const atx_energy_in_mWh = ((metrics.atx_energy) / 3600) * 1000;
const cpu_energy_in_mWh = ((metrics.cpu_energy) / 3600) * 1000;
const memory_energy_in_mWh = ((metrics.memory_energy) / 3600) * 1000;
Expand All @@ -365,6 +376,8 @@ const fillAvgContainers = (stats_data, metrics) => {
else if(total_CO2_in_kg < 0.0001) co2_display = { value: total_CO2_in_kg*(10**6), unit: 'mg'};
else if(total_CO2_in_kg < 0.1) co2_display = { value: total_CO2_in_kg*(10**3), unit: 'g'};

if(psu_ac_energy_in_mWh) document.querySelector("#psu-ac-energy").innerText = psu_ac_energy_in_mWh.toFixed(2) + " mWh"

if(atx_energy_in_mWh) document.querySelector("#atx-energy").innerText = atx_energy_in_mWh.toFixed(2) + " mWh"
if(cpu_energy_in_mWh) document.querySelector("#cpu-energy").innerText = cpu_energy_in_mWh.toFixed(2) + " mWh"
if(cpu_energy_in_mWh) document.querySelector("#component-energy").innerText = (cpu_energy_in_mWh+memory_energy_in_mWh).toFixed(2) + " mWh"
Expand All @@ -373,6 +386,10 @@ const fillAvgContainers = (stats_data, metrics) => {

if(cpu_energy_in_mWh) document.querySelector("#component-power").innerText = ((metrics.cpu_energy+metrics.memory_energy)/stats_data.project.measurement_duration_in_s).toFixed(2) + " W"
if(atx_energy_in_mWh) document.querySelector("#atx-power").innerText = (metrics.atx_energy / stats_data.project.measurement_duration_in_s).toFixed(2) + " W"
<<<<<<< HEAD
=======
if(psu_ac_energy_in_mWh) document.querySelector("#psu-ac-power").innerText = (metrics.psu_ac_energy / stats_data.project.measurement_duration_in_s).toFixed(2) + " W"
>>>>>>> dev


if(network_io) document.querySelector("#network-io").innerText = network_io.toFixed(2) + " MB"
Expand Down
24 changes: 24 additions & 0 deletions frontend/stats.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -252,6 +252,30 @@ <h3>compound metrics</h3>
</div>
</div>
</div>
<div class="ui card">
<div class="ui content">
<div class="ui top attached orange label">PSU AC Energy</div>
<div class="description">
<div class="ui tiny statistic">
<div class="value">
<i class="power off icon"></i> <span id="psu-ac-energy">N/A</span>
</div>
</div>
</div>
</div>
</div>
<div class="ui card">
<div class="ui content">
<div class="ui top attached orange label">PSU AC Power</div>
<div class="description">
<div class="ui tiny statistic">
<div class="value">
<i class="power off icon"></i> <span id="psu-ac-power">N/A</span>
</div>
</div>
</div>
</div>
</div>
<div class="ui card">
<div class="ui content">
<div class="ui top attached orange label">ATX Energy</div>
Expand Down
11 changes: 11 additions & 0 deletions tools/metric_providers/psu/energy/ac/system/README.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
Slimmed down version of check_gude.py from https://github.com/gudesystems/check_gude.py
for the Blauer Engel für Software Measurements
===============

This script expects the GUDE Powermeter to be fixed on the IP 192.168.178.32

- Create **venv**: `python3 -m venv venv`
- Activate: `source venv/bin/activate`
- Install requests: `pip3 install requests`
- Run: `python3 check_gude.py`

44 changes: 44 additions & 0 deletions tools/metric_providers/psu/energy/ac/system/check_gude_modified.py
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
#!/usr/bin/env python3

import requests
import json
import time
import subprocess

def main(resolution):
url = f'http://192.168.178.32/status.json'

ONLY_VALUES = 0x4000
cgi = {'components': ONLY_VALUES} # simple-sensors + and only values

resolution = float(resolution);

target_sleep_time = resolution/1000.0

while True: # loop until CTRL+C
timestamp_before = time.time_ns()
time.sleep(target_sleep_time)

data = json.loads(requests.get(url, params=cgi, verify=False, auth=None).text)

# print(data) # DEBUG
timestamp_after = time.time_ns()
effective_sleep_time = timestamp_after - timestamp_before
# print(effective_sleep_time / 1_000_000_000) # DEBUG
conversion_factor = effective_sleep_time / 1_000_000 # we want microjoule. Therefore / 10**9 to get seconds and then * 10**3 to get mJ
print(int(timestamp_after / 1_000), int(data["sensor_values"][0]['values'][0][4]['v']*conversion_factor), flush=True)

if __name__ == "__main__":
import argparse

parser = argparse.ArgumentParser()
parser.add_argument("-i", type=str, help="Resolution")

args = parser.parse_args()

if args.i is None:
parser.print_help()
print("Please supply -i to set resolution in milliseconds")
exit(2)

main(args.i)
49 changes: 49 additions & 0 deletions tools/metric_providers/psu/energy/ac/system/provider.py
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
import sys, os
import subprocess

if __name__ == "__main__":
sys.path.append(os.path.dirname(os.path.abspath(__file__))+'/../../../../..')
from metric_providers.base import BaseMetricProvider

class PsuEnergyAcSystemProvider(BaseMetricProvider):
def __init__(self, resolution):
self._current_dir = os.path.dirname(os.path.abspath(__file__))
self._metric_name = "psu_energy_ac_system"
self._metrics = {"time":int, "value":int}
self._resolution = resolution
super().__init__()

def start_profiling(self, containers=None):
call_string = f"{self._current_dir}/check_gude_modified.py -i {self._resolution}"

call_string += f" > {self._filename}"

print(call_string)

self._ps = subprocess.Popen(
[call_string],
shell=True,
preexec_fn=os.setsid,
stderr=subprocess.PIPE
# since we are launching the command with shell=True we cannot use ps.terminate() / ps.kill().
# This would just kill the executing shell, but not it's child and make the process an orphan.
# therefore we use os.setsid here and later call os.getpgid(pid) to get process group that the shell
# and the process are running in. These we then can send the signal to and kill them
)


if __name__ == "__main__":
import time
import argparse

parser = argparse.ArgumentParser()
parser.add_argument("container_id", help="Please provide the container_id")
args = parser.parse_args()

o = PsuEnergyAcSystemProvider(resolution=100)

print("Starting to profile")
o.start_profiling()
time.sleep(2)
o.stop_profiling()
print("Done, check ", o._filename)

0 comments on commit 77870b6

Please sign in to comment.