Skip to content

GATorch is a tool seamlessly integrated with PyTorch that enables ML developers to generate an energy consumption report. By attaching your model, the tool automatically tracks the energy consumption of your model's training and generates graphs and plots to gain in-depth insights into the energy consumption of your model.

License

Notifications You must be signed in to change notification settings

GreenAITorch/GATorch

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

30 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

GAtorch

Green AI torch tries to create awareness over energy consumption within the Pytorch ML framework. Its goal is to measure energy consumption throughout the complete process of AI engineers and can give overviews and indications for performance gains with respect to energy consumption.

Currently it supports energy measurement of the training passes per layer.

You can find the full API documentation here.

Installation

You can install GATorch using pip. To install GATorch, at the command line, run:

pip install GATorch

Source Installation

  • cuda >=11.7
  • cuDNN >=8

Create a virtualenv like virtualenv .venv and activate it using source ./.venv/bin/activate and install the other requirements with pip install requirements.txt.

Basic example

from GA import GA

# Create the profiler object and attach a model to it
ga_measure = GA()
ga_measure.attach_model(model)

# Let's try to do a single forward pass and a backward pass
x = torch.zeros([1, 1, 28, 28]).to(device)
y = torch.zeros([1, 10]).to(device)
pred = model(x) # forward

loss = loss_fn(pred, y)
optimizer.zero_grad()
loss.backward() # backward
optimizer.step()

# Now lets print the mean measurements
print(ga_measure.get_mean_measurements())

or run the example scripts in the examples directory.

Compatibility

Some older hardware might not support energy consumption measurements:

  • NVML requires Tesla Architecture NVIDIA GPUs or newer to work.
  • RAPLs DRAM measurements are only available for XENON CPUs.

In case you get compatibility errors due to older hardware you can disable the failing measurement application, use the disable_measurements parameter in the attach_model function. This parameter accepts a list of disabled measurements out of ['cpu', 'ram', 'gpu'], default is []. You need to use at least one measurement that is not disabled. The program will indicate that the disabled devices are unavailable.

Permissions

Due to Platypus attack Intel RAPL requires root permission for energy readings. In order to run this program with the correct permissions, do NOT make Intel RAPL readable for any user as this introduces vulnerability. Instead use Python with sudo instead:

sudo ./.venv/bin/python <script_name>.py

Tensorboard

This tool can automatically generate energy consumption reports and display these in Tensorboard. To use Tensorboard run tensorboard --logdir=runs and open the browser to view the graphs. This tool further allows for custom graph generation and tensorboard integration, but is not complete and needs to be extended.

Roadmap

The current architecture of this tool uses the integrated hooks of the PyTorch library, which restricts the current implementation towards the final goal of complete coverage including data loading, pre-processing, saving and loading a model etc. To give a more thorough analysis of the impact of energy consumption in ML development, this still needs to be developed.

This tool differs from other tools by measuring in-depth layers and system components and could be expanded to provide energy consumption data that can lead to recommendations for eliminating certain layers due to high energy consumption compared to accuracy gain.

PyJoules measures the energy consumption per individual hardware components and this data could be separated in order to provide a relative component view. Another improvement could be to measure the system component utilization over time, which can be an indicator of wasted energy.

About

GATorch is a tool seamlessly integrated with PyTorch that enables ML developers to generate an energy consumption report. By attaching your model, the tool automatically tracks the energy consumption of your model's training and generates graphs and plots to gain in-depth insights into the energy consumption of your model.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages