Guide to tweaking params

In a nutshell, you can use torch-dreams to optimize an input image to activate various parts of a neural network.This would help give an intuition on what each part of the neural network "looks for".

import matplotlib.pyplot as plt
import torchvision.models as models
from torch_dreams.dreamer import dreamer

torch_dreams.dreamer is basically a wrapper over any PyTorch model that would enable us to optimize the input image to activate various feature(s) within the neural network.

model = models.inception_v3(pretrained=True)
dreamy_boi = dreamer(model)

The Config

The config is where we get to customize how exactly we want the optimization to happen.

config = {
    "image_path": "your_image.jpg",
    "layers": [model.Mixed_6c.branch1x1],
    "octave_scale": 1.2,
    "num_octaves": 10,
    "iterations": 20,
    "lr": 0.03,
    "custom_func": None,
    "max_rotation": 0.5,
    "gradient_smoothing_coeff": 0.1,
    "gradient_smoothing_kernel_size": 3
}

• image_path: specifies the relative path to the input image.

• layers: List of layers whose outputs are to be "stored" for optimization later on. For example, if we want to use 2 layers:

  config["layers"] = [
      model.Mixed_6d.branch1x1,
      model.Mixed_5c
  ]

• octave_scale: Factor by which the image is scaled up after each octave.

• num_octaves: Number of times the image is scaled up in order to reach back to the original size.

• iterations: Number of gradient ascent steps taken per octave.

• lr: Learning rate used in each step of the gradient ascent.

• custom_func (optional): Use this to build your own custom optimization functions to optimize on individual channels/units/etc.

  For example, if we want to optimize the th 47th channel of the first layer and the 22nd channel of the 2nd layer simultaneously:

  def my_custom_func(layer_outputs):
      loss = layer_outputs[0][47].mean() + layer_outputs[1][22].mean()
      return loss
  config["custom_func"] = my_custom_func

• max_rotation (optional): Caps the maximum rotation to apply on the image.

• gradient_smoothing_coeff (optional): Higher -> stronger blurring.

• gradient_smoothing_kernel_size: (optional) Kernel size to be used when applying gaussian blur.