Deep Preset: Blending and Retouching Photos with Color Style Transfer (WACV'2021)

[Page] [Paper] [SupDoc] [SupVid] [5-min Presentation] [Slides]

Man M. Ho, Jinjia Zhou

Prerequisites

• Ubuntu 16.04
• Pillow
• PyTorch >= 1.1.0
• Numpy
• gdown (for fetching pretrained models)

Get Started

1. Clone this repo

git clone https://github.com/minhmanho/deep_preset.git
cd deep_preset

2. Fetch our trained model

Positive Pair-wise Loss (PPL) could improve Deep Preset in directly stylizing photos; however, it became worse in predicting preset, as described in our paper. Therefore, depending on your needs, please download Deep Preset with PPL for directly stylizing photos

./models/fetch_model_wPPL.sh

Or Deep Preset without PPL for preset prediction.

./models/fetch_model_woPPL.sh

Blending and Retouching Photos

Run our Deep Preset to stylize photos as:

CUDA_VISIBLE_DEVICES=0 python run.py \
    --content ./data/content/ \
    --style ./data/style/ \
    --out ./data/out/ \
    --ckpt ./models/dp_wPPL.pth.tar \
    --size 512x512 

Where --size is for the photo size [Width]x[Height], which should be divisible by 16. Besides, --size set as 352x352 will activate the preset prediction.

In case of only preset prediction needed, please add --p as:

CUDA_VISIBLE_DEVICES=0 python run.py \
    --content ./data/content/ \
    --style ./data/style/ \
    --out ./data/out/ \
    --ckpt ./models/dp_woPPL.pth.tar \
    --p

After processing, the predicted preset will be stored as a JSON file revealing how Lightroom settings are adjusted, as follows:

{
    "Highlights2012": -23,
    "Shadows2012": 4,
    "BlueHue": -8, 
    "Sharpness": 19, 
    "Clarity2012": -2
    ...
}

Cosplay Portraits

Photos were taken by Do Khang (taking the subject in the top-left one) and the first author (others).

Regarding training data

I planned to share the dataset and Lightroom Add-ons publicly. However, I don't have much time for it these days. Uploading the whole dataset is inefficient since it is about ~500GB. You can download the training images with natural colors labeled as "0" at this Google Drive Afterward, you will need to script Lightroom (or create a Plugin/Add-ons) to generate other styles.

(You can refer to these ugly lines of code)

local catalog = LrApplication.activeCatalog()
local photos = catalog:getTargetPhotos()
local catalog_folder = LrPathUtils.parent(catalog:getPath())

pfolder = "<folder containing json presets>"
outfolder = "<out folder>"
for i=1,500,1 do
    pdir = pfolder .. tostring(i) .. ".json"
    local file = io.open(pdir, 'r')
    if file then
        local contents = file:read( "*a" )
        p = json.decode(contents);
        io.close( file )
    else
        p = nil
    end

    for j, photo in ipairs(photos) do
        catalog:withWriteAccessDo ("Apply preset", function()
            local tmp = LrApplication.addDevelopPresetForPlugin( _PLUGIN, "Preset " .. tostring(i), p)
            photo:applyDevelopPreset (tmp, _PLUGIN)
        end)
    end
    local _out = outfolder .. tostring(i)
    LrFileUtils.createDirectory(_out)
    exportImage.exportPhotos(photos, _out)

Please check Adobe Lightroom software development kit (SDK) for more details.

Regarding the training code

(Updated 01/20/2024) In the past, I refrained from sharing training codes due to their associated costs. However, in response to the growing interest in the training code for this project, you can find the code here. Please note that it is not executable, as I have removed certain personal information and haven't verified it again.

Citation

If you find this work useful, please consider citing:

@InProceedings{Ho_2021_WACV,
    author    = {Ho, Man M. and Zhou, Jinjia},
    title     = {Deep Preset: Blending and Retouching Photos With Color Style Transfer},
    booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
    month     = {January},
    year      = {2021},
    pages     = {2113-2121}
}

Acknowledgements

We would like to thank:

• Do Khang for the photos of Duyen To.

• digantamisra98 for the Unofficial PyTorch Implementation of EvoNorm

Liu, Hanxiao, Andrew Brock, Karen Simonyan, and Quoc V. Le. "Evolving Normalization-Activation Layers." 
arXiv preprint arXiv:2004.02967 (2020).

• and Richard Zhang for the BlurPool.

Zhang, Richard. "Making convolutional networks shift-invariant again." 
ICML (2019).

License

Our code and trained models are for non-commercial uses and research purposes only.

Contact

If you have any questions, feel free to contact me (maintainer) at manminhho.cs@gmail.com