Info about Models (App Store Version)

Quality Benchmarks

• All values in the following tables stand for average PSNR among 229 anime-style illustrations, calculated on RGB channels. Higher means the image is closer to the original (ground truth).
• Core ML results below are only applicable on iOS 15+/macOS 12+. Older iOS/macOS versions produce worse results.
• Test images are scaled to 2x.

Comparison with waifu2x-caffe

	SRCNN	CUnet
PSNR (Core ML)	39.744	N/A
PSNR (MPSCNN)	39.793	41.408
PSNR (waifu2x-caffe)	39.812	41.418

Downsampling method is Lanczos in Pillow.

Different Downsampling Methods

Method	Lanczos	SRCNN (Core ML)	SRCNN (MPSCNN)	CUnet	PAN (HQ)	PAN (M1 fast)	PAN (A13 fast)
Bicubic	35.084	39.744	39.847	42.063	43.055	43.047	43.036
Bilinear	33.076	35.325	35.370	36.064	35.976	35.988	35.980
Box	35.297	39.916	39.988	41.375	42.204	42.167	42.194
Hamming	34.702	39.417	39.512	40.511	40.916	40.946	40.925
Lanczos	35.677	39.700	39.793	41.408	43.231	43.223	43.215
PS Bicubic Sharper*	33.694	34.441	34.462	34.845	35.398	35.387	35.397
PS Bicubic Smoother	35.377	39.933	40.031	41.609	42.156	42.127	42.138

*PS stands for Photoshop resampling methods. Other methods are from Pillow.

**Note that different models used in this app were trained with different downsampling methods. Actual performance will vary between different images.

***PAN HQ stands for high quality mode which is the default mode on version 5.0-5.0.1. Version 5.0.2+ uses fast mode by default.

Introduction

SRCNN

The original waifu2x model, fast and produces great result.

In this App both Core ML and MPSCNN are available. Although Core ML is significantly faster on newer devices (A12+), it is highly recommended that you use MPSCNN since the result from MPSCNN is much more consistent.

Source code for Core ML version is in this repository. For MPSCNN version please navigate to CoreML-MPS.

CUnet

Produces slightly better details than SRCNN. Consumes significantly more time and memory.

In this App only MPSCNN is available.

PAN

Based on this project. Highly efficient on mobile devices.

In this app models were trained on 22,272 anime-style noise-free images. It's slightly faster than CUnet on GPU. But it's significantly faster than CUnet while running on ANE. The quality is better than CUnet except for bilinear, which neither CUnet nor PAN was trained on.

By default models run in fast mode where the precision will be FP16 and ANE will be used. If ANE is not available, high quality mode will always be used where the precision will be forced to FP32. Under fast mode there will be very slight artifact, which is normally unnoticeable until you really calculate difference from FP32 output and bump contrast all the way up.

Due to a bug in ML Programs even if you enable high quality mode 16-bit PNG output will only have 11 bits precision.

FAQ

Why does CUnet require purchase?

I originally developed this App as a tech demo back in 2017. I never expected it to be as popular as it is now. This is also the reason why I kept it free to download and free of ADs.

However, almost 3 years has passed (as of mid-2020). The amount of time and energy I put into this App has been far beyond a tech demo. Some sort of income is important to keep me motivated to continue working on this App in the future.

SRCNN will always be free. But if you feel like using a slightly better model, please consider purchasing additional models. This is a one-time purchase and will be available on all your iOS devices.

Where are the additional models hosted?

On App Store server (link).

How did you convert the models?

Core ML is missing some features used by CUnet model. So that you just cannot convert the original caffe model directly to Core ML. Another issue is that Core ML has a ton of internal optimizations. Although it can improve performance, sometimes it will sacrifice precision and worsen the quality of output images. What I did was just exporting weights from caffe model and built the graph in MPSCNN by hand (very tedious). In this way I can make sure the model is running at the highest precision possible.

Will there be more models added in the future?

I have plan to do so. But cannot promise on anything.