As of this announcement PyTorch 1.3 now officialy supports an end-to-end workflow from Python to deployment on iOS and Android through PyTorch Mobile. Thank you for your interest with this project.
PyTorch on Android is a project to demo how to use PyTorch and ONNX to build an Android mobile application doing real time object classification.
The source code for the demo in this repo was originally based on AICamera repo and as of 2018-12-31, the codebase was based on Soumith's AICamera repo.
Project Status:
- 2019-05-13:
- Introducing PyTorch Lite — a lightweight machine learning framework for ON-DEVICE mobile inference. No cumbersome ONNX, no learning Caffe2. Just PyTorch and libtorch.
- 2018-12-31:
- PyTorch core maintainers have updated AICamera example to work with latest PyTorch master. Once that PR is merged into PyTorch master, you can use the README below to get a working Android app, including changing the Protobuf with your own
init.pb/predict.pbfiles. - Android OSS fixes PR.
- PyTorch core maintainers have updated AICamera example to work with latest PyTorch master. Once that PR is merged into PyTorch master, you can use the README below to get a working Android app, including changing the Protobuf with your own
- 2019-01-01:
- I have tested the Android OSS fixes with my own ResNet18 pre-trained ImageNet model (
resnet18_init_net_v1.pbandresnet18_predict_net_v1.pbProtobuf files) and the Android app is working fine.
- I have tested the Android OSS fixes with my own ResNet18 pre-trained ImageNet model (
- Early release. Still in heavy development. What this means is, important features are missing, things might be moved around quickly and things will break.
Make it easier to ship and test your neural network model in PyTorch on mobile devices.
This is an example for using Caffe2 on Android.
- git clone PyTorch source and switch to remote branch,
android_oss_fixes:
$ git clone --recursive https://github.com/pytorch/pytorch.git
$ cd pytorch
# We are using this PyTorch master commit 39381964bbb2686cf84c78aed638985455f6d3fe (Dec 23 02:08:33 2018 -0500, "fix build_android for newest NDK, and make install work")
$ git checkout --track origin/android_oss_fixes
M third_party/QNNPACK
M third_party/cpuinfo
M third_party/fbgemm
M third_party/ideep
Branch 'android_oss_fixes' set up to track remote branch 'android_oss_fixes' from 'origin'.
Switched to a new branch 'android_oss_fixes'-
You'll need to download the Android NDK (latest r18 release) if you have not.
-
First, we will compile Caffe2 Android libraries like libcaffe2, libqnnpack for arm-v7a ABI and then for x86.
Set build environment:
- PyTorch folder is at
$PYTORCH_ROOT - This repository folder is at
$AICAMERA_ROOT - Android NDK folder is at
$ANDROID_NDK
# make sure $PYTORCH_ROOT, $AICAMERA_ROOT and $ANDROID_NDK are set
export ANDROID_NDK=~/android/sdk/ndk-bundle/
export PYTORCH_ROOT=~/dev/gh/pytorch/
export AICAMERA_ROOT=~/dev/android/android-studio-projects/aicamera/
pushd $PYTORCH_ROOTThen, do the following:
Build Caffe2 android libs and copy them over into AICamera app folder
./scripts/build_android.shIf you encountered build errors related to "quantized/int8_*.cc", get the patch from this "Update QNNPACK" PR. Patch these files in your local copies.
You should see "Install configuration: "Release"" output in your terminal if the process completed successfully.
mv build_android build_android_arm
# copy headers
cp -r install/include/* $AICAMERA_ROOT/app/src/main/cpp/
# copy arm libs
rm -rf $AICAMERA_ROOT/app/src/main/jniLibs/armeabi-v7a/
mkdir $AICAMERA_ROOT/app/src/main/jniLibs/armeabi-v7a
cp -r build_android_arm/lib/lib* $AICAMERA_ROOT/app/src/main/jniLibs/armeabi-v7a/
./scripts/build_android.sh -DANDROID_ABI=x86
mv build_android build_android_x86
# copy x86 libs
rm -rf $AICAMERA_ROOT/app/src/main/jniLibs/x86/
mkdir $AICAMERA_ROOT/app/src/main/jniLibs/x86
cp -r build_android_x86/lib/lib* $AICAMERA_ROOT/app/src/main/jniLibs/x86/- Build the AICamera app using the
Build -> Make Projectmenu option in Android Studio
If you prefer not to build PyTorch from master, I have commited the files for the Caffe2 bits into this git repo. Unfortunately, not all files are uploaded to GitHub due to file size limit.
Grab the remaining large files from my Google Drive.
We created a developer guide on how to ship a convolutional neural network (Resnet18 and SqueezeNet) on Android with PyTorch and Android Studio. Check it out!
We'll walk you through every step, from problem all the way to building and deploying the Android app to mobile phones.
- Android (Java/C++) with Caffe2
- Test SqueezeNet v1.1 model with your own video stream from camera
- Live (real-time) detection
- Performance optimization
- Test build in models (MobileNetV2, ShuffleNet, Resnet18) with your own video stream from camera
- Download your own model on the fly and test it
- Manage models locally on your Android device
- Memory consumption and time elapse data
- Overall control on every layer (from beginner to expert)
- Warm community and welcome to contribute
- Transfer learning SqueezeNet with new datasets (i.e. insect species, not hotdog)
- New classifier for insect species dataset using pre-trained ImageNet weights (transfer learning)
- Upload pre-trained models
- Bug fixes
- Fix intermittent crashes
- React Native native module
- Upload large files from PyTorch 'build_android' to Google Drive and share it here
You can download the Android project source code by running this command:
git clone https://github.com/cedrickchee/pytorch-android.git
- Android Studio: 3.2.1 and above
- Android SDK
- Android NDK r18 and above
- CMake Android SKD component
- Gradle 4.6 and above
| Device | Network | FPS (^) |
|---|---|---|
| Google Nexus 6P | SqueezeNet | 3.0 |
| Google Nexus 6P | Resnet18 | 0.6 |
| Samsung Note 8 | SqueezeNet | TBD |
| Galaxy Note 3 | SqueezeNet | 4.0 |
| Samsung Galaxy S7 | SqueezeNet | 5.8 |
| Google Pixel | SqueezeNet | 5.7 |
Note: ^ the number of FPS is subjective to the camera photo (image) size you send to the mobile device as well as type of the device.
If building this repo is too much of a trouble for you, we also plan to put this in Google Play Store (TBD).
Visualize SqueezeNet 1.1 prediction network serialized as Protobuf file:
| Before update (Oct 2018) | After update (Jan 2019) |
|---|---|
 |
 |
TBD
Scope of work for this project: TBD
- PyTorch team response to "Getting error on Caffe2 AICamera example"
- Troubleshoot "Compiling PyTorch C++ for Android"
- t-vi's PyTorch proposal to "Improve build_android experience"
- t-v1's work to get Android app working again
- Convert fast.ai trained image classification model to iOS app via ONNX and Apple Core ML - note: for older fastai library version 0.7
- Building a Neural Style Transfer app on iOS with PyTorch and CoreML
- Fast neural style transfer for Android Neural Networks API (NNAPI)
Feel free to ask any questions, from preparing development environment to debugging on Android Studio. We are happy to help you.
This repository contains a variety of content; some developed by Cedric Chee, and some from third-parties. The third-party content is distributed under the license provided by those parties.
I am providing code and resources in this repository to you under an open source license. Because this is my personal repository, the license you receive to my code and resources is from me and not my employer.
The content developed by Cedric Chee is distributed under the following license:
The code in this repository, including all code samples in the notebooks listed above, is released under the MIT license. Read more at the Open Source Initiative.
The text content of the book is released under the CC-BY-NC-ND license. Read more at Creative Commons.