diff --git a/.wordlist.txt b/.wordlist.txt index 10571ce17..b38472514 100644 --- a/.wordlist.txt +++ b/.wordlist.txt @@ -308,7 +308,6 @@ Alibaba Altra AmazonRDS Analytics -Andoid Anonymized ArmDeveloperEcosystem ArmNN @@ -3420,4 +3419,95 @@ snortrules techmahindra unreferenced uptime -wC \ No newline at end of file +wC +ApiService +AppHost +ArmPyTorchMNISTInference +Blazor +CameraX +ComputationService +Coroutine +EOF +EVCLI +EVidence +Evcli +GC’s +GenerateMatrix +ImageCapture +InputStream +JWT +JetPack +KBS +MediaPipe's +Mongod +Multimodal +NNAPI +NPUs +NetAspire +OpenTelemetry +PIL +PerformIntensiveCalculations +ReactiveX's +ServiceDefaults +SharedFlow +Skopeo +StateFlow +TestOpenCV +TrustedFirmware +Veraison +WeatherForecast +WebGPU’s +Wiredtiger +androidml +ar +armpytorchmnistinference +codelabs +combinator +cooldown +coroutines +cryptographically +datatracker +debounce +decrypts +diagnosticDataCollectionDirectorySizeMB +eab +eth +evcli +googleblog +hanyin +honorSystemUmask +ietf +jsonviewer +keyFile +livestream +lockCodeSegmentsInMemory +matrixResult +matrixSize +maxIncomingConnections +mongod +mongosh +multimodality +multimodel +oplogSizeMB +optimizable +orchestrator +prebuild +preconfigured +relica +replSetName +rfc +serializable +setParameter +skopeo +subclasses +subproject +subproject's +subrepositories +suppressNoTLSPeerCertificateWarning +systemLog +tlsWithholdClientCertificate +unutilized +vLLM +veraison +verifier +vllm \ No newline at end of file diff --git a/assets/contributors.csv b/assets/contributors.csv index 0f5a4218d..39019b28e 100644 --- a/assets/contributors.csv +++ b/assets/contributors.csv @@ -47,3 +47,5 @@ Koki Mitsunami,Arm,,kmitsunami,, Chen Zhang,Zilliz,,,, Tianyu Li,Arm,,,, Georgios Mermigkis,VectorCamp,gMerm,georgios-mermigkis,,https://vectorcamp.gr/ +Ben Clark,Arm,,,, +Han Yin,Arm,hanyin-arm,nacosiren,, diff --git a/content/install-guides/gcc/arm-gnu.md b/content/install-guides/gcc/arm-gnu.md index cafebc2d0..bf2d7b481 100644 --- a/content/install-guides/gcc/arm-gnu.md +++ b/content/install-guides/gcc/arm-gnu.md @@ -27,7 +27,7 @@ There are many versions of the [Arm GNU Toolchain](https://developer.arm.com/Too However there are reasons you may wish to use earlier compiler versions, so older versions are also available. -## Download toolchain {#download} +## How do I download the Arm GNU Toolchain? {#download} Arm GNU Toolchain releases consist of cross toolchains for the following host operating systems: @@ -45,7 +45,7 @@ macOS Download the correct toolchain variant for your development needs from the [Arm Developer website](https://developer.arm.com/tools-and-software/open-source-software/developer-tools/gnu-toolchain/downloads). -## Installing on Linux +## How do I install the Arm GNU Toolchain on Linux? ### Use package installer @@ -72,13 +72,13 @@ export PATH=/path/to/install/dir/bin:$PATH Here is a specific example for an Arm Linux host and the AArch32 bare-metal target. ```bash { target="ubuntu:latest" } -wget https://developer.arm.com/-/media/Files/downloads/gnu/13.3.rel1/binrel/arm-gnu-toolchain-13.3.rel1-aarch64-arm-none-eabi.tar.xz -tar xJf arm-gnu-toolchain-13.3.rel1-aarch64-arm-none-eabi.tar.xz -C $HOME -echo 'export PATH="$PATH:$HOME/arm-gnu-toolchain-13.3.rel1-aarch64-arm-none-eabi/bin"' >> ~/.bashrc +wget https://developer.arm.com/-/media/Files/downloads/gnu/14.2.rel1/binrel/arm-gnu-toolchain-14.2.rel1-aarch64-arm-none-eabi.tar.xz +tar xJf arm-gnu-toolchain-14.2.rel1-aarch64-arm-none-eabi.tar.xz -C $HOME +echo 'export PATH="$PATH:$HOME/arm-gnu-toolchain-14.2.rel1-aarch64-arm-none-eabi/bin"' >> ~/.bashrc source ~/.bashrc ``` -## Installing on macOS +## How do I install the Arm GNU Toolchain on macOS? Downloads for `macOS` are available as tar files (`.tar.xz`) and package files (`.pkg`). @@ -98,7 +98,7 @@ Use a text editor to add the `bin` directory as a new line in `/etc/paths`. ```console sudo nano /etc/paths ``` -For example the path could be: `/Applications/ArmGNUToolchain/13.3.rel1/arm-none-eabi/bin` +For example the path could be: `/Applications/ArmGNUToolchain/14.2.rel1/arm-none-eabi/bin` The `/etc/paths` file is a list of paths to search. @@ -109,18 +109,21 @@ The `/etc/paths` file is a list of paths to search. /bin /usr/sbin /sbin -/Applications/ArmGNUToolchain/13.3.rel1/arm-none-eabi/bin +/Applications/ArmGNUToolchain/14.2.rel1/arm-none-eabi/bin ``` + ### Apple Silicon + Here is a specific example for macOS with Apple Silicon and the AArch32 bare-metal target. ```console -wget https://developer.arm.com/-/media/Files/downloads/gnu/13.3.rel1/binrel/arm-gnu-toolchain-13.3.rel1-darwin-arm64-arm-none-eabi.pkg -sudo installer -pkg arm-gnu-toolchain-13.3.rel1-darwin-arm64-arm-none-eabi.pkg -target / -echo '/Applications/ArmGNUToolchain/13.3.rel1/arm-none-eabi/bin' | sudo tee -a /etc/paths +wget https://developer.arm.com/-/media/Files/downloads/gnu/14.2.rel1/binrel/arm-gnu-toolchain-14.2.rel1-darwin-arm64-arm-none-eabi.pkg +sudo installer -pkg arm-gnu-toolchain-14.2.rel1-darwin-arm64-arm-none-eabi.pkg -target / +echo '/Applications/ArmGNUToolchain/14.2.rel1/arm-none-eabi/bin' | sudo tee -a /etc/paths ``` -## Installing on Windows +## How do I install the Arm GNU Toolchain on Windows? + Double-click on the installer (e.g. `gcc-arm-_version_--mingw-w64-i686-arm-none-eabi.exe`) and follow on-screen instructions. The installer can also be run on the command line. When run on diff --git a/content/install-guides/gcc/cross.md b/content/install-guides/gcc/cross.md index f34269894..8e31795b9 100644 --- a/content/install-guides/gcc/cross.md +++ b/content/install-guides/gcc/cross.md @@ -35,11 +35,13 @@ The executables for 64-bit are `aarch64-linux-gnu-gcc` and `aarch64-linux-gnu-g+ Software can be compiled on an `x86` or `Arm` host machine. -## Download +## How do I download a GCC cross compiler targeting Arm? The Linux package manager will download the required files so there are no special download instructions. -## Installation +## How do I install a GCC cross compiler on Linux? + +You can install a GCC cross compiler with Arm as a target architecture using Linux package managers. ### Installing on Debian based distributions such as Ubuntu @@ -78,6 +80,14 @@ To install the most common development tools use the commands below. dnf install gcc-arm-linux-gnu -y ``` +## How do I install a GCC cross compiler on macOS? + +You can install a GCC cross compiler with Arm as a target architecture using Homebrew, a package manager for macOS (and Linux). + +```console +brew install arm-none-eabi-gcc +``` + ## Setting up product license {#license} GCC is open source and freely available for use. diff --git a/content/install-guides/gcc/native.md b/content/install-guides/gcc/native.md index d8aa825d9..74a07e0ea 100644 --- a/content/install-guides/gcc/native.md +++ b/content/install-guides/gcc/native.md @@ -39,11 +39,11 @@ aarch64 If you see a different result, you are not using an Arm computer running 64-bit Linux. -## Download +## How do I download a native GCC compiler on Linux? The Linux package manager downloads the required files so there are no special instructions. -## Installation +## How do I install a native GCC compiler on Linux? ### Installing on Debian based distributions such as Ubuntu diff --git a/content/install-guides/skopeo.md b/content/install-guides/skopeo.md new file mode 100644 index 000000000..a3b8965c9 --- /dev/null +++ b/content/install-guides/skopeo.md @@ -0,0 +1,273 @@ +--- +title: Skopeo +draft: true +author_primary: Jason Andrews +minutes_to_complete: 10 +official_docs: https://github.com/containers/skopeo + +additional_search_terms: +- containers +- images +- registry + +layout: installtoolsall +multi_install: false +multitool_install_part: false +test_images: +- ubuntu:latest +test_maintenance: false +tool_install: true +weight: 1 +--- + +Skopeo is a command-line utility that performs various operations on container images and image repositories. It does not require a daemon to be running on your computer. + +This article explains how to install Skopeo for Ubuntu on Arm. + +Skopeo is available for Windows, macOS, and Linux and supports the Arm architecture. Refer to [Installing Skopeo](https://github.com/containers/skopeo/blob/main/install.md) for information about other operating systems and architectures. + +## What should I consider before installing Skopeo on Arm? + +Confirm you are using an Arm machine by running: + +```bash +uname -m +``` + +The output should be: +```output +aarch64 +``` + +If you see a different result, you are not using an Arm computer running 64-bit Linux. + +## How do I download and Install Skopeo for Ubuntu on Arm? + +The easiest way to install Skopeo is to use the package manager: + +```bash +sudo apt update +sudo apt install -y skopeo +``` + +Confirm the installation by checking the version: + +```bash +skopeo --version +``` + +To see the help message: + +```bash +skopeo --help +``` + +The output is: + +```output +Various operations with container images and container image registries + +Usage: + skopeo [flags] + skopeo [command] + +Available Commands: + copy Copy an IMAGE-NAME from one location to another + delete Delete image IMAGE-NAME + generate-sigstore-key Generate a sigstore public/private key pair + help Help about any command + inspect Inspect image IMAGE-NAME + list-tags List tags in the transport/repository specified by the SOURCE-IMAGE + login Login to a container registry + logout Logout of a container registry + manifest-digest Compute a manifest digest of a file + standalone-sign Create a signature using local files + standalone-verify Verify a signature using local files + sync Synchronize one or more images from one location to another + +Flags: + --command-timeout duration timeout for the command execution + --debug enable debug output + -h, --help help for skopeo + --insecure-policy run the tool without any policy check + --override-arch ARCH use ARCH instead of the architecture of the machine for choosing images + --override-os OS use OS instead of the running OS for choosing images + --override-variant VARIANT use VARIANT instead of the running architecture variant for choosing images + --policy string Path to a trust policy file + --registries.d DIR use registry configuration files in DIR (e.g. for container signature storage) + --tmpdir string directory used to store temporary files + -v, --version Version for Skopeo + +Use "skopeo [command] --help" for more information about a command. +``` + +## How do I get started with Skopeo? + +Some commands to get you started with Skopeo are demonstrated below. + +### How can I check if a container image supports Arm? + +To find out if an image is multi-architecture, including Arm, you can inspect the image's manifest. + +For example, to check if the dev container available for creating Arm Learning Paths supports the Arm architecture run: + +```bash +skopeo inspect --raw docker://docker.io/armswdev/learn-dev-container:latest | jq '.manifests[] | select(.platform.architecture == "arm64")' +``` + +The output is similar to: + +```output +{ + "mediaType": "application/vnd.docker.distribution.manifest.v2+json", + "size": 1574, + "digest": "sha256:b7abfdc06d4cb06dfbb644f0d7c50202f99f83298da7903ea6463de23b55fb10", + "platform": { + "architecture": "arm64", + "os": "linux" + } +} +``` + +If the command returns a result for an image, the image supports the `arm64` architecture. + +### How can I check if a container image supports multiple architectures? + +To find out if an image supports both `arm64` and `amd64` architectures, you can inspect the image's manifest for both architectures. + +For example, to check if the same dev container supports both architectures run: + +```bash +skopeo inspect --raw docker://docker.io/armswdev/learn-dev-container:latest | jq '.manifests[] | select(.platform.architecture == "arm64" or .platform.architecture == "amd64")' +``` + +The output confirms that both `arm64` and `amd64` are supported architectures as shown below: + +```output +{ + "mediaType": "application/vnd.docker.distribution.manifest.v2+json", + "size": 1574, + "digest": "sha256:15fe2dc0925c6e5da27048edcd034660f51216ad700cb7cf12cb7779c16e9bce", + "platform": { + "architecture": "amd64", + "os": "linux" + } +} +{ + "mediaType": "application/vnd.docker.distribution.manifest.v2+json", + "size": 1574, + "digest": "sha256:b7abfdc06d4cb06dfbb644f0d7c50202f99f83298da7903ea6463de23b55fb10", + "platform": { + "architecture": "arm64", + "os": "linux" + } +} +``` + +## What are some other uses for Skopeo? + +Copy an image from a registry to a local directory. This command is similar to `docker pull` and will copy the image from the remote registry to your local directory. + +```bash +skopeo copy docker://docker.io/armswdev/uname:latest dir:./uname +``` + +The output is: + +```output +Getting image source signatures +Copying blob cd741b12a7ea done | +Copying config d11135df72 done | +Writing manifest to image destination +``` + +Inspect an image in a remote registry: + +```bash +skopeo inspect docker://docker.io/armswdev/uname:latest +``` + +The output is: + +```output +{ + "Name": "docker.io/armswdev/uname", + "Digest": "sha256:4f1fe1e1e1ad179bb2cec6b3c8b458d6ead02bd7459798a357791353b867462d", + "RepoTags": [ + "latest" + ], + "Created": "2023-03-08T04:32:41.063980445Z", + "DockerVersion": "", + "Labels": { + "org.opencontainers.image.ref.name": "ubuntu", + "org.opencontainers.image.version": "22.04" + }, + "Architecture": "arm64", + "Os": "linux", + "Layers": [ + "sha256:cd741b12a7eaa64357041c2d3f4590c898313a7f8f65cd1577594e6ee03a8c38" + ], + "LayersData": [ + { + "MIMEType": "application/vnd.oci.image.layer.v1.tar+gzip", + "Digest": "sha256:cd741b12a7eaa64357041c2d3f4590c898313a7f8f65cd1577594e6ee03a8c38", + "Size": 27347481, + "Annotations": null + } + ], + "Env": [ + "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" + ] +} +``` + +List tags in a remote registry for the Ubuntu image with many tags: + +```bash +skopeo list-tags docker://docker.io/library/ubuntu +``` + +The partial output is: + +```output +{ + "Repository": "docker.io/library/ubuntu", + "Tags": [ + "10.04", + "12.04", + "12.04.5", + "12.10", + "13.04", + "13.10", + "14.04", + "14.04.1", + "14.04.2", + "14.04.3", + "14.04.4", + "14.04.5", + "14.10", + "15.04", + "15.10", + "16.04", + "16.10", + "17.04", + "17.10", + "18.04", + "18.10", + "19.04", + "19.10", + "20.04", + "20.10", + "21.04", + "21.10", + "22.04", + "22.10", + "23.04", + "23.10", + "24.04", + "24.10", + "25.04", +< many more tag with Ubuntu release names omitted> +``` + +You are ready to use Skopeo for your projects. diff --git a/content/learning-paths/cross-platform/_example-learning-path/setup.md b/content/learning-paths/cross-platform/_example-learning-path/setup.md index d3bc15bb7..a2cbec60b 100644 --- a/content/learning-paths/cross-platform/_example-learning-path/setup.md +++ b/content/learning-paths/cross-platform/_example-learning-path/setup.md @@ -8,9 +8,9 @@ weight: 3 # 1 is first, 2 is second, etc. layout: "learningpathall" --- -## Setup a computer +## Set up a computer -There are multiple ways to setup a computer for Learning Path creation. +There are multiple ways to set up a computer for Learning Path creation. Three tools are mandatory: - A text editor to create and modify markdown files @@ -30,12 +30,12 @@ This provides your own copy for you to make changes without impacting the main r ## Set up a development machine -Three possible ways to setup a development machine are covered below. +Three possible ways to set up a development machine are covered below. Select **one** that works for you. Please share other ways that work for you - A [local computer (Linux, macOS, or Windows)](#option-1-set-up-a-local-computer) -- A [Gitpod instance](#option-2-set-up-gitpod) with tools pre-installed (easiest to setup and manage) +- A [Gitpod instance](#option-2-set-up-gitpod) with tools pre-installed (easiest to set up and manage) - A [remote Linux server](#option-3-set-up-a-remote-linux-server) via SSH (on your local network or from a Cloud Service Provider) ## Option 1: Set up a local computer diff --git a/content/learning-paths/cross-platform/docker/_images/ecr.png b/content/learning-paths/cross-platform/docker/_images/ecr.png new file mode 100644 index 000000000..5e9658d37 Binary files /dev/null and b/content/learning-paths/cross-platform/docker/_images/ecr.png differ diff --git a/content/learning-paths/cross-platform/docker/_images/hub.png b/content/learning-paths/cross-platform/docker/_images/hub.png new file mode 100644 index 000000000..c8c14ab3a Binary files /dev/null and b/content/learning-paths/cross-platform/docker/_images/hub.png differ diff --git a/content/learning-paths/cross-platform/docker/buildx.md b/content/learning-paths/cross-platform/docker/buildx.md index 384b72c46..62728d35f 100644 --- a/content/learning-paths/cross-platform/docker/buildx.md +++ b/content/learning-paths/cross-platform/docker/buildx.md @@ -13,11 +13,13 @@ layout: "learningpathall" Any computer running Docker can be used for this section. Before you begin, confirm Docker `buildx` is installed by running the `docker buildx` command. + ```console -docker buildx help +docker buildx --help ``` The result should be the usage message from `buildx`. It starts with the the text below: + ```output Usage: docker buildx [OPTIONS] COMMAND ``` diff --git a/content/learning-paths/cross-platform/docker/check-images.md b/content/learning-paths/cross-platform/docker/check-images.md new file mode 100644 index 000000000..aaa067a8c --- /dev/null +++ b/content/learning-paths/cross-platform/docker/check-images.md @@ -0,0 +1,203 @@ +--- +title: "Check container images for multi-architecture support" +weight: 6 +layout: "learningpathall" +--- + +There are many ways to check if container images are built for multiple architectures and if they support the Arm architecture. + +Some of the common ways are provided below. + +## Look at the image information in the container registry + +One way to check images is to use a browser and inspect the image in the registry. + +For example, on Docker Hub, the architectures are printed. + +![Docker Hub architectures](_images/hub.png) + +AWS ECR Public registry also prints the architectures. + +![AWS ECR Public architectures](_images/ecr.png) + +If you see `arm64`, `ARM64`, or `arm64v8` on the list, then the image supports Arm. + +Not all container registries have a web view showing architecture support. Additional ways to find the information are provided below. + +## Use Docker CLI commands + +#### Inspect the image manifest + +```bash +docker manifest inspect armswdev/uname:latest +``` + +Look for "arm64" in the output. + +You can use `grep` to check for `arm64`: + +```bash +docker manifest inspect armswdev/uname:latest | grep arm64 +``` + +The output confirms `arm64` support: + +```output +"architecture": "arm64", +``` + +#### Use Docker buildx to inspect the image + +```bash +docker buildx imagetools inspect --format '{{json (index .Image "linux/arm64")}}' armswdev/uname:latest +``` + +If this command returns JSON data, the image supports Arm. + +If the image does not support Arm an error is printed: + +```output +ERROR: template: :1:8: executing "" at : error calling index: can't index item of type v1.Image +``` + +#### Inspect image metadata + +The `image inspect` command works on local images so use `docker pull` to get the image. + +```bash +docker image inspect armswdev/uname:latest +``` + +Look for "Architecture": "arm64" in the output. + +### Use a formatted inspect command + +For concise output print only the architecture: + +```bash +docker inspect -f '{{.Architecture}}' armswdev/uname:latest +``` + +This will directly output the architecture. + +```output +arm64 +``` + +### Can I use a script to check if a container image supports the Arm architecture? + +You can run a script to check container images for `arm64` support. + +The script performs the following tasks: +- Get a token for the registry +- Read the image manifest +- Check the manifest for architecture support + +Make sure Python3 is installed on your computer. + +Use a text editor to copy the Python code below to a file named `check-image.py`. + +```python +import requests +import sys +import os +import argparse +from typing import List, Dict, Tuple + +# Target architectures to check +TARGET_ARCHITECTURES = {'amd64', 'arm64'} +TIMEOUT_SECONDS = 10 + +def get_auth_token(repository: str) -> str: + """Get Docker Hub authentication token.""" + url = "https://auth.docker.io/token" + params = { + "service": "registry.docker.io", + "scope": f"repository:{repository}:pull" + } + try: + response = requests.get(url, params=params, timeout=TIMEOUT_SECONDS) + response.raise_for_status() + return response.json()['token'] + except requests.exceptions.RequestException as e: + print(f"Failed to get auth token: {e}", file=sys.stderr) + sys.exit(1) + +def get_manifest(repository: str, tag: str, token: str) -> Dict: + """Fetch manifest for specified image.""" + headers = { + 'Accept': 'application/vnd.docker.distribution.manifest.list.v2+json', + 'Authorization': f'Bearer {token}' + } + url = f"https://registry-1.docker.io/v2/{repository}/manifests/{tag}" + try: + response = requests.get(url, headers=headers, timeout=TIMEOUT_SECONDS) + response.raise_for_status() + return response.json() + except requests.exceptions.RequestException as e: + print(f"Failed to get manifest: {e}", file=sys.stderr) + sys.exit(1) + +def check_architectures(manifest: Dict) -> List[str]: + """Check available architectures in the manifest.""" + if manifest.get('manifests'): + archs = [m['platform']['architecture'] for m in manifest['manifests']] + return archs + else: + return [] + +def parse_image_spec(image: str) -> Tuple[str, str]: + """Parse image specification into repository and tag.""" + if ':' in image: + repository, tag = image.split(':', 1) + else: + repository, tag = image, 'latest' + + if '/' not in repository: + repository = f'library/{repository}' + return repository.lower(), tag + +def parse_args(): + """Parse command line arguments.""" + parser = argparse.ArgumentParser(description='Check Docker image architectures') + parser.add_argument('image', help='Docker image name (format: name:tag)') + return parser.parse_args() + +if __name__ == "__main__": + args = parse_args() + repository, tag = parse_image_spec(args.image) + + token = get_auth_token(repository) + manifest = get_manifest(repository, tag, token) + architectures = check_architectures(manifest) + + if not architectures: + print(f"No architectures found for {args.image}", file=sys.stderr) + sys.exit(1) + + available_targets = TARGET_ARCHITECTURES.intersection(architectures) + missing_targets = TARGET_ARCHITECTURES - set(architectures) + + if not missing_targets: + print(f"✓ Image {args.image} supports all required architectures") + else: + print(f"✗ Image {args.image} is missing architectures: {', '.join(missing_targets)}") + print(f"Available architectures: {', '.join(architectures)}") +``` + +The script queries Docker Hub. If needed, you can change the registry and the architecture list to meet your needs. + +Run the script asking about Alpine Linux: + +```bash +python3 ./check-image.py alpine:3.21.0 +``` + +The output indicates that the image supports both `arm64` and `amd64`: + +```output +✓ Image alpine:3.21.0 supports all required architectures +``` + +You can now identify if container images are built for multi-architecture support, including the Arm architecture. + diff --git a/content/learning-paths/cross-platform/function-multiversioning/examples2.md b/content/learning-paths/cross-platform/function-multiversioning/examples2.md index ef76e5077..2ed806dec 100644 --- a/content/learning-paths/cross-platform/function-multiversioning/examples2.md +++ b/content/learning-paths/cross-platform/function-multiversioning/examples2.md @@ -10,8 +10,6 @@ This example computes the dot product of two vectors using Arm C Language Extens The intention is to enable the compiler to use SVE instructions in the specialized case, while restricting it to use only Armv8 instructions in the default case. -More details on the default implementation can be found in [Implement dot product of two vectors](/learning-paths/smartphones-and-mobile/android_neon/dot_product_neon). - Use a text editor to create a file named `dotprod.c` with the code below: ```c diff --git a/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/Figures/05.png b/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/Figures/05.png new file mode 100644 index 000000000..97e2fb65f Binary files /dev/null and b/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/Figures/05.png differ diff --git a/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/Figures/06.png b/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/Figures/06.png new file mode 100644 index 000000000..b76e2d056 Binary files /dev/null and b/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/Figures/06.png differ diff --git a/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/Figures/07.jpg b/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/Figures/07.jpg new file mode 100644 index 000000000..fdd5a3152 Binary files /dev/null and b/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/Figures/07.jpg differ diff --git a/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/Figures/08.jpg b/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/Figures/08.jpg new file mode 100644 index 000000000..f6c7721c5 Binary files /dev/null and b/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/Figures/08.jpg differ diff --git a/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/_index.md b/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/_index.md index 23ac166cb..f76e3568d 100644 --- a/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/_index.md +++ b/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/_index.md @@ -1,28 +1,32 @@ --- title: Create and train a PyTorch model for digit classification -minutes_to_complete: 80 +minutes_to_complete: 160 -who_is_this_for: This is an introductory topic for software developers interested in learning how to use PyTorch to create and train a feedforward neural network for digit classification. +who_is_this_for: This is an advanced topic for software developers interested in learning how to use PyTorch to create and train a feedforward neural network for digit classification. You will also learn how to use the trained model in an Android application. Finally, you will apply model optimizations. learning_objectives: - Prepare a PyTorch development environment. - Download and prepare the MNIST dataset. - Create a neural network architecture using PyTorch. - Train a neural network using PyTorch. + - Create an Android app and loading the pre-trained model. + - Prepare an input dataset. + - Measure the inference time. + - Optimize a neural network architecture using quantization and fusing. + - Use an optimized model in the Android application. prerequisites: - - A computer that can run Python3 and Visual Studio Code. The OS can be Windows, Linux, or macOS. + - A computer that can run Python3, Visual Studio Code, and Android Studio. The OS can be Windows, Linux, or macOS. author_primary: Dawid Borycki ### Tags -skilllevels: Introductory +skilllevels: Advanced subjects: ML armips: - Cortex-A - - Cortex-X - Neoverse operatingsystems: - Windows @@ -31,6 +35,7 @@ operatingsystems: tools_software_languages: - Android Studio - Coding + - VS Code shared_path: true shared_between: - servers-and-cloud-computing diff --git a/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/_review.md b/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/_review.md index fb1980742..8347d010f 100644 --- a/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/_review.md +++ b/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/_review.md @@ -34,12 +34,12 @@ review: Which loss function was used to train the PyTorch model on the MNIST dataset? answers: - Mean Squared Error Loss - - CrossEntropyLoss + - Cross Entropy Loss - Hinge Loss - Binary Cross-Entropy Loss correct_answer: 2 explanation: > - The CrossEntropyLoss function was used to train the model because it is suitable for multi-class classification tasks like digit classification. It measures the difference between the predicted probabilities and the true class labels, helping the model learn to make accurate predictions. + Cross Entropy Loss was used to train the model because it is suitable for multi-class classification tasks like digit classification. It measures the difference between the predicted probabilities and the true class labels, helping the model learn to make accurate predictions. # ================================================================================ # FIXED, DO NOT MODIFY diff --git a/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/app.md b/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/app.md new file mode 100644 index 000000000..d591afe57 --- /dev/null +++ b/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/app.md @@ -0,0 +1,35 @@ +--- +# User change +title: "Run the Application" + +weight: 10 + +layout: "learningpathall" +--- + +You are now ready to run the Android application. You can use an emulator or a physical device. + +The screenshots below show an emulator. + +To run the app in Android Studio using an emulator, follow these steps: + +1. Configure the Emulator: +* Go to Tools > Device Manager (or click the Device Manager icon on the toolbar). +* Click Create Device to set up a new virtual device (if you haven’t done so already). +* Choose a device model, such as Pixel 4, and click Next. +* Select a system image, such as Android 11, API level 30, and click Next. +* Review the settings and click Finish to create the emulator. + +2. Run the App: +* Make sure the emulator is selected in the device dropdown menu in the toolbar (next to the “Run” button). +* Click the Run button (a green triangle). Android Studio will build the app, install it on the emulator, and launch it. + +3. View the App on the Emulator: Once the app is installed, it will automatically open on the emulator screen, allowing you to interact with it as if it were running on a real device. + +Once the application is started, click the Load Image button. It will load a randomly selected image. Then, click Run Inference to recognize the digit. The application will display the predicted label and the inference time as shown below: + +![img](Figures/05.png) + +![img](Figures/06.png) + +In the next step you will learn how to further optimize the model. diff --git a/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/datasets-and-training.md b/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/datasets-and-training.md index d50b6d3c4..d1e499113 100644 --- a/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/datasets-and-training.md +++ b/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/datasets-and-training.md @@ -1,12 +1,14 @@ --- # User change -title: "Datasets and training" +title: "Perform training and save the model" weight: 5 layout: "learningpathall" --- +## Prepare the MNIST data + Start by downloading the MNIST dataset. Proceed as follows: 1. Open the pytorch-digits.ipynb you created earlier. @@ -60,7 +62,7 @@ The certifi Python package provides the Mozilla root certificates, which are ess Make sure to replace `x` with the number of Python version you have installed. -After running the code you will see the output that might look like shown below: +After running the code you see output similar to the screenshot below: ![image](Figures/01.png) @@ -122,18 +124,18 @@ for t in range(epochs): test_loop(test_dataloader, model, loss_fn) ``` -After running this code, you will see the following output that shows the training progress. +After running the code, you see the following output showing the training progress. ![image](Figures/02.png) -Once the training is complete, you will see something like the following: +Once the training is complete, you see output similar to: ```output Epoch 10: Accuracy: 95.4%, Avg loss: 1.507491 ``` -which shows the model achieved around 95% of accuracy. +The output shows the model achieved around 95% accuracy. # Save the model @@ -174,4 +176,4 @@ Setting the model to evaluation mode before tracing is important for several rea 3. Correct Tracing. Tracing captures the operations performed by the model using a given input. If the model is in training mode, the traced graph may include operations related to dropout and batch normalization updates. These operations can affect the correctness and performance of the model during inference. -In the next step, you will use the saved model for inference. +In the next step, you will use the saved model for ML inference. diff --git a/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/inference.md b/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/inference.md index 1e4e85ef7..9aed5754e 100644 --- a/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/inference.md +++ b/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/inference.md @@ -1,6 +1,6 @@ --- # User change -title: "Inference" +title: "Use the model for inference" weight: 6 @@ -26,7 +26,7 @@ Start by installing matplotlib package: pip install matplotlib ``` -Then, in Visual Studio Code create a new file named `pytorch-digits-inference.ipynb` and modify the file to include the code below: +Use Visual Studio Code to create a new file named `pytorch-digits-inference.ipynb` and modify the file to include the code below: ```python import torch @@ -101,12 +101,14 @@ After running the code, you should see results similar to the following figure: ![image](Figures/03.png) -# What you have learned +# What have you learned? -In this exercise, you went through the complete process of training and using a PyTorch model for digit classification on the MNIST dataset. Using the training dataset, you optimized the model’s weights and biases over multiple epochs. You employed the CrossEntropyLoss function and the Adam optimizer to minimize prediction errors and improve accuracy. You periodically evaluated the model on the test dataset to monitor its performance, ensuring it was learning effectively without overfitting. +You have completed the process of training and using a PyTorch model for digit classification on the MNIST dataset. Using the training dataset, you optimized the model’s weights and biases over multiple epochs. You employed the CrossEntropyLoss function and the Adam optimizer to minimize prediction errors and improve accuracy. You periodically evaluated the model on the test dataset to monitor its performance, ensuring it was learning effectively without overfitting. After training, you saved the model using TorchScript, which captures both the model’s architecture and its learned parameters. This made the model portable and independent of the original class definition, simplifying deployment. Next, you performed inference. You loaded the saved model and set it to evaluation mode to ensure that layers like dropout and batch normalization behaved correctly during inference. You randomly selected 16 images from the MNIST test dataset to evaluate the model’s performance on unseen data. For each selected image, you used the model to predict the digit, comparing the predicted labels with the actual ones. You displayed the images alongside their actual and predicted labels in a 4x4 grid, visually assessing the model’s accuracy and performance. This comprehensive process, from model training and saving to inference and visualization, illustrates the end-to-end workflow for building and deploying a machine learning model in PyTorch. It demonstrates how to train a model, save it in a portable format, and then use it to make predictions on new data. + +In the next step, you will learn how to use the model in an Android application. \ No newline at end of file diff --git a/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/intro-android.md b/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/intro-android.md new file mode 100644 index 000000000..0e29ad251 --- /dev/null +++ b/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/intro-android.md @@ -0,0 +1,36 @@ +--- +# User change +title: "Understand inference on Android" + +weight: 7 + +layout: "learningpathall" +--- + +Running pre-trained machine learning models on mobile and edge devices has become increasingly common as it enables these devices to gain intelligence and perform complex tasks directly on-device. This capability allows smartphones, IoT devices, and embedded systems to execute advanced functions such as image recognition, natural language processing, and real-time decision-making without relying on cloud-based services. + +By leveraging on-device inference, applications can offer faster responses, reduced latency, enhanced privacy, and offline functionality, making them more efficient and capable of handling sophisticated tasks in various environments. + +Arm provides a wide range of hardware and software accelerators designed to optimize the performance of machine learning (ML) models on edge devices. These include specialized processors like Arm's Neural Processing Units (NPUs) and Graphics Processing Units (GPUs), as well as software frameworks like the Arm Compute Library and Arm NN, which are tailored to leverage these hardware capabilities. + +Running a machine learning model on Android involves a few key steps. + +First, you train and save the model in a mobile-friendly format, such as TensorFlow Lite, ONNX, or TorchScript, depending on the framework you are using. + +Next, you add the model file to your Android project's assets directory. In your application's code, use the corresponding framework's Android library, such as TensorFlow Lite or PyTorch Mobile, to load the model. + +You then prepare the input data, ensuring it is formatted and preprocessed in the same way as during model training. The input data is passed through the model, and the output predictions are retrieved and interpreted accordingly. For improved performance, you can leverage hardware acceleration using Android’s Neural Networks API (NNAPI) or use GPU support if available. This process enables the Android app to make real-time predictions and execute complex machine learning tasks directly on the device. + +In this Learning Path, you will learn how to perform inference in an Android application using the pre-trained digit classifier from the previous sections. + +## Before you begin + +Before you begin make [Android Studio](https://developer.android.com/studio/install) is installed on your system. + +## Project source code + +The following steps explain how to build an Android application for MNIST inference. The application can be constructed from scratch, but there are two GitHub repositories available if you need to copy any files from them as you learn how to create the Android application. + +The complete source code for the [Android application](https://github.com/dawidborycki/Arm.PyTorch.MNIST.Inference.git) is available on GitHub. + +The [Python scripts](https://github.com/dawidborycki/Arm.PyTorch.MNIST.Inference.Python.git) used in the previous steps are also available on GitHub. diff --git a/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/intro-opt.md b/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/intro-opt.md new file mode 100644 index 000000000..870aa445d --- /dev/null +++ b/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/intro-opt.md @@ -0,0 +1,65 @@ +--- +# User change +title: "Optimizing neural network models in PyTorch" + +weight: 11 + +layout: "learningpathall" +--- + +## Optimizing models + +Optimizing models is crucial to achieving efficient performance while minimizing resource consumption. + +Because mobile and edge devices can have limited computational power, memory, and energy availability, various strategies are used to ensure that ML models can run effectively in these constrained environments. + +### Quantization + +Quantization is one of the most widely used techniques, which reduces the precision of the model's weights and activations from floating-point to lower-bit representations, such as int8 or float16. This not only reduces the model size but also accelerates inference speed on hardware that supports lower precision arithmetic. + +### Layer fusion + +Another key optimization strategy is layer fusion, where multiple operations, such as combining linear layers with their subsequent activation functions (like ReLU), into a single layer. This reduces the number of operations that need to be executed during inference, minimizing latency and improving throughput. + +### Pruning + +In addition to these techniques, pruning, which involves removing less important weights or neurons from the model, can help in creating a leaner model that requires fewer resources without significantly affecting accuracy. + + +### Android NNAPI + +Leveraging hardware-specific optimizations, such as the Android Neural Networks API (NNAPI) allows you to take full advantage of the underlying hardware acceleration available on edge devices. + +### More on optimization + +By employing these strategies, you can significantly enhance the efficiency of ML models for deployment on mobile and edge platforms, ensuring a balance between performance and resource utilization. + +PyTorch offers robust support for various optimization techniques that enhance the performance of machine learning models for edge and mobile inference. + +One of the key PyTorch features is its quantization toolkit, which provides a streamlined workflow for applying quantization to models. PyTorch supports both static and dynamic quantization, allowing developers to reduce model size and improve inference speed without sacrificing accuracy. + +Additionally, PyTorch enables layer fusion through its torch.quantization module, enabling seamless integration of operations like fusing linear layers with their activation functions, thus optimizing execution by minimizing computational overhead. + +Furthermore, the TorchScript functionality allows for the creation of serializable and optimizable models that can be efficiently deployed on mobile devices. + +PyTorch’s integration with hardware acceleration libraries, such as NNAPI for Android, enables developers to leverage specific hardware capabilities, ensuring optimal model performance tailored to the device's architecture. + +Overall, PyTorch provides a comprehensive ecosystem that empowers developers to implement effective optimizations for mobile and edge deployment, enhancing both speed and efficiency. + +### Optimization Next steps + +In the following sections, you will delve into the techniques of quantization and fusion using the previously created neural network model and Android application. + +By applying quantization, you will reduce the model's weight precision, transitioning from floating-point representations to lower-bit formats, which not only minimizes the model size but also enhances inference speed. This process is crucial for optimizing our model for deployment on resource-constrained devices. + +Additionally, you will explore layer fusion, which combines multiple operations within the model, such as fusing linear layers with their subsequent activation functions into a single operation. This reduction in operational complexity further streamlines the model, leading to improved performance during inference. + +By implementing these optimizations, you can enhance the efficiency of the digit classification model, making it well-suited for deployment in mobile and edge environments. + +First, you will modify the previous Python scripts for training and inference to incorporate model optimizations like quantization and fusion. + +After adjusting the training pipeline to produce an optimized version of the model, you will update the inference script to handle both the original and optimized models. + +Once these changes are made, you will modify the Android application to load either the original or the optimized model based on user input, allowing you to switch between them dynamically. + +This setup enables you to compare the inference speed of both models on the device, providing valuable insights into the performance benefits of model optimization techniques in real-world scenarios. \ No newline at end of file diff --git a/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/intro.md b/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/intro.md index af7cffde5..536228ccc 100644 --- a/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/intro.md +++ b/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/intro.md @@ -7,6 +7,8 @@ weight: 2 layout: "learningpathall" --- +## Introduction to PyTorch + PyTorch is an open-source deep learning framework that is developed by Meta AI and is now part of the Linux Foundation. PyTorch is designed to provide a flexible and efficient platform for building and training neural networks. It is widely used due to its dynamic computational graph, which allows users to modify the architecture during runtime, making debugging and experimentation easier. @@ -18,11 +20,11 @@ Prior to PyTorch, many frameworks used static computation graphs that require th Additionally, PyTorch seamlessly integrates with Python, encouraging a native coding experience. Its deep integration with GPU acceleration also makes it a powerful tool for both research and production environments. This combination of flexibility, usability, and performance has contributed to PyTorch’s rapid adoption, especially in academic research, where experimentation and iteration are crucial. -A typical process for creating a feedforward neural network in PyTorch involves defining a sequential stack of fully-connected layers, which are also known as *linear layers*. Each layer transforms the input by applying a set of weights and biases, followed by an activation function like ReLU. PyTorch supports this process using the torch.nn module, where layers are easily defined and composed. +A typical process for creating a feedforward neural network in PyTorch involves defining a sequential stack of fully-connected layers, which are also known as linear layers. Each layer transforms the input by applying a set of weights and biases, followed by an activation function like ReLU. PyTorch supports this process using the torch.nn module, where layers are easily defined and composed. To create a model, users subclass the torch.nn.Module class, defining the network architecture in the __init__ method, and implement the forward pass in the forward method. PyTorch’s intuitive API and support for GPU acceleration make it ideal for building efficient feedforward networks, particularly in tasks such as image classification and digit recognition. -In this Learning Path, you will explore how to use PyTorch for creating a model for digit recognition, before then proceeding to train it. +In this Learning Path, you will explore how to use PyTorch to create and train a model for digit recognition. ## Before you begin @@ -40,7 +42,7 @@ Python 3.11.2 If Python3 is not installed, download and install it from [python.org](https://www.python.org/downloads/). -Alternatively, you can also install Python3 using package managers such as Brew or APT. +Alternatively, you can also install Python3 using package managers such as Homebrew or APT. If you are using Windows on Arm you can refer to the [Python install guide](https://learn.arm.com/install-guides/py-woa/). @@ -72,9 +74,9 @@ pytorch-env\Scripts\activate source pytorch-env/bin/activate ``` -Once activated, you should see the virtual environment name in your terminal prompt. +Once activated, you see the virtual environment name `(pytorch-env)` before your terminal prompt. -3. Install PyTorch using `pip`: +3. Install PyTorch using Pip: ```console pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu diff --git a/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/intro2.md b/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/intro2.md index ae6126132..c1e3a1021 100644 --- a/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/intro2.md +++ b/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/intro2.md @@ -1,12 +1,14 @@ --- # User change -title: "PyTorch model training" +title: "About PyTorch model training" weight: 4 layout: "learningpathall" --- +## PyTorch model training + In the previous section, you created a feedforward neural network for digit classification using the MNIST dataset. The network was left untrained and lacks the ability to make accurate predictions. To enable the network to recognize handwritten digits effectively, training is needed. Training in PyTorch involves configuring the network's parameters, such as weights and biases, by exposing the model to labeled data and iteratively adjusting these parameters to minimize prediction errors. This process allows the model to learn the patterns in the data, enabling it to make accurate classifications on new, unseen inputs. @@ -15,13 +17,13 @@ The typical approach to training a neural network in PyTorch involves several ke First, obtain and preprocess the dataset, which usually includes normalizing the data and converting it into a format suitable for the model. -Next, the dataset is split into training and testing subsets. Training data is used to update the model’s parameters, while testing data evaluates its performance. During training, feed batches of input data through the network, calculate the prediction error or loss using a loss function (such as cross-entropy for classification tasks), and optimize the model’s weights and biases using backpropagation. Backpropagation involves computing the gradient of the loss with respect to each parameter and then updating the parameters using an optimizer, like Stochastic Gradient Descent (SGD) or Adam. This process is repeated for multiple epochs until the model achieves satisfactory performance, balancing accuracy and generalization. +Next, the dataset is split into training and testing subsets. Training data is used to update the model's parameters, while testing data evaluates its performance. During training, feed batches of input data through the network, calculate the prediction error or loss using a loss function (such as cross-entropy for classification tasks), and optimize the model's weights and biases using backpropagation. Backpropagation involves computing the gradient of the loss with respect to each parameter and then updating the parameters using an optimizer, like Stochastic Gradient Descent (SGD) or Adam. This process is repeated for multiple epochs until the model achieves satisfactory performance, balancing accuracy and generalization. ### Loss, gradients, epoch and backpropagation -Loss is a measure of how well a model’s predictions match the true labels of the data. It quantifies the difference between the predicted output and the actual output. The lower the loss, the better the model’s performance. In classification tasks, a common loss function is Cross-Entropy Loss, while Mean Squared Error (MSE) is often used for regression tasks. The goal of training is to minimize the loss, which indicates that the model’s predictions are getting closer to the actual labels. +Loss is a measure of how well a model's predictions match the true labels of the data. It quantifies the difference between the predicted output and the actual output. The lower the loss, the better the model's performance. In classification tasks, a common loss function is Cross-Entropy Loss, while Mean Squared Error (MSE) is often used for regression tasks. The goal of training is to minimize the loss, which indicates that the model's predictions are getting closer to the actual labels. -Gradients represent the rate of change of the loss with respect to each of the model’s parameters (weights and biases). They are used to update the model’s parameters in the direction that reduces the loss. Gradients are calculated during the backpropagation step, where the loss is propagated backward through the network to compute how each parameter contributes to the overall loss. Optimizers like SGD or Adam use these gradients to adjust the parameters, effectively “teaching” the model to improve its predictions. +Gradients represent the rate of change of the loss with respect to each of the model's parameters (weights and biases). They are used to update the model's parameters in the direction that reduces the loss. Gradients are calculated during the backpropagation step, where the loss is propagated backward through the network to compute how each parameter contributes to the overall loss. Optimizers like SGD or Adam use these gradients to adjust the parameters, effectively “teaching” the model to improve its predictions. An epoch refers to one complete pass through the entire training dataset. During each epoch, the model sees every data point once and updates its parameters accordingly. Multiple epochs are typically required to train a model effectively because, during each epoch, the model learns and fine-tunes its parameters based on the data it processes. The number of epochs is a hyperparameter that you set before training, and increasing it can improve the model’s performance, but too many epochs may lead to overfitting, where the model performs well on training data but poorly on new, unseen data. diff --git a/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/mobile-app.md b/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/mobile-app.md new file mode 100644 index 000000000..fe897f817 --- /dev/null +++ b/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/mobile-app.md @@ -0,0 +1,243 @@ +--- +# User change +title: "Update the Android application" + +weight: 14 + +layout: "learningpathall" +--- + +You can now use the optimized model in the Android application you developed earlier. + +Start by modifying the `activity_main.xml` by adding a `CheckBox` to use the optimized model: + +```XML + + +``` + +Copy the optimized model to the `assets` folder of the Android project. + +Replace the `MainActivity.kt` by the following code: + +```Kotlin +package com.arm.armpytorchmnistinference + +import android.graphics.Bitmap +import android.graphics.BitmapFactory +import android.os.Bundle +import android.widget.Button +import android.widget.CheckBox +import android.widget.ImageView +import android.widget.TextView +import androidx.activity.enableEdgeToEdge +import androidx.appcompat.app.AppCompatActivity +import org.pytorch.IValue +import org.pytorch.Module +import org.pytorch.Tensor +import java.io.File +import java.io.FileOutputStream +import java.io.IOException +import java.io.InputStream +import kotlin.random.Random +import kotlin.system.measureNanoTime + +class MainActivity : AppCompatActivity() { + private lateinit var imageView: ImageView + private lateinit var trueLabel: TextView + private lateinit var selectImageButton: Button + private lateinit var runInferenceButton: Button + private lateinit var predictedLabel: TextView + private lateinit var inferenceTime: TextView + private lateinit var model: Module + private lateinit var checkboxOptimizedModel: CheckBox // Add CheckBox for switching models + private var currentBitmap: Bitmap? = null + private var currentTrueLabel: Int? = null + + override fun onCreate(savedInstanceState: Bundle?) { + super.onCreate(savedInstanceState) + enableEdgeToEdge() + setContentView(R.layout.activity_main) + + // Initialize UI elements + imageView = findViewById(R.id.imageView) + trueLabel = findViewById(R.id.trueLabel) + selectImageButton = findViewById(R.id.selectImageButton) + runInferenceButton = findViewById(R.id.runInferenceButton) + predictedLabel = findViewById(R.id.predictedLabel) + inferenceTime = findViewById(R.id.inferenceTime) + checkboxOptimizedModel = findViewById(R.id.checkboxOptimizedModel) + + // Initially load the appropriate model based on the CheckBox state + loadModel() + + // Set up button click listener for selecting random image + selectImageButton.setOnClickListener { + selectRandomImageFromAssets() + } + + // Set up button click listener for running inference + runInferenceButton.setOnClickListener { + currentBitmap?.let { bitmap -> + runInference(bitmap) + } + } + + // Set up listener for checkbox to switch models dynamically + checkboxOptimizedModel.setOnCheckedChangeListener { _, _ -> + loadModel() // Reload the model when checkbox state changes + } + } + + // Load the appropriate model based on the CheckBox state + private fun loadModel() { + try { + model = if (checkboxOptimizedModel.isChecked) { + // Load the optimized model + Module.load(assetFilePath("optimized_model.ptl")) + } else { + // Load the original model + Module.load(assetFilePath("model.pth")) + } + } catch (e: IOException) { + e.printStackTrace() + trueLabel.text = "Error loading model" + } + } + + private fun selectRandomImageFromAssets() { + try { + // Get list of files in the mnist_bitmaps folder + val assetManager = assets + val files = assetManager.list("mnist_bitmaps") ?: arrayOf() + + if (files.isEmpty()) { + trueLabel.text = "No images found in assets/mnist_bitmaps" + return + } + + // Select a random file from the list + val randomFile = files[Random.nextInt(files.size)] + val inputStream: InputStream = assetManager.open("mnist_bitmaps/$randomFile") + val bitmap = BitmapFactory.decodeStream(inputStream) + + // Extract the true label from the filename (e.g., 07_00.png -> true label is 7) + currentTrueLabel = randomFile.split("_")[0].toInt() + + // Display the image and its true label + imageView.setImageBitmap(bitmap) + trueLabel.text = "True Label: $currentTrueLabel" + + // Set the current bitmap for inference + currentBitmap = bitmap + } catch (e: IOException) { + e.printStackTrace() + trueLabel.text = "Error loading image from assets" + } + } + + // Method to convert a grayscale bitmap to a float array and create a tensor with shape [1, 1, 28, 28] + private fun createTensorFromBitmap(bitmap: Bitmap): Tensor { + // Ensure the bitmap is in the correct format (grayscale) and dimensions [28, 28] + if (bitmap.width != 28 || bitmap.height != 28) { + throw IllegalArgumentException("Expected bitmap of size [28, 28], but got [${bitmap.width}, ${bitmap.height}]") + } + + // Convert the grayscale bitmap to a float array + val width = bitmap.width + val height = bitmap.height + val floatArray = FloatArray(width * height) + val pixels = IntArray(width * height) + bitmap.getPixels(pixels, 0, width, 0, 0, width, height) + + for (i in pixels.indices) { + // Normalize pixel values to [0, 1] range, assuming the grayscale image stores values in the R channel + floatArray[i] = (pixels[i] and 0xFF) / 255.0f + } + + // Create a tensor with shape [1, 1, 28, 28] (batch size, channels, height, width) + return Tensor.fromBlob(floatArray, longArrayOf(1, 1, height.toLong(), width.toLong())) + } + + private fun runInference(bitmap: Bitmap, N: Int = 100) { + // Convert bitmap to a float array and create a tensor with shape [1, 1, 28, 28] + val inputTensor = createTensorFromBitmap(bitmap) + + // Run inference and measure time + val inferenceTimeMicros = measureTimeMicros { + var maxIndex = 0 + var scores: FloatArray = FloatArray(10) + + // Run inference N times + for (i in 1..N) { + // Forward pass through the model + val outputTensor = model.forward(IValue.from(inputTensor)).toTensor() + scores = outputTensor.dataAsFloatArray + } + + // Get the index of the class with the highest score + maxIndex = scores.indices.maxByOrNull { scores[it] } ?: -1 + + predictedLabel.text = "Predicted Label: $maxIndex" + } + + // Update inference time TextView in microseconds + inferenceTime.text = "Inference Time: $inferenceTimeMicros µs" + } + + // Method to measure execution time in microseconds + private inline fun measureTimeMicros(block: () -> Unit): Long { + val time = measureNanoTime(block) + return time / 1000 // Convert nanoseconds to microseconds + } + + // Helper function to get the file path from assets + private fun assetFilePath(assetName: String): String { + val file = File(filesDir, assetName) + assets.open(assetName).use { inputStream -> + FileOutputStream(file).use { outputStream -> + val buffer = ByteArray(4 * 1024) + var read: Int + while (inputStream.read(buffer).also { read = it } != -1) { + outputStream.write(buffer, 0, read) + } + outputStream.flush() + } + } + return file.absolutePath + } +} +``` + +The updated version of the Android application includes modifications to the Android Activity to dynamically load the model based on the state of the `CheckBox`. + +When the `CheckBox` is selected, the app loads the optimized model, which is quantized and fused for improved performance. + +If the `CheckBox` is not selected, the app loads the original model. + +After the model is loaded, the inference is run. To better estimate the execution time, the `runInference()` method executes the inference 100 times in a loop. This provides a more reliable measure of the average inference time by smoothing out any inconsistencies from single executions. + +The results for a run on a physical device are shown below. These results indicate that, on average, the optimized model reduced the inference time to about 65% of the original model's execution time, showing a significant improvement in performance. + +This optimization showcases the benefits of quantization and layer fusion for mobile inference, and there is further potential for enhancement by enabling hardware acceleration on supported devices. + +This would allow the model to take full advantage of the device's computational capabilities, potentially reducing the inference time even more. + +![fig](Figures/07.jpg) + +![fig](Figures/08.jpg) + +# What have you learned? + +You have successfully optimized a neural network model for mobile inference using quantization and layer fusion. + +Quantization and layer fusion removed unnecessary elements such as dropout layers during inference. + +By running multiple iterations of the inference process, you learned that the optimized model significantly reduced the average inference time to around 65% of the original time. + +You also learned that there is potential for further performance improvements by leveraging hardware acceleration. \ No newline at end of file diff --git a/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/model-opt.md b/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/model-opt.md new file mode 100644 index 000000000..dc5b8556e --- /dev/null +++ b/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/model-opt.md @@ -0,0 +1,234 @@ +--- +# User change +title: "Create an optimized PyTorch model for MNIST" + +weight: 12 + +layout: "learningpathall" +--- + +You can create and train an optimized feedforward neural network to classify handwritten digits from the MNIST dataset. As a reminder, the dataset contains 70,000 images, comprising 60,000 training and 10,000 testing images, of handwritten numerals (0-9), each with dimensions of 28x28 pixels. + +This time you will introduce several changes to enable model quantization and fusing. + +# Model architecture + +Start by creating a new notebook named `pytorch-digits-model-optimisations.ipynb`. + +Then define the model architecture using the code below. You can also find the source code on [GitHub](https://github.com/dawidborycki/Arm.PyTorch.MNIST.Inference.Python) + +```python +import torch +from torch import nn +from torchsummary import summary + +class_names = range(10) + +class NeuralNetwork(nn.Module): + def __init__(self, use_dropout=True): + super(NeuralNetwork, self).__init__() + self.use_dropout = use_dropout + self.flatten = nn.Flatten() + + self.linear1 = nn.Linear(28*28, 96) + self.relu1 = nn.ReLU() + self.dropout1 = nn.Dropout(0.2) if use_dropout else nn.Identity() + + self.linear2 = nn.Linear(96, 256) + self.relu2 = nn.ReLU() + self.dropout2 = nn.Dropout(0.2) if use_dropout else nn.Identity() + + self.linear3 = nn.Linear(256, len(class_names)) + # Softmax is removed from the model + + def forward(self, x): + x = self.flatten(x) + x = self.linear1(x) + x = self.relu1(x) + x = self.dropout1(x) + x = self.linear2(x) + x = self.relu2(x) + x = self.dropout2(x) + x = self.linear3(x) + return x # Outputs raw logits +``` + +This code defines a neural network in PyTorch for digit classification, consisting of three linear layers with ReLU activations and optional dropout layers for regularization. The network first flattens the input (a 28x28 image) and passes it through two linear layers, each followed by a ReLU activation and a dropout layer (if enabled). The final layer produces raw logits as the output. Notably, the softmax layer has been removed to enable quantization and layer fusion during model optimization, allowing better performance when deploying the model on mobile or edge devices. + +The output is left as logits, and the softmax function can be applied during post-processing, particularly during inference. + +This model includes dropout layers, which are used during training to randomly set a portion of the neurons to zero in order to prevent overfitting and improve generalization. + +The `use_dropout` parameter allows you to enable or disable dropout, with the option to bypass dropout by replacing it with an `nn.Identity` layer when set to `False`, which is typically done during inference or quantization for more consistent behavior. + +Add the following lines to display the model architecture: + +```Python +model = NeuralNetwork() + +summary(model, (1, 28, 28)) +``` + +After running the code, you see the following output: + +```output +---------------------------------------------------------------- + Layer (type) Output Shape Param # +================================================================ + Flatten-1 [-1, 784] 0 + Linear-2 [-1, 96] 75,360 + ReLU-3 [-1, 96] 0 + Dropout-4 [-1, 96] 0 + Linear-5 [-1, 256] 24,832 + ReLU-6 [-1, 256] 0 + Dropout-7 [-1, 256] 0 + Linear-8 [-1, 10] 2,570 +================================================================ +Total params: 102,762 +Trainable params: 102,762 +Non-trainable params: 0 +---------------------------------------------------------------- +Input size (MB): 0.00 +Forward/backward pass size (MB): 0.01 +Params size (MB): 0.39 +Estimated Total Size (MB): 0.41 +---------------------------------------------------------------- +``` + +The output shows the structure of the neural network, including the layers, their output shapes, and the number of parameters. + +* The network starts with a Flatten layer, which reshapes the input from [1, 28, 28] to [1, 784] without adding any parameters. +* This is followed by two Linear (fully connected) layers with ReLU activations and optional Dropout layers in between, contributing to the parameter count. +* The first linear layer (from 784 to 96 units) has 75,360 parameters, while the second (from 96 to 256 units) has 24,832 parameters. +* The final linear layer, which outputs raw logits for the 10 classes, has 2,570 parameters. +* The total number of trainable parameters in the model is 102,762, with no non-trainable parameters. + +# Training the model + +Now add the data loading, train, and test loops to actually train the model. This proceeds exactly the same as in the original model: + +``` +from torchvision import transforms, datasets +from torch.utils.data import DataLoader + +# Training data +training_data = datasets.MNIST( + root="data", + train=True, + download=True, + transform=transforms.ToTensor() +) + +# Test data +test_data = datasets.MNIST( + root="data", + train=False, + download=True, + transform=transforms.ToTensor() +) + +# Dataloaders +batch_size = 32 + +train_dataloader = DataLoader(training_data, batch_size=batch_size) +test_dataloader = DataLoader(test_data, batch_size=batch_size) + +learning_rate = 1e-3; + +loss_fn = nn.CrossEntropyLoss() +optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate) + +def train_loop(dataloader, model, loss_fn, optimizer): + size = len(dataloader.dataset) + for batch, (x, y) in enumerate(dataloader): + # Compute prediction and loss + pred = model(x) + loss = loss_fn(pred, y) + + # Backpropagation + optimizer.zero_grad() + loss.backward() + optimizer.step() + + +def test_loop(dataloader, model, loss_fn): + size = len(dataloader.dataset) + num_batches = len(dataloader) + test_loss, correct = 0, 0 + + with torch.no_grad(): + for x, y in dataloader: + pred = model(x) + test_loss += loss_fn(pred, y).item() + correct += (pred.argmax(1) == y).type(torch.float).sum().item() + + test_loss /= num_batches + correct /= size + + print(f"Accuracy: {(100*correct):>0.1f}%, Avg loss: {test_loss:>8f} \n") + +epochs = 10 + +for t in range(epochs): + print(f"Epoch {t+1}:") + train_loop(train_dataloader, model, loss_fn, optimizer) + test_loop(test_dataloader, model, loss_fn) +``` + +You begin by preparing the MNIST dataset for training and testing our neural network model. + +Using the torchvision library, you download the MNIST dataset and apply a transformation to convert the images into tensors, making them suitable for input into the model. + +Next, create two data loaders: one for the training set and one for the test set, each configured with a batch size of 32. These data loaders allow you to easily feed batches of images into the model during training and testing. + +Next, define a training loop, which is the core of the model’s learning process. For each batch of images and labels, the model generates predictions, and you calculate the cross-entropy loss to measure how far off the predictions are from the true labels. + +The Adam optimizer is used to perform backpropagation, updating the model's weights to reduce this error. The process repeats for every batch in the training dataset, gradually improving model accuracy over time. + +To ensure the model is learning effectively, you also define a testing loop. + +Here, the model is evaluated on a separate set of test images that it hasn't seen during training. You calculate both the average loss and the accuracy of the predictions, giving a clear sense of how well the model is performing. Importantly, this evaluation is done without updating the model's weights, as the goal is simply to measure its performance. + +Finally, run the training and testing loops over the course of 10 epochs. With each epoch, the model trains on the full training dataset, and afterward, you test it to monitor its progress. By the end of the process, the model has learned to classify the MNIST digits with a high degree of accuracy, as reflected in the final test results. + +This setup efficiently trains and evaluates the model for digit classification, providing feedback after each epoch on accuracy and loss. + +After running the code you will see the following output: + +```output +Epoch 1: +Accuracy: 94.0%, Avg loss: 0.196315 + +Epoch 2: +Accuracy: 95.3%, Avg loss: 0.155560 + +Epoch 3: +Accuracy: 95.9%, Avg loss: 0.138764 + +Epoch 4: +Accuracy: 95.4%, Avg loss: 0.156163 + +Epoch 5: +Accuracy: 95.5%, Avg loss: 0.163152 + +Epoch 6: +Accuracy: 96.3%, Avg loss: 0.129509 + +Epoch 7: +Accuracy: 96.8%, Avg loss: 0.124872 + +Epoch 8: +Accuracy: 96.6%, Avg loss: 0.127252 + +Epoch 9: +Accuracy: 96.4%, Avg loss: 0.134298 + +Epoch 10: +Accuracy: 96.5%, Avg loss: 0.137004 +``` + +The above shows a similar accuracy as the original model. + +You now have the trained model with the modified architecture. + +In the next step you will optimize it for mobile inference. \ No newline at end of file diff --git a/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/model.md b/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/model.md index abfc9f117..1c03a3e1f 100644 --- a/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/model.md +++ b/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/model.md @@ -7,7 +7,7 @@ weight: 3 layout: "learningpathall" --- -You can create and train a feedforward neural network to classify handwritten digits from the MNIST dataset. This dataset contains 70,000 images, comprising 60,000 training and 10,000 testing images, of handwritten numerals (0-9), each with dimensions of 28x28 pixels. Some representative MNIST digits with their corresponding labels are shown below. +You can create and train a feedforward neural network to classify handwritten digits from the MNIST dataset. This dataset contains 70,000 images, comprised of 60,000 training images and 10,000 testing images, of handwritten numerals (0-9), each with dimensions of 28x28 pixels. Some representative MNIST digits with their corresponding labels are shown below. ![img3](Figures/3.png) @@ -60,7 +60,7 @@ class NeuralNetwork(nn.Module): To build the neural network in PyTorch, define a class that inherits from PyTorch’s nn.Module. This approach is similar to TensorFlow’s subclassing API. In this case, define a class named NeuralNetwork, which consists of two main components: -1. **__init__** method +1. __init__ method This method serves as the constructor for the class. @@ -75,7 +75,7 @@ The network consists of: * Another Dropout layer, that removes 20% of the nodes. * A final Linear layer, with 10 nodes (matching the number of classes in the dataset), followed by a Softmax activation function that outputs class probabilities. -2. **forward** method +2. forward method This method defines the forward pass of the network. It takes an input tensor x, flattens it using self.flatten, and then passes it through the defined sequential stack of layers (self.linear_stack). @@ -99,14 +99,14 @@ You will see a detailed summary of the NeuralNetwork model’s architecture, inc The summary lists each layer of the network sequentially, including: -* The Flatten layer, which reshapes the 28x28 input images into a 784-element vector. -* The Linear layers with 96 and 256 nodes, respectively, along with the activation functions (Tanh and Sigmoid) applied after each linear transformation. -* The Dropout layers that randomly-deactivate 20% of the neurons in the respective layers. -* The final Linear layer with 10 nodes, corresponding to the output probabilities for the 10 digit classes, followed by the Softmax function. +* The flatten layer, which reshapes the 28x28 input images into a 784-element vector. +* The linear layers with 96 and 256 nodes, respectively, along with the activation functions (Tanh and Sigmoid) applied after each linear transformation. +* The dropout layers that randomly-deactivate 20% of the neurons in the respective layers. +* The final linear layer with 10 nodes, corresponding to the output probabilities for the 10 digit classes, followed by the softmax function. 2. Input and Output Shapes -For each layer, the summary shows the shape of the input and output tensors, helping to trace how the data flows through the network. For example, the input shape starts as (1, 28, 28) for the image, which gets flattened to (1, 784) after the Flatten layer. +For each layer, the summary shows the shape of the input and output tensors, helping to trace how the data flows through the network. For example, the input shape starts as (1, 28, 28) for the image, which gets flattened to (1, 784) after the flatten layer. 3. The summary @@ -129,7 +129,7 @@ The output is still a probability distribution over the 10 digit classes (0-9), Technically, the code will run without errors as long as you provide it with an input image of the correct dimensions, which is 28x28 pixels. The model can accept input, pass it through the layers, and return a prediction - a vector of 10 probabilities. However, the results are not useful until the model is trained. -# What you have learned so far +# What have you learned so far? You have successfully defined and initialized a feedforward neural network using PyTorch. diff --git a/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/optimisation.md b/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/optimisation.md new file mode 100644 index 000000000..06778bf47 --- /dev/null +++ b/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/optimisation.md @@ -0,0 +1,65 @@ +--- +# User change +title: "Run optimization" + +weight: 13 + +layout: "learningpathall" +--- + +To optimize the model use the `pytorch-digits-model-optimisations.ipynb` to add the following lines: + +```python +from torch.utils.mobile_optimizer import optimize_for_mobile + +# Instantiate the model without Dropout layers +model_for_quantization = NeuralNetwork(use_dropout=False) + +# Load the trained state_dict (excluding Dropout parameters) +model_for_quantization.load_state_dict(model.state_dict()) +model_for_quantization.eval() + +# Define quantization configuration +model.qconfig = torch.quantization.get_default_qconfig('qnnpack') + +# Fuse modules +modules_to_fuse = [['linear1', 'relu1'], ['linear2', 'relu2']] +torch.quantization.fuse_modules(model_for_quantization, modules_to_fuse, inplace=True) + +# Prepare for static quantization +torch.quantization.prepare(model_for_quantization, inplace=True) + +# Calibrate the model +with torch.no_grad(): + for inputs, _ in train_dataloader: + model_for_quantization(inputs) + +# Convert to quantized model +torch.quantization.convert(model, inplace=True) + +# Trace and optimize the quantized model +example_input = torch.rand(1, 1, 28, 28) +traced_quantized_model = torch.jit.trace(model, example_input) +optimized_model = optimize_for_mobile(traced_quantized_model) + +# Save the optimized model +optimized_model._save_for_lite_interpreter("optimized_model.ptl") +``` + +In this code, the neural network model is being prepared for optimization and quantization to make it more suitable for mobile deployment. + +First, the model is instantiated without Dropout layers by setting `use_dropout=False`, as dropout is typically disabled during inference. The model's trained weights are then loaded using the `load_state_dict()` function, ensuring that it retains the knowledge learned during training. The model is set to evaluation mode with `eval()` to prepare it for inference. + +Next, the quantization process is configured. + +A quantization configuration is applied using the `qnnpack` backend, which is designed for efficient quantization on mobile devices. Certain layers of the model, specifically the linear layers and their corresponding activation functions (ReLU), are fused using `torch.quantization.fuse_modules()`. This fusion reduces the computational overhead by combining operations, a common optimization technique. + +After fusing the layers, the model is prepared for static quantization with `torch.quantization.prepare()`, which involves calibrating the model on the training data to collect statistics needed for quantization. The calibration phase runs the model on some training data without updating the weights. + +Once calibration is complete, the model is converted to a quantized version using `torch.quantization.convert()`. The quantized model is then traced with `torch.jit.trace()`, which captures the model’s computational graph. + +Finally, the traced model is optimized for mobile using `optimize_for_mobile()`, further refining it for performance on mobile devices. + +The optimized model is saved in a format suitable for the PyTorch Lite Interpreter for efficient deployment on mobile platforms. + +The result is an optimized and quantized model stored as `"optimized_model.ptl"`, ready for deployment. \ No newline at end of file diff --git a/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/prepare-data.md b/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/prepare-data.md new file mode 100644 index 000000000..c16cb4c20 --- /dev/null +++ b/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/prepare-data.md @@ -0,0 +1,81 @@ +--- +# User change +title: "Prepare Test Data" + +weight: 9 + +layout: "learningpathall" +--- + +In this section you will add the pre-trained model and copy the bitmap image data to the Android project. + +## Model + +To add the model, create a folder named `assets` in the `app/src/main` folder. + +Copy the pre-trained model you created in the previous steps, `model.pth` to the `assets` folder. + +The model is also available in the [GitHub repository](https://github.com/dawidborycki/Arm.PyTorch.MNIST.Inference.git) if you need to copy it. + +## Image data + +The data preparation script is shown below: + +```Python +from torchvision import datasets, transforms +from PIL import Image +import os + +# Constants +NUM_DIGITS = 10 # Number of unique digits in MNIST (0-9) +EXAMPLES_PER_DIGIT = 2 # Number of examples per digit + +# Define the transformation to convert the image to a tensor +transform = transforms.Compose([transforms.ToTensor()]) + +# Load the MNIST test dataset +test_data = datasets.MNIST( + root="data", + train=False, + download=True, + transform=transform +) + +# Create a directory to save the bitmaps +os.makedirs("mnist_bitmaps", exist_ok=True) + +# Dictionary to keep track of collected examples per digit +collected_examples = {digit: 0 for digit in range(NUM_DIGITS)} + +# Loop through the dataset and collect the required number of images +for i, (image, label) in enumerate(test_data): + if collected_examples[label] < EXAMPLES_PER_DIGIT: + # Convert tensor to PIL image + pil_image = transforms.ToPILImage()(image) + # Create the filename with zero-padding + filename = f"mnist_bitmaps/{label:02d}_{collected_examples[label]:02d}.png" + # Save the image as PNG + pil_image.save(filename) + print(f"Saved: {filename}") + + # Update the count for the current label + collected_examples[label] += 1 + + # Break the loop if all required examples are collected + if all(count == EXAMPLES_PER_DIGIT for count in collected_examples.values()): + break +``` + +The above code processes the MNIST test dataset to generate and save bitmap images for digit classification. + +It defines constants for the number of unique digits (0-9) and the number of examples to collect per digit. The dataset is loaded using `torchvision.datasets` with a transformation to convert images to tensors. + +A directory named `mnist_bitmaps` is created to store the images. A dictionary tracks the number of collected examples for each digit. The code iterates through the dataset, converting each image tensor back to a PIL image, and saves two examples of each digit in the format `digit_index_example_index.png`. + +The loop breaks once the specified number of examples per digit is saved, ensuring that exactly 20 images (2 per digit) are generated and stored in the specified directory. + +For your convenience the data is included in the [GitHub repository](https://github.com/dawidborycki/Arm.PyTorch.MNIST.Inference.git) + +Copy the `mnist_bitmaps` folder to the `assets` folder. + +Once you have the `model.pth` and the `mnist_bitmaps` folder in the `assets` folder continue to the next step to run the Android application. \ No newline at end of file diff --git a/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/user-interface.md b/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/user-interface.md new file mode 100644 index 000000000..bcf84520b --- /dev/null +++ b/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/user-interface.md @@ -0,0 +1,332 @@ +--- +# User change +title: "Create an Android application" + +weight: 8 + +layout: "learningpathall" +--- + +In this section you will create an Android application to run digit classification. + +The application randomly loads a selected image containing a handwritten digit and its true label. + +The application runs an inference on the image and predicts the digit value. + +## Create an Android project + +Start by creating a project: + +1. Open Android Studio and create a new project with an “Empty Views Activity.” + +2. Set the project name to **ArmPyTorchMNISTInference**, set the package name to: **com.arm.armpytorchmnistinference**, select **Kotlin** as the language, and set the minimum SDK to **API 27 ("Oreo" Android 8.1)**. + +Set the API to Android 8.1 (API level 27) because this version introduced NNAPI, providing a standard interface for running computationally intensive machine learning models on Android devices. + +Devices with hardware accelerators can leverage NNAPI to offload ML tasks to specialized hardware, such as NPUs (Neural Processing Units), DSPs (Digital Signal Processors), or GPUs (Graphics Processing Units). + +## User interface design + +The user interface design contains the following: + +- A header. +- `ImageView` and `TextView` sections to display the image and its true label. +- A button to load the image. +- A button to run inference. +- Two `TextView` controls to display the predicted label and inference time. + +Use the Android Studio editor to replace the contents of `activity_main.xml`, located in `src/main/res/layout` with the following code: + +```XML + + + + + + + + + + + + + +