Awesome AI for Edge Computing

     

A curated list of awesome tools, frameworks, libraries, and resources for running AI models on edge devices, including smartphones, IoT devices, embedded systems, and hardware accelerators. Edge AI focuses on processing data locally on the device, reducing latency and enhancing privacy.

Contents

• Frameworks and Libraries
• Hardware and Accelerators
• Deployment Platforms
• Optimization Tools
• Applications
• Learning Resources
• Books
• Community
• Contributing
• License

Frameworks and Libraries

• TensorFlow Lite - A lightweight version of TensorFlow designed for mobile and embedded devices.
• ONNX Runtime - A cross-platform, high-performance scoring engine for running ONNX models on edge devices.
• PyTorch Mobile - A version of PyTorch tailored for mobile devices, enabling deployment of PyTorch models on iOS and Android.
• Apache TVM - An open-source deep learning compiler stack for running machine learning models on edge devices.
• Edge Impulse SDK - A toolkit for building, optimizing, and deploying machine learning models on edge devices.
• DeepC - A framework for deploying deep learning models on microcontrollers and edge devices with limited resources.

Hardware and Accelerators

• NVIDIA Jetson - A family of embedded AI computing platforms for edge devices, offering powerful GPU acceleration.
• Google Coral - Edge AI hardware by Google, featuring the Edge TPU for fast, efficient inference.
• Intel Movidius Neural Compute Stick - A USB-based neural compute accelerator for running AI models at the edge.
• Raspberry Pi - A popular, low-cost single-board computer that can run AI models locally with the help of libraries like TensorFlow Lite.
• Arduino Nano 33 BLE Sense - An Arduino board designed for AI and machine learning projects at the edge.
• Xilinx Edge AI - AI-enabled FPGAs for real-time processing on edge devices.

Deployment Platforms

• AWS IoT Greengrass - A service for running local compute, messaging, data caching, sync, and ML inference on edge devices.
• Azure IoT Edge - A platform by Microsoft for deploying cloud intelligence on local edge devices.
• Google Cloud IoT Edge - A service by Google for running AI inference on edge devices using TensorFlow Lite and Edge TPU.
• EdgeX Foundry - An open-source platform for building interoperable edge computing solutions.
• Balena - A platform for building, deploying, and managing containerized applications on edge devices.

Optimization Tools

• TensorFlow Model Optimization Toolkit - Tools for model pruning, quantization, and optimization to run efficiently on edge devices.
• ONNX Quantization - Tools for optimizing ONNX models through quantization for faster inference on edge hardware.
• NVIDIA TensorRT - A high-performance deep learning inference optimizer and runtime for NVIDIA GPUs, including Jetson devices.
• OctoML - An automated machine learning optimization platform for deploying efficient AI models on edge hardware.
• TinyML - A community and set of tools focused on running machine learning models on microcontrollers and other low-power devices.

Applications

• Computer Vision at the Edge - Real-time object detection, facial recognition, and image classification on edge devices using frameworks like TensorFlow Lite and PyTorch Mobile.
• Voice Assistants - Running wake word detection and natural language processing models on devices like Amazon Echo and Google Home using Edge AI.
• Smart Home Automation - Using AI at the edge for local processing of smart home devices, reducing the need for cloud processing.
• Healthcare Monitoring - Edge AI for wearable devices that monitor vital signs and detect anomalies without needing constant cloud connectivity.
• Autonomous Vehicles - Running AI models on edge devices in vehicles for real-time decision-making and navigation.

Learning Resources

• TinyML Course by HarvardX - A course focused on building machine learning models for microcontrollers and edge devices.
• Coursera: Edge AI - Courses on deploying AI models on edge devices.
• Google Coral Tutorials - Tutorials for running AI inference on Google Coral hardware.
• NVIDIA Jetson Tutorials - Getting started guides and tutorials for the NVIDIA Jetson platform.
• PyTorch Mobile Documentation - Official documentation for deploying PyTorch models on mobile devices.

Books

• TinyML: Machine Learning with TensorFlow Lite on Arduino and Ultra-Low-Power Microcontrollers by Pete Warden and Daniel Situnayake - A comprehensive guide to TinyML.
• AI at the Edge: Solving Real-World Problems with Edge Computing and Artificial Intelligence by Chris Partridge - A book focused on the applications of AI at the edge.
• Hands-On Edge Analytics with Azure IoT by Colin Dow - A practical guide to building edge AI solutions using Azure IoT.
• Deep Learning for Edge Computing by Mohan Kumar and Manimegalai - A book on deploying deep learning models on edge devices.
• Embedded Machine Learning by Floyd B. Adams - A book on using embedded systems for machine learning applications.

Community

• TinyML Foundation - A community focused on machine learning for tiny, low-power devices.
• Edge AI and Vision Alliance - A group dedicated to advancing the use of computer vision and AI at the edge.
• AI on the Edge Forum - A forum for discussing AI applications and deployments on edge devices.
• Reddit: r/TinyML - A subreddit for discussing TinyML and edge AI projects.
• NVIDIA Developer Forums - A community for discussing NVIDIA’s edge AI hardware and software.

Contribute

Contributions are welcome!

License