This repository mimics a client interaction with a server based machine learning service. Models are trained, saved and served to classify incoming images from a client machine. This interaction is carried out in a robust fashion using a choice of two message brokers: Apache Kafka and Google Pub/Sub.
The repository is separated into 3 sub-modules:
A simple multi-class CNN trainer using TensorFlow.
A unified message broker API, allowing both Google Pub/Sub and Apache Kafka to be used.
An example app combining both the ImageClassifier and Unified API to mimic client requests (images) and model server responses (predictions).
Individual README files are located within each sub-package directory.
The package has been made using Poetry, and can be installed via:
$ git clone https://github.com/Acusick1/ImageClassifier
$ cd ImageClassifier
$ pip -m install .
The message brokers used require some additional setup.
This project uses the Pub/Sub client library. A project and service account with the necessary project permissions are required, please follow the instructions in the documentation.
Authentication requires the environment variable below to be set:
GOOGLE_APPLICATION_CREDENTIALS=path/to/service_account_key.json
Finally, the PROJECT variable in UnifiedAPI/settings.py must be set to Pub/Sub project identifier that you created.
Follow the quickstart instructions to install Apache Kafka.
It is assumed you are running Kafka on localhost:9092. If this is not the case, change the KAFKA_HOST variable in UnifiedAPI/settings.py to the required Kafka server.
To validate the installation and message broker setup, run the test cases from the top level directory:
$ python -m unittest