This repository contains a demo of the pipeline described in the paper "An Affordance Detection Pipeline for Resource-Constrained Devices" by Apicella, T. et al.
A brief description of the method: the object detector crops the objects of interest present in the scene and the affordance detector segments the patches pixel-wise in three classes: Background (black), Grasp (blue) and No-grasp (green). Both object detector and affordance detector are lightweight models and have been trained on IIT-AFF Dataset.
The requirements to run the python code are the following:
- Python 3.6
- Tensorflow
- Keras
- Numpy
- OpenCV
- Keras segmentation
For additional details, see requirements.txt file.
The demo_object_detection_affordance.py runs the pipeline described in the paper on a mp4 video.
The object detector SavedModel format is in object_detector folder, while affordance detector weights and config files are available in affordance_detector folder.
If you find the code or pre-trained models useful, please cite the following paper:
An Affordance Detection Pipeline for Resource-Constrained Devices. , T. Apicella, A. Cavallaro, R. Berta, P. Gastaldo, and E. Ragusa. IEEE International Conference on Electronics, Circuits, and Systems (ICECS), 2021. DOI
@inproceedings{apicella2021affordance,
title={An Affordance Detection Pipeline for Resource-Constrained Devices},
author={Apicella, Tommaso and Cavallaro, Andrea and Berta, Riccardo and Gastaldo, Paolo and Bellotti, Francesco and Ragusa, Edoardo},
booktitle={2021 28th IEEE International Conference on Electronics, Circuits, and Systems (ICECS)},
pages={1--6},
organization={IEEE}
}