Image Captioning - Attend2u

Implementation of Attend to You: Personalized Image Captioning with Context Sequence Memory Networks

• Work in progress *

Requirements

• Python 3.5
• TensorFlow > 1.10
• Other requirements - check requirements.txt

Installation

• conda create -name attend2u python=3.5 pip
• conda activate attend2u
• pip install -r requirements.txt

Dataset

Download Instagram dataset.
    JSON file & Images

Save the files to ${project_root}/data

Example of /data directory structure

├── data
│   ├── caption_dataset
│   ├── hashtag_dataset
│   ├── images
│   ├── json
│   ├── resnet_pool5_features

Running Code

1. Download ResNet trained model
       Run scripts/download_pretrained_resnet_101.sh

2. Extract images features
       Run scripts/extract_features.sh

3. Configure the network
       Edit model parameters, training/evaluation parameters and data paths in configs.py

4. Train
       Run train.py
           python train.py

5. Evaluate
       Run eval.py
           python eval.py

---

Original code : https://github.com/cesc-park/attend2u

Coco evaluation tools for Python 3 : https://github.com/Illuminati91/pycocoevalcap