Second-Place Winner, Berkeley's 2023 Annual Datathon for Social Good

DeepOculos: Generative AI For Eye Retina Disease Tracking

by Timothy Gao, Henry Hong, Marcus Koh, Ohm Rajpal, and Henry Lee

Slides: https://docs.google.com/presentation/d/1p5pmAzQGegYffPu98yMIs9NvyDUnuPysgHVNScweZCU/edit#slide=id.g1388be8e982_0_3

Generating Synthetic Retinal Eye Motion Videos with Deep Convolutional GAN

Original Retinal Eye Motion Video (data provided by C. Light Technologies):

512x512

Our DCGAN-generated Synthetic Eye Motion Video (with embedded pupil trace statistics):

64x64 (upscaled with cubic interpolation)

DCGAN Training Progress:

• First 1000 Epochs

• 2000 Epochs

• 3000 Epochs

Generating Eye Motion Trace Statistics Via Long Short-Term Memory Network (LSTM)

• We view horizontal and vertical location of pupil as independent time series and apply LSTM to generate future prediction for each
• LSTM-generated locations are fed into DCGAN for the pupil trace statistics

The Challenge: Use generative AI to create artificial retinal video sequences & eye motion traces given a disease state.

Implementation Details:

1. Used a four-layer LSTM network with dropout regularization after each LSTM layer and a single dense neuron output layer
2. Trained over 50 epochs, with rmsprop optimizer and mean_squared_error as loss function
3. Utilized first 80% of each time series as training data, last 20% as testing data
4. Used MinMaxScaler to normalize all pixel values to [-1, 1] domain
5. Used previous 60 frames as context for each sliding window

Technologies Used:

1. Tensorflow: To build LSTMs, GANs
2. Scikit-Learn: MinMaxScaler and Train Test Split
3. Keras: Deep learning API (Sequential Model and neural network layers)
4. Seaborn and Matplotlib: Data Visualization
5. NumPy: For calculations and array manipulation
6. Pandas: For processing CSV files