Shakespearean Text Generation with GRU-based RNN

(using RNN)

Overview

This project focuses on training a Recurrent Neural Network (RNN) to generate text in the distinctive style of William Shakespeare. The dataset used is "The Complete Works of William Shakespeare," which can be found here. The RNN is implemented using TensorFlow, and the model architecture is based on a Gated Recurrent Unit (GRU) with an embedding layer.

Libraries Used

• NumPy
• Pandas
• Matplotlib
• TensorFlow

Project Steps

Step 1: Reading The Data

The chosen dataset comprises all of Shakespeare's works, providing a large corpus of text with a distinctive style. This dataset is preferred for its size and unique characteristics.

Step 2: Text Processing

Text Vectorization

To input text into the neural network, numeric indices are assigned to each character. Two dictionaries are created to map between numeric indices and characters.

Step 3: Creating Batches

Training Sequences

Training sequences are generated by shifting the text sequence by one character. For instance:

• Sequence In: "Hello my nam"
• Sequence Out: "ello my name"

Generating Training Batches

Batches are created by shuffling the sequences into a random order to prevent overfitting to specific sections of the text.

Step 4: Creating the Model

The model is built using a GRU-based architecture with additional features, including an embedding layer. The embedding layer serves as the input layer, mapping character indices to vectors. The model uses sparse categorical crossentropy as the loss function.

Model Architecture

Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
embedding (Embedding)        (128, None, 64)           5376      
_________________________________________________________________
gru (GRU)                    (128, None, 1026)         3361176   
_________________________________________________________________
dense (Dense)                (128, None, 84)           86268     
=================================================================
Total params: 3,452,820
Trainable params: 3,452,820
Non-trainable params: 0

Step 5: Training the Model

A check is performed to ensure that the model predicts random characters without any training by passing in a batch.

Step 6: Generating Text

A custom function is created to generate text from a given prompt using the trained GRU-based RNN model. The model is configured to expect a batch size of 1 for text generation.

Usage

To run the project, follow these steps:

1. Download the dataset from here.
2. Execute the provided Jupyter Notebook or Python script.

Feel free to experiment with different parameters and model architectures to enhance text generation capabilities.

Enjoy creating Shakespearean-like text with your trained RNN!