New Proposal: A CNN Builder Pattern Structure Using AlexNet as an Example

[Samyssmile]

Hello everyone 👋,

this proposal is related to: #43

I've been working on a simple and efficient way to define CNN architectures, and I'd love to get your feedback! Inspired by the Builder design pattern, I've crafted a clean and easy-to-read implementation of a Convolutional Neural Network (CNN) Builder using AlexNet as an example. Below is the code snippet to illustrate how the Builder structure is utilized:

AlexNet Architecture: https://en.wikipedia.org/wiki/AlexNet

Classifier alexNet = new Architect.Builder()
                .addLayer(new ConvLayer(11, 11, 3, 96, 4, 0)) // 1. Convolutional Layer
                .addLayer(new MaxPoolingLayer(3, 3, 2)) // 1. Max-Pooling Layer
                .addLayer(new ConvLayer(5, 5, 96, 256, 1, 2)) // 2. Convolutional Layer
                .addLayer(new MaxPoolingLayer(3, 3, 2)) // 2. Max-Pooling Layer
                .addLayer(new ConvLayer(3, 3, 256, 384, 1, 1)) // 3. Convolutional Layer
                .addLayer(new ConvLayer(3, 3, 384, 384, 1, 1)) // 4. Convolutional Layer
                .addLayer(new ConvLayer(3, 3, 384, 256, 1, 1)) // 5. Convolutional Layer
                .addLayer(new MaxPoolingLayer(3, 3, 2)) // 3. Max-Pooling Layer
                .addLayer(new FlattenLayer()) // Flattening Layer
                .addLayer(new DenseLayer(4096)) // 1. Fully Connected (Dense) Layer
                .addLayer(new DropOutLayer(0.5)) // 1. Dropout Layer
                .addLayer(new DenseLayer(4096)) // 2. Fully Connected (Dense) Layer
                .addLayer(new DropOutLayer(0.5)) // 2. Dropout Layer
                .addLayer(new DenseLayer(NUM_CLASSES)) // 3. Fully Connected (Dense) Layer
                .build();

                alexNet.train(epoch, optimizer)

Benefits of the Proposed Structure:

• Expressiveness: Intuitive and expressive syntax for defining layers and architectures.
• Extensibility: Easy to extend and add new types of layers or functionality without modifying existing code. The method can seamlessly adapt to various types of neural network architectures, including MultiLayer Perceptrons (MLPs).

Classifier mlp = new Architect.Builder()
    .addLayer(new DenseLayer(NUM_INPUTS)) // Input Layer
    .addLayer(new ActivationLayer(RELU))  // Activation function (e.g., ReLU)
    .addLayer(new DenseLayer(128))       // 1. Hidden Layer
    .addLayer(new ActivationLayer(RELU))  // Activation function
    .addLayer(new DenseLayer(64))        // 2. Hidden Layer
    .addLayer(new ActivationLayer(RELU))  // Activation function
    .addLayer(new DenseLayer(NUM_CLASSES)) // Output Layer
    .addLayer(new ActivationLayer(SOFTMAX)) // Output activation function (e.g., softmax)
    .build();

• Readability: Clear representation of the network structure, improving readability and maintainability.

Seeking Feedback:

• Usability: How do you find the usability of this approach? Does it make defining architectures easier for you?
• Improvements: Any ideas for improvements or additional features that could be added?
• Applicability For MLPs: How do you perceive this approach adapting for defining MLP architectures, given its simple yet expressive syntax?
• General Thoughts: Your general thoughts and whether you see potential issues in specific use-cases.

Indeed, the adaptability of this approach doesn’t just stop with CNNs like AlexNet. The straightforward layer addition syntax provides a clear and concise way to build various architectures, including MLPs, making it a versatile tool in neural network implementation. Your insights into its viability for MLPs and other architectures are particularly welcome.

Looking forward to all your insightful comments and suggestions. Let's work together and create a robust and intuitive API for building neural networks!