README.md

Code Implementation Notes from Undergraduate Studies

This repository contains various code implementations and notes on topics I studied during my undergraduate years. The sections are organized by subject, focusing on different techniques and frameworks in deep learning, machine learning, and algorithm design.

Contents

Advanced Transformer

This section includes code implementations and notes on advanced transformer architectures used in various applications.

• Base_Transformer
  Basic transformer model implementation as a foundation for more complex variations.
  Paper: Attention Is All You Need

• Deformable_Transformer
  Implementation of deformable transformers for handling variable-length sequences and dynamic data structures.
  Paper: Deformable DETR: Deformable Transformers for End-to-End Object Detection

• Dense_Prediction_Transformer
  A transformer model focused on dense prediction tasks, such as image segmentation and object detection.
  Paper: DPT: Dense Prediction Transformer

• DINO
  Self-supervised learning based on transformers with no labeled data.
  Paper: Emerging Properties in Self-Supervised Vision Transformers

• Llama
  Code for the LLaMA model, a large language model optimized for language-based tasks.
  Paper: LLaMA: Open and Efficient Foundation Language Models

• Vision_Transformer
  Vision Transformer (ViT) implementation for image classification and vision-related tasks.
  Paper: An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

Agent

Code implementation for agent-based systems, where multiple agents interact with the environment to perform tasks.

Algorithm

This section contains algorithm templates and blog posts about various algorithm patterns.

DeepSpeed

Notes and code for using DeepSpeed for model training and optimization.

Fine-Tuning

Detailed notes on model fine-tuning techniques and strategies for various pre-trained models.

Machine Learning

Code implementations of machine learning algorithms, from basic models to more advanced techniques.

Ongoing

Future implementations and research notes will be added here.

Usage

To use the code in this repository:

1. Navigate to the relevant directory for the transformer model or algorithm you're interested in.
2. Each directory contains:
   • The code implementation for the specific model or algorithm.
   • Notes and documentation for understanding the code and its usage.
3. Follow the specific instructions in each directory's README for setup and execution details.

For model-specific setups, please refer to the individual subdirectories for more detailed instructions and usage examples.