Testing Playground for Azure AI Services

This repository is primarily used to test and evaluate techniques with Azure AI services for the purposes of learning and experimentation. The repository is organized into different directories, each of which contains a different set of experiments and tests.

This repository is not intended for production use, and the code contained within is not guaranteed to be secure or optimized. It is intended for educational purposes only.

Note

The playgrounds are primarily built with various languages including .NET, Python, and TypeScript. However, not every playground has an environment for each language. Libraries to interface with Azure services are available in each of these languages, and the code can be easily translated to them with studying the API surface layer and utilizing AI.

Pre-requisites

This repository contains a devcontainer configuration for Visual Studio Code. This configuration contains all the necessary tools and libraries to run the code in this repository. To use the devcontainer, you must have the following installed on your local machine:

• Install Visual Studio Code
• Install Docker Desktop
• Install Remote - Containers extension for Visual Studio Code

Note

If you are not planning on using the Dev Container, you must also install PowerShell Core, Azure CLI, .NET SDK, Nodejs, and Python on your local machine, as well as the necessary Python packages from the requirements.txt file.

Setting up the Azure AI Services

To setup an environment in Azure, simply run the Setup-Environment.ps1 script from the root of the project:

Important

Ensure that you have run the az login command to authenticate with Azure before running the script, and selected your subscription using az account set --subscription <SubscriptionId>.

.\Setup-Environment.ps1 -DeploymentName <DeploymentName> -Location <Location> -SkipInfrastructure $false

This script will deploy the necessary Azure services using the Azure Bicep template in the infra folder.

Once deployed, the script will create a .env file in the root of the project with the necessary environment variables to connect to the Azure services.

Playgrounds

Document Data Extraction

Azure AI Document Intelligence

• Pre-built Layout Markdown to Structured Object Extraction with GPT
  • Note: This playground has been matured into a sample - Using Azure AI Document Intelligence and Azure OpenAI GPT-3.5 Turbo to extract structured data from documents
• Pre-built Invoice Extraction with Custom Fields
  • Demonstrate how to use the Azure AI Document Intelligence service's pre-built invoice extraction model to extract fields from any invoice, including custom fields.

Azure OpenAI

• Document Data Extraction with PyMuPDF and Azure OpenAI GPT models
  • Demonstrate how to use PyMuPDF as an OCR tool in application code, and Azure OpenAI GPT models as the language model to extract structured data from documents.
• Document Data Extraction with Marker and Azure OpenAI GPT models
  • Demonstrate how to use Marker as an OCR tool in application code, and Azure OpenAI GPT models as the language model to extract structured data from documents.
• Document Data Extraction with Florence-2 and Azure OpenAI GPT models
  • Demonstrate how to use Florence-2 as an OCR tool in application code, and Azure OpenAI GPT models as the language model to extract structured data from documents.