Maintenance notice

Note: I stopped maintaining this repository. For Virtual Data Center / Hub & Spoke guidance, check Enterprise-scale Landing Zones.

Automated VDC

This project contains a sample starter Virtual Datacenter (VDC), which follows a Hub & Spoke network topology and includes PaaS services with private networking. Two demo applications (one IaaS, one PaaS) are deployed into it.

TL;DR - Quickstart

If you want to get going without reading what's below, choose either:

• Setup option A, the fastest way to provision infrastructure if you already have Azure CLI and Terraform set up. You can use any shell.
• Setup option B, the fastest if you have nothing set up yet. All you need is a browser (Chrome or Edge) and an Azure subscription.

Architecture description

Infrastructure

This repo deploys the following components:

• A hub network with subnets for shared components (dmz, mgmt, etc)
• Azure Firewall used as Internet Access Gateway (egress, outbound FQDN whitelisting)
• Application Gateway as Web Application Firewall (WAF, HTTP ingress)
• A Management VM that is used as jump server to connect to other VM's
• A Managed Bastion as well
• A Point to Site (P2S VPN), with transitive access to PaaS services
• An IIS VM application deployed in a spoke network, with subnet segregation (app, data)
• An App Service web application integrated into another spoke network
  • Ingress through Private Endpoint
  • Egress through VNet integration (delegated subnet)
• Several PaaS services connected as Private Endpoints

Identify flow

Private networking provides some isolation from uninvited guests. However, a zero trust 'assume breach' approach uses multiple methods of isolation. This is why identity is called the the new perimeter. With Azure Active Directory Authentication, most application level communication can be locked down and access controlled through RBAC. This is done in the following places:

1. App Service uses Service Principal & RBAC to access Container Registry
2. User AAD auth (SSO with MFA) to App Service web app
3. App Service web app uses MSI to access SQL Database (using least privilege database roles, see grant_database_access.ps1)
4. User AAD auth (SSO with MFA) on Point-to-Site VPN
5. User AAD auth to VM's (RDP, VM MSI & AADLoginForWindows extension)
6. SQL Database tools (Azure Data Studio, SQL Server Management Studio) use AAD Autnentication with MFA
7. Azure DevOps (inc. Terraform) access to ARM using Service Principal

Deployment automation

The diagram conveys the release pipeline, with end-to-end orchestration by Azure Pipelines (YAML). The pipeline provisions infrastructure, and deploys 2 applications:

• An ASP.NET Core app deployed on PaaS App Service, SQl Database & more
• An ASP.NET Framework application deployed on IaaS VM's

The high-level pipeline steps are:

1. Create Pipeline environment using multi-stage YAML pipeline
2. Infrastructure provisioning with Terraform
   This diagram only shows resources (App Service, SQL Database & VM's) that participate in downstream deployments. Many more resources are created that are not displayed.
3. Provision SQL Database
4. Provision App Service (with SQL DB connection string)
5. App Service pulls configured ASP.NET Core app container running offline from database
6. Provision Virtual Machines
7. Import database (PowerShell with Azure CLI)
8. SQL Database pulls bacpac image
9. Swap deployment slots (PowerShell with Azure CLI)
   ASP.NET Core app now uses live database
10. Deploy ASP.NET Framework application

Provisioning

To get started you need Git, Terraform (to get that I use tfenv on Linux & macOS and chocolatey on Windows) and Azure CLI. Make sure you have the latest version of Azure CLI. This requires some tailored work on Linux (see http://aka.ms/azure-cli) e.g. for Debian/Ubuntu:
curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash

Alternatively, you can create a Visual Studio Codespace with this repo, using this link

Make sure you clean up, this creates quite a number of resources (see disclaimer).

Option A: Local Terraform

Use this option if you're using bash, zsh and/or don't have PowerShell Core.

1. Clone repository:
   git clone https://github.com/geekzter/azure-vdc.git

2. Change to the terraform directrory
   cd azure-vdc/terraform

3. Login into Azure with Azure CLI:
   az login

4. This also authenticates the Terraform azurerm provider when working interactively. Optionally, you can select the subscription to target:
   az account set --subscription 00000000-0000-0000-0000-000000000000
ARM_SUBSCRIPTION_ID=$(az account show --query id -o tsv) (bash, zsh)
   $env:ARM_SUBSCRIPTION_ID=$(az account show --query id -o tsv) (pwsh)

5. You can then provision resources by first initializing Terraform:
   terraform init

6. And then running:
   terraform apply

7. When you want to destroy resources, run:
   terraform destroy

The default configuration will work with any shell. Additional features may require PowerShell and one of the options below.

Option B: Visual Studio Codespace

This will use Visual Studio Codespaces as the environment to provision from. A Codespace is an online version of Visual Studio Code, with a repository cloned into a Linux container and required tools configured. This repo can create a customized Codespace with the resources in the .devcontainer directory. This will install pre-requisites needed on the Codespace.

Terraform can use optional Azure backend state, and invocation is wrapped by tf_deploy.ps1. This unlocks features dependent on using PowerShell (e.g. which use the Terraform local-exec provisioner).

1. Create a Codespace plan if you don't have one yet.

2. Create a Codespace by following this link:
   https://online.visualstudio.com/environments/new?name=azure-vdc&repo=geekzter/azure-vdc
   This should prompt you to clone this repo when creating the Codespace.

3. Once the Codespace has been created, open a terminal by typing Ctrl-` (backquote). This opens a PowerShell session.

4. (Optional) A Terraform Backend allows multi-host, multi-user collaboration on the same Terraform configuration. To set up a Terraform Azure Backend, create a storage account and configure backend.tf (copy backend.tf.sample) with the details of the storage account you created. Make sure the user used for Azure CLI has the Storage Blob Data Contributor or Storage Blob Data Owner role (it is not enough to have Owner/Contributor rights, as this doesn't grant Data Plane access). Alternatively, you can set ARM_ACCESS_KEY or ARM_SAS_TOKEN environment variables e.g.
   $env:ARM_ACCESS_KEY=$(az storage account keys list -n <STORAGE_ACCOUNT> --query "[0].value" -o tsv)
   or
   $env:ARM_SAS_TOKEN=$(az storage container generate-sas -n <STORAGE_CONTAINER> --permissions acdlrw --expiry 202Y-MM-DD --account-name <STORAGE_ACCOUNT> -o tsv)
   Initialize Terraform azurerm backend by running
   tf_deploy.ps1 -init

5. (Optional) Customize variables.tf or create a .auto.tfvars file that contains your customized configuration (see Features below)

6. Run
   tf_deploy.ps1 -apply
   to provision resources (this will create a plan that you will be prompted to apply).
   You will be prompted to select a subscription if $env:ARM_SUBSCRIPTION_ID is not set.

7. When you want to destroy resources, run:
   tf_deploy.ps1 -destroy (with Terraform, recommended)
   or
   erase.ps1 -destroy (with Azure CLI, as a last resort)

Option C: Azure Pipelines

Aforementioned options only provision infrastructure, and does so interactively. There are a number of pipelines that do this end-to-end and also deploy 2 demo applications:

• vdc-terraform-apply-ci.yml
  CI pipeline that does a full provisioning, deployment and tear down
• vdc-terraform-apply-release.yml
  Release pipeline that takes artifacts published by CI pipeline
• vdc-terraform-apply-cd.yml
  CD pipeline that combines both CI & release in a single multi-stage pipeline

All these pipelines share the same template.

Feature toggles

The Automated VDC has a number of features that are turned off by default. This can be because the feature has pre-requisites (e.g. certificates, or you need to own a domain). Another reason is the use of Azure preview features, or features that just simply take a long time to provision. Features are toggled by a corresponding variable in variables.tf.

Feature	Toggle	Dependencies and Pre-requisites
Azure Bastion. Provisions the Azure Bastion service in each Virtual Network	deploy_managed_bastion	None
Monitoring VM Extensions. Controls whether these extensions are provisioned: IaaSDiagnostics, MicrosoftMonitoringAgent, DependencyAgentWindows	deploy_monitoring_vm_extensions	None
Non‑essential VM Extensions. Controls whether these extensions are provisioned: TeamServicesAgent (for VM's that are not a deployment target for an Azure Pipeline), BGInfo	deploy_non_essential_vm_extensions	PowerShell 7
Network Watcher	deploy_network_watcher	deploy_non_essential_vm_extensions also needs to be set. This requires PowerShell 7
Security VM Extensions. Controls whether these extensions are provisioned: AADLoginForWindows, AzureDiskEncryption	deploy_security_vm_extensions	None
VPN, provisions Point-to-Site (P2S) VPN	deploy_vpn	You need to have the Azure VPN application provisioned in your Azure Active Directory tenant.
Disable all access public ip address of SQL Database, regardless of SQL Firewall settings	disable_public_database_access	enable_private_link also needs to be set
AAD Authentication. Configure App Service to authenticate using Azure Active Directory	enable_app_service_aad_auth	SSL and a vanity domain needs to have been set up. You also need to create an Azure AD App registration and configure the paas_aad_auth_client_id_map map for at least the default workspace (see example in config.auto.tfvars.sample)). (Note: Terraform could provision this pre-requiste as well, but I'm assuming you don't have suffiient AAD permissions as this requires a Service Principal to create Service Principals in automation)
Private Link	enable_private_link	None
[Storage Logging]. Enables storage logging to Log Analytics workspace.(https://docs.microsoft.com/en-us/azure/storage/common/monitor-storage)	enable_storage_diagnostic_setting	The subscription used needs to be enabled for the private preview of the Log Analytics diagnostics setting for Azure Storage
Grant access to SQL Database for App Service MSI and user/group defined by admin_object_id. This is required for database import and therefore application deployment	grant_database_access	PowerShell 7
Limits access to PaaS services to essential admin IPs, Virtual Networks	restrict_public_access	PowerShell 7. Scripting is required for reentrancy. The Terraform IP can change as multiple users collaborate, you simply execute from a different location, or your ISP gave you a dynamic IP address.
Pipeline agent type. By default a Deployment Group will be used. Setting this to true will instead use an Environment	use_pipeline_environment	Multi-stage YAML Pipelines
SSL & Vanity domain. Use HTTPS and Vanity domains (e.g. yourdomain.com)	use_vanity_domain_and_ssl	You need to own a domain, and delegate the management of the domain to Azure DNS. The domain name and resource group holding the Azure DNS for it need to be configured using vanity_domainname and shared_resources_group respectively. You need a wildcard SSL certificate and configure its location by setting vanity_certificate_* (see example in config.auto.tfvars.sample).

Dashboard

A portal dashboard will be generated:

This dashboard can be reverse engineered into the template that creates it by running:
templatize_dashboard.ps1
This recreates dashboard.tpl, which in turn generates the dashboard. Hence full round-trip dashboard editing support is provided.

Resources

• Azure CLI
• Azure Pipelines
• PowerShell Core
• Terraform Azure Backend
• Terraform Azure Provider
• Terraform on Azure documentation
• Terraform Learning
• Visual Studio Code

Disclaimer

This project is provided as-is, and is not intended as a blueprint on how a VDC should be deployed, or Azure components and Terraform should be used. It is merely an example on how you can use the technology. The project creates a number of Azure resources, you are responsible for monitoring and managing cost. Auto shutdown on is configured on VM's only.