You might have to submit a support to increase Batch quotas
Copy the .auto.tfvars
file template:
cp config/template.tfvars .auto.tfvars
Create the account infrastructure:
terraform init
terraform apply -auto-approve
ℹ️ The pool will be provisioned with 0 nodes. Adjust your preferences accordingly.
Optionally, Configure your Batch account logs to be sent to the Log Analytics Workspace by setting up the Diagnostic Settings using the portal.
Use the script to create the application package:
cd scripts
bash scripts/create-batch-application.sh
Set the application to the pool:
az batch pool set \
--account-name bafastbrains \
--pool-id dev \
--application-package-references "molecular-analysis#1.0"
One option to easily interact with the CLI is to login to the Batch account:
az batch account login \
--name bafastbrains \
--resource-group rg-fastbrains \
--shared-key-auth
Alternatively, if you need to use the keys, add --shared-key-auth
.
Run a task:
az batch task create --task-id sciTask001 --command-line "echo task001" --job-id dev-job
View task status:
az batch task show \
--job-id dev-job \
--task-id sciTask001
View task output:
az batch task file list \
--job-id dev-job \
--task-id sciTask001 \
--output table
It is possible to create a task with the --json-file
option:
The file containing the task(s) to create in JSON(formatted to match REST API request body). When submitting multiple tasks, accepts either an array of tasks or a TaskAddCollectionParameter. If this parameter is specified, all other parameters are ignored.
Additional functionality for the CLI is available through extensions:
az extension add --name azure-batch-cli-extensions
The jump box already has System-Assigned Identity. To use it:
# Using the System-Assigned identity within the VM
az login --identity
This is not required if you use az batch account login
, but another option to interact with a private endpoint Batch/pools using the jump box:
export AZURE_BATCH_ACCOUNT=""
export AZURE_BATCH_ENDPOINT=""
export AZURE_BATCH_ACCESS_KEY=""
Now it is possible to use the private endpoints:
az batch pool list
Use the node IP and frontend port:
ssh batch@<ip> -p 50000
Batch files are located in the temporary storage.
For the mounted storage, on Ubuntu batch the path is /mnt/batch/tasks/fsmounts/blobs
.
Detailed usage of the applicationinsights
package can be found in the full documentation. For details on Azure Monitor custom metrics, link for the guidelines.
Configuration of the instruction can be done via opentelemetry
available variables:
OTEL_RESOURCE_ATTRIBUTES="service.namespace=<namespace>,service.instance.id=<instance>"
OTEL_SERVICE_NAME="<service>"
From the renewed official docs:
python3 -m venv .venv
source .venv/bin/activate
python3 -m pip install --upgrade pip
python3 -m pip install -r requirements.txt
For troubleshooting:
which python
.venv/bin/python
A user named BatchUser
will be created.
This user has Reader
permissions in the resource group, and Contributor
access in the Batch account.
It should be possible to created Pools with this user.