Skip to content

tadelesh/depth-coverage-pipeline

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Overview

Depth-coverage pipeline is to help on automatically generate Azure CLI commands or Terraform provider code to achieve 100% depth coverage. It will be scheduled trigged to retrieve missing operations/resources from Coverage-Result report DB and automatically generate Azure CLI commands/Terraform resources which can be used by CLI/Terraform team to onboard them. There are two separated pipeline:

Codegen controller pipeline

Codegen controller pipeline is the interface of the whole framework. It will accept request and trigger code generation pipeline. Also, it will provide the service onboarding report to customer. Codegen controller pipeline is scheduled to run

Code generate pipeline

Code generate pipeline will be triggered by controller pipeline via repo Pull Request. It is responsible for generation code and validate service and generated code. Each codegen pipeline runtime instance is responsible for one RP.

How to Use

Setup

By default the depth-coverage will try to trigger pipelines for all missing resources/operations at once. You can set candidate resources manually. If the candicates are set, only trigger pipelines for those candidates to generate missing operations or resources.

candidate setting

configure candidates in CLICandidate.csv(for cli) and TFCandidate.csv(for terraform) https://dev.azure.com/devdiv/DevDiv/_git/codegen-pipeline?path=%2Fconfig

CLICandidate.csv

Edit these two csv files directly to add/remove candidates for cli/terraform.

Trigger depth-coverage

Codegen controller pipeline (codegen-pipeline, https://dev.azure.com/devdiv/DevDiv/_build?definitionId=14195 ) is to trigger depth-coverage. It is is scheduled to run once every week by default. You can manual trigger it if needed.

codegen-pipeline

After trigger depth-coverage, each code generation pipeline will be trigger for each candidate resource provider to generate. And code generation pipeline will automatically generates code, compile, run mock test and live test. When failure occur or pipeline completed, email will be sent to you to ask your interaction.

Failure Alert email

Failure Alert email

Onboarding confirm email

Onboarding confirm email

interactive

Failure resolve

When any failure occur during code generation pipeline (generate code, build, mock test, live test), the owner will receive the email, and they can take action to resolve the failure.

  • click 'Error Log' link to see the detail logs.
  • if need to update the swagger to resolve the issue, click 'code pull request' link to enter the work branch, update the file and commit changes.
  • click 'Resolve' link after you resolve the issue. The pipeline then will be re-trigger.
  • Customize

    When code generation is completed, and pass Tests, an onboarding confirm email will sent to the feature owener. If owner wants to customize, please following following steps:

  • click 'code' link to enter the work branch, and update the files (readme.**.md, schema.json) to customize. And then commit the changes.
  • click 'Customize' to re-trigger the code generation pipeline. In this round of pipeline, your customization will be included in the generated code. If you just want to generate the code, and don't want to run test, click 'Customize without Test' link.

    storage blob

    The generated code and logs are stored in Azure stroage blob: depthcoverage/depthcoverage

    https://ms.portal.azure.com/#blade/Microsoft_Azure_Storage/ContainerMenuBlade/overview/storageAccountId/%2Fsubscriptions%2F7fd08dcc-a653-4b0f-8f8c-4dac889fdda4%2FresourceGroups%2Ftrenton%2Fproviders%2FMicrosoft.Storage%2FstorageAccounts%2Fdepthcoverage/path/depthcoverage/etag/%220x8D8C3F7E45F943F%22/defaultEncryptionScope/%24account-encryption-key/denyEncryptionScopeOverride//defaultId//publicAccessVal/Container

    depthcoverage stroage blob

  • depth-clicore: store cli-core code
  • depth-terraform: store terraform code
  • logs: store all the pipeline running logs. each pipeline has a log folder with the pipeline build id as folder name.
  • generated code blob

    code blob

    logs blob

    log blob

    About

    No description, website, or topics provided.

    Resources

    Stars

    Watchers

    Forks

    Packages

    No packages published

    Languages

    • Go 53.1%
    • Shell 28.1%
    • Python 15.1%
    • JavaScript 3.7%