Skip to content

Commit

Permalink
Merge branch 'microsoft:master' into master
Browse files Browse the repository at this point in the history
  • Loading branch information
noakup authored Sep 8, 2024
2 parents f81bd77 + 1e4792b commit 8a3d53a
Show file tree
Hide file tree
Showing 213 changed files with 87,729 additions and 45,067 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ bld/
[Oo]bj/
[Ll]og/
[Oo]utput/
scripts/dist/

# localized repos that get cloned into scripts
scripts/cs-cz/
Expand Down
6 changes: 6 additions & 0 deletions CODEOWNERS
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,10 @@
/Workbooks/Azure*Private*MEC/** @microsoft/azure-private-mec-workbooks
/Workbooks/Azure*API*Management/** @microsoft/azure-api-management

/Workbooks/Azure*Logic*Apps/** @microsoft/azure-logic-apps-admin

/Workbooks/Azure*Feature*Experimentation/** @microsoft/exp-devs

# Section for gallery files
/gallery/workbook/WaaSUpdateInsights.json @microsoft/update-compliance-workbooks
/gallery/workbook/Azure*Monitor.json @microsoft/applicationinsights-devtools
Expand All @@ -153,6 +157,7 @@
/gallery/workbook/microsoft.cache-redis.json @microsoft/azure-cache-for-redis @microsoft/applicationinsights-devtools
/gallery/workbook/microsoft.network* @microsoft/network-insights @microsoft/applicationinsights-devtools
/gallery/workbook/microsoft.wvd-insights* @microsoft/windows-virtual-desktop
/gallery/workbook/Microsoft.DesktopVirtualization* @microsoft/windows-virtual-desktop
/gallery/adxcluster-insights/microsoft.kusto-clusters.json @microsoft/kusto-workbooks
/gallery/workbook/microsoft.dbformysql-flexibleservers.json @microsoft/azuremysql
/gallery/workbook/microsoft.dbforpostgresql-flexibleservers.json @microsoft/azurepostgresql
Expand Down Expand Up @@ -180,6 +185,7 @@
/gallery/apimanagement-service/** @microsoft/azure-api-management
/gallery/workbook/microsoft.mobilenetwork-mobilenetwork-sites.json @microsoft/azure-private-mec-workbooks
/gallery/workbook/microsoft.apimanagement-service.json @microsoft/azure-api-management
/gallery/workbook/microsoft.experimentation-experimentworkspaces @microsoft/exp-devs

# leave this at the bottom!
# EVERYTHING ending with resjson* is for loc, and is owned by the workbooks team and the loc team
Expand Down
12 changes: 5 additions & 7 deletions Documentation/Deployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,17 +3,15 @@
## Template deployment via Safe Deploy process
Once content is checked into master, the template content is packaged into an npm package, and the release pipeline will publish that npm package to our internal team packaging feed where other dependencies live. That npm package of templates will be picked up by the AppInsights extension's daily build, and deployed via the extension's daily release train.

1. An [official build is queued](https://github-private.visualstudio.com/microsoft/_build?definitionId=474) (you might not have access to this) @ noon (pacific) daily which packages all the processed templates into a versioned NPM package.
1. An [official build is queued](https://msazure.visualstudio.com/One/_build?definitionId=299360) (you might not have access to this) @ noon PST/1pm PDT daily which packages all the processed templates into a versioned NPM package and uploads the NPM package to an Azure Devops package feed.

2. After the official build completes, [a release pipeline](https://github-private.visualstudio.com/microsoft/_release?_a=releases&view=mine&definitionId=65) (again, you might not have access to this), uploads the built NPM package to an Azure Devops package feed.
2. Every weekday @ 3pm (pacific), a daily build of the Application Insights Azure Portal extension takes place. This build consumes the NPM package from the ADO package feed.

3. Every weekday @ 3pm (pacific), a daily build of the Application Insights Azure Portal extension takes place. This build consumes the NPM package from the ADO package feed.
3. Every weekday @ ~4pm (pacific), [a daily deployment of the AppInsightsExtension occcurs](https://eng.ms/docs/cloud-ai-platform/azure/aep-platform-infrastructure/observability/application-insights/portal/operations/deployment#deployment-pipeline), which deploys the extension to a pre-production environment ([PPE](https://portal.azure.com/?feature.canmodifystamps=true&appInsightsExtension=ppe))

4. Every weekday @ ~4pm (pacific), [a daily deployment of the AppInsightsExtension occcurs](https://eng.ms/docs/cloud-ai-platform/azure/aep-platform-infrastructure/observability/application-insights/portal/operations/deployment#deployment-pipeline), which deploys the extension to a pre-production environment ([PPE](https://portal.azure.com/?feature.canmodifystamps=true&appInsightsExtension=ppe))
4. Every weekday @ ~noon (pacific), the previous day's build moves up a stage, from PPE to another internal environment ([MPAC](https://portal.azure.com/?feature.canmodifystamps=true&appInsightsExtension=mpac)), and other non-public cloud test environments.

5. Every weekday @ ~noon (pacific), the previous day's build moves up a stage, from PPE to another internal environment ([MPAC](https://portal.azure.com/?feature.canmodifystamps=true&appInsightsExtension=mpac)), and other non-public cloud test environments.

6. Every weekday @ ~noon (pacific), the previous previous day's build moves up a stage and starts a rolling deployment to production Azure environments, including other non-public clouds. The production rollouts take several hours as they deploy region by region.
5. Every weekday @ ~noon (pacific), the previous previous day's build moves up a stage and starts a rolling deployment to production Azure environments, including other non-public clouds. The production rollouts take several hours as they deploy region by region.

In the usual case, this means that template changes will not show up in Production Azure for at least 48 hours.

Expand Down
Binary file added Documentation/Images/GalleryRedirectWarning.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified Documentation/Images/TestDeployTroubleshooting.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
107 changes: 34 additions & 73 deletions Documentation/Testing.md
Original file line number Diff line number Diff line change
@@ -1,93 +1,54 @@
# How to test your changes

There are 3 primary ways to test changes to a template, from simplest to more complicated but more powerful
1. [Using advanced mode](#using-advanced-mode) - this will only work for you, locally in your browser
2. [Redirecting the gallery to a github branch](#redirecting-the-gallery-to-a-github-branch) - can work for anyone with the url, as a short term testing solution
3. [Deploying your own gallery](#deploying-your-own-gallery) - can work for anyone, can add/move items in galleries. most powerful but more setup
Testing changes to a template can be done in two primary ways:

## Using Advanced Mode
It is possible to test your changes without merging your content to master.
the simplest possible testing is by opening workbooks in the place you expect your template to work,
1. Create an empty workbook
2. Go to advanced mode
3. Paste the contents of the `.workbook` template file into advanced mode
4. Use the `Apply` button
1. [**Using Advanced Mode**](#using-advanced-mode): Works locally in your browser, exclusively for you.
2. [**Redirecting to a GitHub Branch**](#redirecting-the-gallery-to-a-github-branch): Allows anyone with the URL to test changes temporarily; slightly more complicated but widely accessible.

Assuming your template content is valid, your template will appear in the view. If the template content was not valid, you will get an error notification displaying why your content is not valid.
## Using Advanced Mode
If you're only modifying a workbook, you can test your changes locally without any usage of Git. Follow these steps:

## Redirecting the gallery to a github branch
1. **Create an empty workbook.**
2. **Access Advanced Mode.**
3. **Paste the contents of the `.workbook` template file into Advanced Mode.**
4. **Click on the `Apply` button.**

If you are only changing the contents of an existing template, not adding new templates or altering which galleries a template appears in, you can use the feature flag `feature.workbookGalleryBranch` setting to tell the Workbooks view to look in a specific published github branch for the new content. Doing testing this way will let other users also see the changes to the template.
> [!TIP]
> If your template content is valid, it will appear in the view. If not, you'll receive an error notification explaining why your content is invalid.
1. Make your changes to your branch
2. Push the branch to github
3. Add `?feature.workbookGalleryBranch=[name of branch]` to the portal url, so your URL looks something like [https://portal.azure.com/?feature.workbookGalleryBranch=master](https://portal.azure.com/?feature.workbookGalleryBranch=master)
## Redirecting the Gallery to a GitHub Branch

> [!IMPORTANT]
> This feature flag is intended for short-term testing and not as a permanent solution.
If it works correctly, you'll see a banner in the gallery:
![Gallery Redirect Banner](Images/GalleryBranchRedirect.png)
To test changes in an existing template without merging to master or altering galleries, you can use the `feature.workbookGalleryBranch` setting. This setting instructs the Workbooks view to fetch content from a specific GitHub branch for testing purposes. Other users can also view these changes.

> **Limitations**
> 1. This only works for existing templates which are already exposed in a gallery, and which have `.workbook` file names that are the same as the parent directory.
> 2. If templates are renamed, moved, or the branch is deleted, this method will stop working.
> 3. This will cause your browser to read directly from `https://raw.githubusercontent.com/microsoft/Application-Insights-Workbooks/`, which may be slower and may cause throttling errors if you attempt to load too many items too quickly.
>
> This feature flag is intended only for short term test usage, and should not be used as a long term solution.
### Steps:

## Deploying your own gallery
If you are adding new items to a gallery, or adding new gallery entirely, you can use the feature flag `feature.workbookGalleryRedirect` to redirect the entire workbook gallery to a url that you control.
1. **Make Changes to Your Branch**: Implement the desired modifications on your branch.
2. **Push to GitHub**: Once changes are made, push the branch to our GitHub repository.
3. **Update Portal URL**: Append `?feature.workbookGalleryBranch=[name of branch]` to the portal URL. For example: `https://portal.azure.com/?feature.workbookGalleryBranch=master`.

If this worked, a banner will display in the gallery indicating the redirection:

1. Clone the repo, create your local branch, and make your changes locally
2. From the `scripts` folder of the repo, run `processDevPackageOnly.cmd` (ideally from a command prompt so you can see any output/errors)
- note: repeated runs of this script may generate error lines that files already exist and are being overwritten
![Workbook gallery branch message](./Images/GalleryRedirectWarning.png)

You should now have an `outputs\package` folder in the repo that contains the built package of content, but only the en-us version.
3. Every time you update any content, re-run the `processDevPackageOnly` script to repackage your changes.
4. Copy/upload your package content (see below)
5. Add `?feature.workbookGalleryRedirect=[url to your package]` to the portal url and reload the portal (if you already have other query parameters on the portal url)
### Limitations:

If it works correctly, you'll see a banner in the gallery:
![Gallery Redirect Banner](Images/GalleryRedirect.png)
- **Performance**: Loading workbooks from GitHub may slow down loading times and lead to throttling errors if too many items are loaded in a short succession of time.

### Uploading package content
There are 2 ways to host your built packages. Choose one of them.

1. **Setting up a storage account to deploy your package**

By setting up a storage account to host your changes, you can share the portal link outlined in step 7 with other team members or create a private previews with select customers.
1. Create azure storage account
2. In that storage account create blob container, like "azure_monitor_workbook_templates"
3. In that storage account, [enable CORS rules](https://docs.microsoft.com/en-us/rest/api/storageservices/cross-origin-resource-sharing--cors--support-for-the-azure-storage-services) so your machine will be able to read from that storage account
4. In that storage account, [configure public access](https://docs.microsoft.com/en-us/azure/storage/blobs/anonymous-read-access-configure?tabs=portal) to enable unauthenticated access to that storage account.
5. Upload contents of `outputs\package` directory to the blob container (so you now have a path like `azure_monitor_workbook_templates/package` in the storage account)
- The [Microsoft Azure Storage Explorer](https://azure.microsoft.com/en-us/features/storage-explorer/#features) allows you to upload entire directories directly to the container.
6. Get the url to that folder; it will be something like `https://[name of storage account].blob.core.windows.net/azure_monitor_workbook_templates/package`
- At this point, attempt to navigate directly to that folder url from a browser to make sure you have the right settings. (note that navigating directly will not test CORS, only access from the portal will)
7. Set that as a feature flag setting on the portal url. the feature flag will be `feature.workbookGalleryRedirect=[your url]`
- You'll end up with something like `https://portal.azure.com/?feature.workbookGalleryRedirect=https://[yourblob].blob.core.windows.net/azure_monitor_workbook_templates/package`
8. Test this in the portal. Ensure you have no network errors in the network console, this is where you will see CORS related errors about missing headers if CORS is not enabled.
9. As you make changes to your templates, rebuild the package and re-upload changed content.

2. **Running a local web server**

If you are already running something like Apache or IIS locally and you don't intend on sharing your changes, you can upload built package to your web server.
1. Use your web server settings to expose the `outputs\package` folder as readable. Ensure it is available via HTTPS, and ensure that CORS is enabled to allow loading that url from the portal.
2. Set that as a feature flag setting on the portal url. the feature flag will be `feature.workbookGalleryRedirect=[your url]`
- so you'll end up with something like `https://portal.azure.com/?feature.workbookGalleryRedirect=https://localhost/package`
3. Test this in the portal. Ensure you have no network errors in the network console, this is where you will see CORS related errors about missing headers if CORS is not enabled.
4. As you make changes to your templates, rebuild the package and re-upload changed content.

### Troubleshooting deployed gallery
1. How do I know if content is being served from my gallery?
> [!WARNING]
> As of June 2024, this previously available method is no longer supported. Passing the `feature.workbookGalleryRedirect` feature flag will be ignored.
### Troubleshooting
1. How do I know if content is being served from my GitHub branch?
- Press on `F12` or `Ctrl+Shift+I` to bring up the developer console.
- Switch to the network tab, and filter to your storage container name or local web server name
- Switch to the network tab, and filter by the "GitHub" keyword.
- Navigate to a workbook gallery (For example: Azure Monitor -> Workbooks)
- Inspect that the gallery file loaded is from your own gallery
- Inspect that the gallery file loaded is from your own gallery:
![Troubleshooting image](./Images/TestDeployTroubleshooting.png)
- Visit your modified workbooks and galleries. Ensure your changes are being reflected.

2. After making changes, running the processDevPackage.cmd, and uploading the contents to my gallery, I don't see my changes.
- Press on `F12` or `Ctrl+Shift+I` to bring up the developer console.
- Switch to the network tab
- Ensure that the checkbox 'Disabled cache' is checked
- With the developer console still open, refresh the gallery, and you should see your changes

If after trying these steps you can't deploy your gallery or templates, [file an issue](https://github.com/microsoft/Application-Insights-Workbooks/issues) and we'll help debug the problem.

Loading

0 comments on commit 8a3d53a

Please sign in to comment.