Skip to content

Commit

Permalink
Merge pull request #2088 from MicrosoftDocs/main
Browse files Browse the repository at this point in the history
12/19/2024 PM Publish
  • Loading branch information
Taojunshen authored Dec 19, 2024
2 parents 34757fc + 8e071aa commit 53fdfa0
Show file tree
Hide file tree
Showing 22 changed files with 81 additions and 42 deletions.
36 changes: 32 additions & 4 deletions articles/ai-services/agents/how-to/tools/openapi-spec.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,10 +27,29 @@ automated, and scalable API integrations that enhance the capabilities and effic
describing HTTP APIs. This allows people to understand how an API works, how a sequence of APIs
work together, generate client code, create tests, apply design standards, and more. Currently, we support 3 authentication types with the OpenAPI 3.0 specified tools: `anonymous`, `API key`, `managed identity`.

## Set up
## Prerequisites
1. Ensure you've completed the prerequisites and setup steps in the [quickstart](../../quickstart.md).

1. [optional]If your OpenAPI spec requires API key, you can store your API key in a `custom keys` connection and use `connection` authentication
1. Check the OpenAPI spec for the following requirements:
1. `operationId` should only contain letters, `-` and `_`. You can modify it to meet the requirement. We recommend using descriptive name to help models efficiently decide which function to use.

## Authenticating with API Key

1. Verify that the OpenAPI spec supports API keys: it has `securitySchemes` section and has one scheme of type `apiKey". An example would be:
```json
"securitySchemes": {
"apiKeyHeader": {
"type": "apiKey",
"name": "x-api-key",
"in": "header"
}
}
```
If the security schemes include multiple schemes, we recommend keeping only one of them.

1. Remove any parameter in the OpenAPI spec that needs API key, because API key will be stored and passed through a connection, as described later in this article.

1. Create a `custom keys` connection to store your API key.

1. Go to the [Azure AI Foundry portal](https://ai.azure.com/) and select the AI Project. Click **connected resources**.
:::image type="content" source="../../media/tools/bing/project-settings-button.png" alt-text="A screenshot of the settings button for an AI project." lightbox="../../media/tools/bing/project-settings-button.png":::
Expand All @@ -46,9 +65,18 @@ work together, generate client code, create tests, apply design standards, and m
:::image type="content" source="../../media/tools/bing/api-key-connection.png" alt-text="A screenshot of the custom keys selection for the AI project." lightbox="../../media/tools/bing/api-key-connection.png":::

1. Enter the following information
- `key`: "key"
- key: `name` of your security scheme. In this example, it should be `x-api-key`
```json
"securitySchemes": {
"apiKeyHeader": {
"type": "apiKey",
"name": "x-api-key",
"in": "header"
}
}
```
- value: YOUR_API_KEY
- Connection name: `YOUR_CONNECTION_NAME` (You will use this connection name in the sample code below.)
- Connection name: YOUR_CONNECTION_NAME (You will use this connection name in the sample code below.)
- Access: you can choose either *this project only* or *shared to all projects*. Just make sure in the sample code below, the project you entered connection string for has access to this connection.

::: zone-end
Expand Down
2 changes: 1 addition & 1 deletion articles/ai-studio/concepts/concept-model-distillation.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ description: Learn how to do distillation in Azure AI Foundry portal.
manager: scottpolly
ms.service: azure-ai-studio
ms.topic: how-to
ms.date: 07/23/2024
ms.date: 12/15/2024
ms.reviewer: vkann
reviewer: anshirga
ms.author: ssalgado
Expand Down
2 changes: 1 addition & 1 deletion articles/ai-studio/concepts/concept-synthetic-data.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ description: Learn how to generate a synthetic dataset in Azure AI Foundry porta
manager: scottpolly
ms.service: azure-ai-studio
ms.topic: how-to
ms.date: 07/23/2024
ms.date: 12/15/2024
ms.reviewer: vkann
reviewer: anshirga
ms.author: ssalgado
Expand Down
2 changes: 1 addition & 1 deletion articles/ai-studio/how-to/concept-data-privacy.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ manager: scottpolly
ms.service: azure-ai-studio
ms.custom: references_regions, build-2024
ms.topic: conceptual
ms.date: 10/29/2024
ms.date: 12/19/2024
ms.reviewer: shubhirajMsft
ms.author: scottpolly
author: s-polly
Expand Down
2 changes: 1 addition & 1 deletion articles/ai-studio/how-to/fine-tune-model-llama.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ description: Learn how to fine-tune Meta Llama models in Azure AI Foundry portal
manager: scottpolly
ms.service: azure-ai-studio
ms.topic: how-to
ms.date: 7/23/2024
ms.date: 12/16/2024
ms.reviewer: rasavage
reviewer: shubhirajMsft
ms.author: ssalgado
Expand Down
2 changes: 1 addition & 1 deletion articles/ai-studio/how-to/fine-tune-phi-3.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ manager: scottpolly
ms.service: azure-ai-studio
ms.custom:
ms.topic: how-to
ms.date: 7/16/2024
ms.date: 12/16/2024
ms.author: ssalgado
author: ssalgadodev
---
Expand Down
2 changes: 1 addition & 1 deletion articles/ai-studio/how-to/index-add.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ ms.custom:
- build-2024
- ignite-2024
ms.topic: how-to
ms.date: 5/21/2024
ms.date: 12/11/2024
ms.reviewer: estraight
ms.author: ssalgado
author: ssalgadodev
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
Expand Up @@ -274,7 +274,7 @@ Billing goes into effect when API calls to Azure AI services resources exceed 20
Keyless and key-based connections are used for billing, but not for enrichment operations' connections. For connections, a search service [connects over the internal network](search-security-overview.md#internal-traffic) to an Azure AI services resource that's located in the [same physical region](search-region-support.md). Most regions that offer Azure AI Search also offer other Azure AI services such as Language. If you attempt AI enrichment in a region that doesn't have both services, you'll see this message: "Provided key isn't a valid CognitiveServices type key for the region of your search service."
Indexers can be configured to run in a [private execution environment](search-howto-run-reset-indexers.md#indexer-execution) for dedicated processing using just the search nodes of your own search service. Even if you're using private execution environment, Azure AI Search still uses its internally provisioned Azure AI multiservice resource to perform all skill enrichments.
Indexers can be configured to run in a [private execution environment](search-howto-run-reset-indexers.md#indexer-execution-environment) for dedicated processing using just the search nodes of your own search service. Even if you're using private execution environment, Azure AI Search still uses its internally provisioned Azure AI multiservice resource to perform all skill enrichments.
> [!NOTE]
> Some built-in skills are based on non-regional Azure AI services (for example, the [Text Translation Skill](cognitive-search-skill-text-translation.md)). Using a non-regional skill means that your request might be serviced in a region other than the Azure AI Search region. For more information on non-regional services, see the [Azure AI services product by region](https://aka.ms/allinoneregioninfo) page.
Expand Down
2 changes: 1 addition & 1 deletion articles/search/cognitive-search-defining-skillset.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ After the name and description, a skillset has four main properties:

Inside the skillset definition, the skills array specifies which skills to execute. Three to five skills are common, but you can add as many skills as necessary, subject to [service limits](search-limits-quotas-capacity.md#indexer-limits).

The end result of an enrichment pipeline is textual content in either a search index or a knowledge store. For this reason, most skills either create text from images (OCR text, captions, tags), or analyze existing text to create new information (entities, key phrases, sentiment). Skills that operate independently are processed in parallel. Skills that depend on each other specify the output of one skill (such as key phrases) as the input of a second skill (such as text translation). The search service determines the order of skill execution and the execution environment.
The end result of an enrichment pipeline is textual content in either a search index or a knowledge store. For this reason, most skills either create text from images (OCR text, captions, tags), or analyze existing text to create new information (entities, key phrases, sentiment). Skills that operate independently are processed in parallel. Skills that depend on each other specify the output of one skill (such as key phrases) as the input of a second skill (such as text translation). The search service determines the order of skill execution and the [execution environment](search-howto-run-reset-indexers.md#indexer-execution-environment).

All skills have a type, context, inputs, and outputs. A skill might optionally have a name and description. The following example shows two unrelated [built-in skills](cognitive-search-predefined-skills.md) so that you can compare the basic structure.

Expand Down
13 changes: 11 additions & 2 deletions articles/search/search-get-started-rbac.md
Original file line number Diff line number Diff line change
Expand Up @@ -120,7 +120,7 @@ Several quickstarts and tutorials use a REST client, such as Visual Studio Code

You should have a `.rest` or `.http` file, similar to the one described in [Quickstart: Vector search](search-get-started-vector.md).

1. Get an access token:
1. Generate an access token.

```azurecli
az account get-access-token --scope https://search.azure.com/.default --query accessToken --output tsv
Expand Down Expand Up @@ -149,13 +149,22 @@ You should have a `.rest` or `.http` file, similar to the one described in [Quic
}
```

If the call fails, revisit the previous steps to make sure you didn't skip any. You might also want to restart your device.
### Troubleshoot 401 errors

- Check the active subscription and tenant (`az account show`) and make sure it's valid for your search service.

- Check the search service **Settings** > **Keys** options in the Azure portal and confirm the service is configured for **Both"** or **Role-based access control**.

- For the REST client only: Check the token and endpoint specified in your file and make sure there's no surrounding quotes or extra spaces.

If all else fails, restart your device to remove any cached tokens, and then repeat the steps in this section, starting with `az login`.

## Additional configuration

Configure a managed identity for outbound connections:

- [Configure a system-assigned or user-assigned managed identity](search-howto-managed-identities-data-sources.md) for your search service.

- [Use role assignments](keyless-connections.md) to authorize access to other Azure resources.

Network access configuration:
Expand Down
2 changes: 1 addition & 1 deletion articles/search/search-how-to-create-indexers.md
Original file line number Diff line number Diff line change
Expand Up @@ -197,7 +197,7 @@ POST /indexers?api-version=[api-version]
}
```

Parameters are used to set the batch size and how to handle processing failures. The [execution environment](search-howto-run-reset-indexers.md#indexer-execution) determines whether indexer and skillset processing can use the multitenant capabilities provided by Microsoft or the private processing nodes allocated exclusively to your search service. If your search service is Standard2 or higher, you can set `executionEnvironment` to private to pin all indexer processing to just your search service clusters.
Parameters are used to set the batch size and how to handle processing failures. The [execution environment](search-howto-run-reset-indexers.md#indexer-execution-environment) determines whether indexer and skillset processing can use the multitenant capabilities provided by Microsoft or the private processing nodes allocated exclusively to your search service. If your search service is Standard2 or higher, you can set `executionEnvironment` to private to pin all indexer processing to just your search service clusters.

There are numerous tutorials and examples that demonstrate REST clients for creating objects. [Quickstart: Text search using REST](search-get-started-rest.md) can get you started.

Expand Down
2 changes: 1 addition & 1 deletion articles/search/search-how-to-large-index.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ When there are no longer any new or updated documents in the data source, indexe
For more information about setting schedules, see [Create Indexer REST API](/rest/api/searchservice/indexers/create) or see [Schedule indexers for Azure AI Search](search-howto-schedule-indexers.md).

> [!NOTE]
> Some indexers that run on an older runtime architecture have a 24-hour rather than 2-hour maximum processing window. The two-hour limit is for newer content processors that run in an [internally managed multi-tenant environment](search-indexer-securing-resources.md#indexer-execution-environment). Whenever possible, Azure AI Search tries to offload indexer and skillset processing to the multi-tenant environment. If the indexer can't be migrated, it runs in the private environment and it can run for as long as 24 hours. If you're scheduling an indexer that exhibits these characteristics, assume a 24-hour processing window.
> Some indexers that run on an older runtime architecture have a 24-hour rather than 2-hour maximum processing window. The two-hour limit is for newer content processors that run in an [internally managed multitenant environment](search-howto-run-reset-indexers.md#indexer-execution-environment). Whenever possible, Azure AI Search tries to offload indexer and skillset processing to the multi-tenant environment. If the indexer can't be migrated, it runs in the private environment and it can run for as long as 24 hours. If you're scheduling an indexer that exhibits these characteristics, assume a 24-hour processing window.
<a name="parallel-indexing"></a>

Expand Down
10 changes: 6 additions & 4 deletions articles/search/search-howto-run-reset-indexers.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ ms.service: azure-ai-search
ms.custom:
- ignite-2023
ms.topic: how-to
ms.date: 10/25/2024
ms.date: 12/19/2024
---

# Run or reset indexers, skills, or documents
Expand Down Expand Up @@ -38,16 +38,18 @@ Once indexer execution starts, you can't pause or stop it. Indexer execution sto

You can run multiple indexers at one time assuming sufficient capacity, but each indexer itself is single-instance. Starting a new instance while the indexer is already in execution produces this error: `"Failed to run indexer "<indexer name>" error: "Another indexer invocation is currently in progress; concurrent invocations are not allowed."`

## Indexer execution environment

An indexer job runs in a managed execution environment. Currently, there are two environments:

+ A private execution environment runs on search clusters that are specific to your search service. If your search service is Standard2 or nigher, you can [set the `executionEnvironment` parameter](search-how-to-create-indexers.md?tabs=indexer-rest#create-an-indexer) in the indexer definition to always run an indexer in the private execution environment.
+ A private execution environment runs on search clusters that are specific to your search service. If your search service is Standard2 or higher, you can [set the `executionEnvironment` parameter](search-how-to-create-indexers.md?tabs=indexer-rest#create-an-indexer) in the indexer definition to always run an indexer in the private execution environment.

+ A multitenant environment has content processors that are managed and secured by Microsoft at no extra cost. This environment is used to offload computationally intensive processing, leaving service-specific resources available for routine operations. Whenever possible, most skillsets execute in the multitenant environment. This is the default.

Computationally intensive processing includes skillsets running on content processors, and high volume indexer jobs or indexer jobs with large documents. Non-skillset processing on the multitenant content processors is determined by hueristics and system information and isn't under customer control. S2 services and higher support pinning an indexer and skillset processing exclusively to your search clusters through the `executionEnvironment` parameter.
*Computationally intensive processing* refers to skillsets running on content processors and indexer jobs that process a high volume of documents, or documents of a large size. Non-skillset processing on the multitenant content processors is determined by hueristics and system information and isn't under customer control. S2 services and higher support pinning an indexer and skillset processing exclusively to your search clusters through the `executionEnvironment` parameter.

> [!NOTE]
> [IP firewalls](search-indexer-securing-resources.md#indexer-execution-environment) block the multitenant environment, so if you have a firewall, create a rule that allows multitenant processing.
> [IP firewalls](search-indexer-securing-resources.md#network-access-and-indexer-execution-environments) block the multitenant environment, so if you have a firewall, create a rule that allows multitenant processing.
Indexer limits vary for each environment:

Expand Down
2 changes: 1 addition & 1 deletion articles/search/search-howto-schedule-indexers.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@ You can run multiple indexers simultaneously, but each indexer is single instanc

For text-based indexing, the scheduler can kick off as many indexer jobs as the search service supports, which is determined by the number of [search units](search-capacity-planning.md#concepts-search-units-replicas-partitions). For example, if the service has three replicas and four partitions, you can have 12 indexer jobs in active execution, whether initiated on demand or on a schedule.

For skills-based indexing, indexers run in a specific [execution environment](search-howto-run-reset-indexers.md#indexer-execution). For this reason, the number of service units has no bearing on the number of skills-based indexer jobs you can run. Multiple skills-based indexers can run in parallel, but doing so depends on content processor availability within the execution environment.
For skills-based indexing, indexers run in a specific [execution environment](search-howto-run-reset-indexers.md#indexer-execution-environment). For this reason, the number of service units has no bearing on the number of skills-based indexer jobs you can run. Multiple skills-based indexers can run in parallel, but doing so depends on content processor availability within the execution environment.

**Do scheduled jobs always start at the designated time?**

Expand Down
4 changes: 2 additions & 2 deletions articles/search/search-indexer-how-to-access-private-sql.md
Original file line number Diff line number Diff line change
Expand Up @@ -151,9 +151,9 @@ This article assumes a [REST client](search-get-started-rest.md) and uses the RE
}
```
1. [Create the indexer definition](search-howto-create-indexers.md), setting the indexer execution environment to "private".
1. [Create the indexer definition](search-howto-create-indexers.md), setting the indexer `executionEnvironment` to "private".
[Indexer execution](search-indexer-securing-resources.md#indexer-execution-environment) occurs in either a private environment that's specific to the search service, or a multi-tenant environment that's used internally to offload expensive skillset processing for multiple customers. **When connecting over a private endpoint, indexer execution must be private.**
[Indexer execution](search-howto-run-reset-indexers.md#indexer-execution-environment) occurs in either a private execution environment that's specific to your search service, or a multi-tenant environment hosted by Microsoft and used to offload expensive skillset processing for multiple customers. **When connecting over a private endpoint, indexer execution must be private.**
```http
POST https://myservice.search.windows.net/indexers?api-version=2024-07-01
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ For ping, the request times out, but the IP address is visible in the response.

## Get IP addresses for "AzureCognitiveSearch" service tag

You'll also need to create an inbound rule that allows requests from the [multitenant execution environment](search-indexer-securing-resources.md#indexer-execution-environment). This environment is managed by Microsoft and it's used to offload processing intensive jobs that could otherwise overwhelm your search service. This section explains how to get the range of IP addresses needed to create this inbound rule.
You'll also need to create an inbound rule that allows requests from the [multitenant execution environment](search-indexer-securing-resources.md#network-access-and-indexer-execution-environments). This environment is managed by Microsoft and it's used to offload processing intensive jobs that could otherwise overwhelm your search service. This section explains how to get the range of IP addresses needed to create this inbound rule.

An IP address range is defined for each region that supports Azure AI Search. Specify the full range to ensure the success of requests originating from the multitenant execution environment.

Expand Down
Loading

0 comments on commit 53fdfa0

Please sign in to comment.