diff --git a/articles/ai-services/content-safety/overview.md b/articles/ai-services/content-safety/overview.md index 50acd74a0c7..72a261ac8e6 100644 --- a/articles/ai-services/content-safety/overview.md +++ b/articles/ai-services/content-safety/overview.md @@ -140,25 +140,31 @@ For more information, see [Language support](/azure/ai-services/content-safety/l To use the Content Safety APIs, you must create your Azure AI Content Safety resource in the supported regions. Currently, the Content Safety features are available in the following Azure regions: -|Region | Moderation APIs
(text and image) | Prompt Shields
(preview) | Protected material
detection (preview) | Groundedness
detection (preview) | Custom categories
(rapid) (preview) | Custom categories
(standard) | Blocklists | -|---|---|---|---|---|---|---|--| -| East US | ✅ | ✅| ✅ |✅ |✅ |✅|✅ | -| East US 2 | ✅ | | | ✅ |✅ | |✅| -| West US | | | | | ✅ | | | -| West US 2 | ✅ | | | |✅ | |✅ | -| Central US | ✅ | | | | | |✅ | -| North Central US | ✅ | | | |✅ | | ✅| -| South Central US | ✅ | | | | ✅| |✅ | -| Canada East | ✅ | | | | ✅| | ✅| -| Switzerland North | ✅ | | | |✅ | ✅ | ✅| -| Sweden Central | ✅ | | |✅ |✅ | | ✅| -| UK South | ✅ | | | | ✅| |✅ | -| France Central | ✅ | | | |✅ | | ✅| -| West Europe | ✅ | ✅ |✅ | |✅ | |✅ | -| Japan East | ✅ | | | |✅ | |✅ | -| Australia East| ✅ | | | |✅ | ✅| ✅| -| USGov Arizona | ✅ | | | | | | | -| USGov Virginia | ✅ | | | | | | | +| Region | Moderation APIs (text and image) | Prompt Shields | Protected material detection for Text | Groundedness detection (preview) | Custom categories (rapid) (preview) | Custom categories (standard) | Blocklists | +|---------------------|----------------------------------|----------------|--------------------------------------|-----------------------------------|------------------------------------|-------------------------------|------------| +| East US | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| East US 2 | ✅ | | ✅ | ✅ | ✅ | | ✅ | +| West US | | ✅ | ✅ | | ✅ | | | +| West US 2 | ✅ | ✅ | ✅ | | ✅ | | ✅ | +| West US 3 | ✅ | ✅ | ✅ | | ✅ | | ✅ | +| Poland Central | ✅ | ✅ | ✅ | | ✅ | | ✅ | +| South East Asia | ✅ | | ✅ | | ✅ | | ✅ | +| Central US | ✅ | | ✅ | | ✅ | | ✅ | +| North Central US | ✅ | | ✅ | | ✅ | | ✅ | +| South Central US | ✅ | | ✅ | | ✅ | | ✅ | +| Canada East | ✅ | ✅ | ✅ | | ✅ | | ✅ | +| Switzerland North | ✅ | ✅ | ✅ | | ✅ | ✅ | ✅ | +| Sweden Central | ✅ | ✅ | ✅ | ✅ | ✅ | | ✅ | +| UK South | ✅ | | ✅ | | ✅ | | ✅ | +| France Central | ✅ | ✅ | ✅ | | ✅ | | ✅ | +| West Europe | ✅ | ✅ | ✅ | | ✅ | | ✅ | +| Japan East | ✅ | ✅ | ✅ | | ✅ | | ✅ | +| Australia East | ✅ | ✅ | ✅ | | ✅ | | ✅ | +| South India | ✅ | | ✅ | | ✅ | ✅ | ✅ | +| USGov Arizona | ✅ | ✅ | ✅ | | | | | +| USGov Virginia | ✅ | ✅ | ✅ | | | | | + + Feel free to [contact us](mailto:contentsafetysupport@microsoft.com) if you need other regions for your business. diff --git a/articles/ai-services/language-service/conversational-language-understanding/service-limits.md b/articles/ai-services/language-service/conversational-language-understanding/service-limits.md index b05c27963a5..081c7b0d2a7 100644 --- a/articles/ai-services/language-service/conversational-language-understanding/service-limits.md +++ b/articles/ai-services/language-service/conversational-language-understanding/service-limits.md @@ -59,7 +59,7 @@ The following limits are observed for the conversational language understanding. |Item|Lower Limit| Upper Limit | | --- | --- | --- | -|Number of utterances per project | 1 | 25,000| +|Number of utterances per project | 1 | 50,000| |Utterance length in characters (authoring) | 1 | 500 | |Utterance length in characters (prediction) | 1 | 1000 | |Number of intents per project | 1 | 500| diff --git a/articles/ai-services/openai/concepts/model-retirements.md b/articles/ai-services/openai/concepts/model-retirements.md index 1776c55c7b8..c2ce58c5716 100644 --- a/articles/ai-services/openai/concepts/model-retirements.md +++ b/articles/ai-services/openai/concepts/model-retirements.md @@ -4,7 +4,7 @@ titleSuffix: Azure OpenAI description: Learn about the model deprecations and retirements in Azure OpenAI. ms.service: azure-ai-openai ms.topic: conceptual -ms.date: 08/22/2024 +ms.date: 09/03/2024 ms.custom: manager: nitinme author: mrbullwinkle @@ -89,17 +89,17 @@ For information on the model upgrade process, see [How to upgrade to a new model These models are currently available for use in Azure OpenAI Service. -| Model | Version | Retirement date | Suggested replacement | +| Model | Version | Retirement date | Suggested replacements | | ---- | ---- | ---- | --- | -| `gpt-35-turbo` | 0301 | No earlier than November 1, 2024 | `gpt-4o-mini` | -| `gpt-35-turbo`
`gpt-35-turbo-16k` | 0613 | November 1, 2024 | `gpt-4o-mini` | -| `gpt-35-turbo` | 1106 | No earlier than Nov 17, 2024 | `gpt-4o-mini` | +| `gpt-35-turbo` | 0301 | No earlier than November 1, 2024

Deployments set to [**Auto-update to default**](/azure/ai-services/openai/how-to/working-with-models?tabs=powershell#auto-update-to-default) will be automatically upgraded to version: `0125`, starting on November 15, 2024. | `gpt-35-turbo` (0125)

`gpt-4o-mini` | +| `gpt-35-turbo`
`gpt-35-turbo-16k` | 0613 | November 1, 2024

Deployments set to [**Auto-update to default**](/azure/ai-services/openai/how-to/working-with-models?tabs=powershell#auto-update-to-default) will be automatically upgraded to version: `0125`, starting on November 15, 2024. | `gpt-35-turbo` (0125)

`gpt-4o-mini`| +| `gpt-35-turbo` | 1106 | No earlier than Nov 17, 2024

Deployments set to [**Auto-update to default**](/azure/ai-services/openai/how-to/working-with-models?tabs=powershell#auto-update-to-default) will be automatically upgraded to version: `0125`, starting on November 15, 2024. | `gpt-35-turbo` (0125)

`gpt-4o-mini` | | `gpt-35-turbo` | 0125 | No earlier than Feb 22, 2025 | `gpt-4o-mini` | -| `gpt-4`
`gpt-4-32k` | 0314 | **Deprecation:** November 1, 2024
**Retirement:** June 6, 2025 | `gpt-4o` | -| `gpt-4`
`gpt-4-32k` | 0613 | **Deprecation:** November 1, 2024
**Retirement:** June 6, 2025 | `gpt-4o` | -| `gpt-4` | 1106-preview | To be upgraded to `gpt-4` Version: `turbo-2024-04-09`, starting on November 15, 2024, or later **1** | `gpt-4o`| -| `gpt-4` | 0125-preview |To be upgraded to `gpt-4` Version: `turbo-2024-04-09`, starting on November 15, 2024, or later **1** | `gpt-4o` | -| `gpt-4` | vision-preview | To be upgraded to `gpt-4` Version: `turbo-2024-04-09`, starting on November 15, 2024, or later **1** | `gpt-4o`| +| `gpt-4`
`gpt-4-32k` | 0314 | **Deprecation:** November 15, 2024
**Retirement:** June 6, 2025 | `gpt-4o` | +| `gpt-4`
`gpt-4-32k` | 0613 | **Deprecation:** November 15, 2024
**Retirement:** June 6, 2025 | `gpt-4o` | +| `gpt-4` | 1106-preview | To be upgraded to `gpt-4` version: `turbo-2024-04-09`, starting on November 15, 2024, or later **1** | `gpt-4o`| +| `gpt-4` | 0125-preview |To be upgraded to `gpt-4` version: `turbo-2024-04-09`, starting on November 15, 2024, or later **1** | `gpt-4o` | +| `gpt-4` | vision-preview | To be upgraded to `gpt-4` version: `turbo-2024-04-09`, starting on November 15, 2024, or later **1** | `gpt-4o`| | `gpt-3.5-turbo-instruct` | 0914 | No earlier than Sep 14, 2025 | | | `text-embedding-ada-002` | 2 | No earlier than April 3, 2025 | `text-embedding-3-small` or `text-embedding-3-large` | | `text-embedding-ada-002` | 1 | No earlier than April 3, 2025 | `text-embedding-3-small` or `text-embedding-3-large` | @@ -111,6 +111,14 @@ These models are currently available for use in Azure OpenAI Service. > [!IMPORTANT] > Vision enhancements preview features including Optical Character Recognition (OCR), object grounding, video prompts will be retired and no longer available once `gpt-4` Version: `vision-preview` is upgraded to `turbo-2024-04-09`. If you are currently relying on any of these preview features, this automatic model upgrade will be a breaking change. +## Default model versions + +| Model | Current default version | New default version | Default upgrade date | +|---|---|---|---| +| `gpt-35-turbo` | 0301 | 0125 | Deployments of versions `0301`, `0613`, and `1106` set to [**Auto-update to default**](/azure/ai-services/openai/how-to/working-with-models?tabs=powershell#auto-update-to-default) will be automatically upgraded to version: `0125`, starting on November 15, 2024.| + + + ## Deprecated models These models were deprecated on July 6, 2023 and were retired on June 14, 2024. These models are no longer available for new deployments. Deployments created before July 6, 2023 remain available to customers until June 14, 2024. We recommend customers migrate their applications to deployments of replacement models before the June 14, 2024 retirement. @@ -147,8 +155,13 @@ If you're an existing customer looking for information about these models, see [ | code-search-babbage-code-001 | July 6, 2023 | June 14, 2024 | text-embedding-3-small | | code-search-babbage-text-001 | July 6, 2023 | June 14, 2024 | text-embedding-3-small | + ## Retirement and deprecation history +## September 3, 2024 + +* Updated tables to include information on `gpt-35-turbo` default version upgrades. Deployments of versions `0301`, `0613`, and `1106` set to [**Auto-update to default**](/azure/ai-services/openai/how-to/working-with-models?tabs=powershell#auto-update-to-default) will be automatically upgraded to version: `0125`, starting on November 15, 2024.| + ### August 22, 2024 * Updated `gpt-35-turbo` (0301) retirement date to no earlier than November 1, 2024. diff --git a/articles/ai-services/openai/concepts/models.md b/articles/ai-services/openai/concepts/models.md index 820e24b5ed0..e727beef1cc 100644 --- a/articles/ai-services/openai/concepts/models.md +++ b/articles/ai-services/openai/concepts/models.md @@ -4,7 +4,7 @@ titleSuffix: Azure OpenAI description: Learn about the different model capabilities that are available with Azure OpenAI. ms.service: azure-ai-openai ms.topic: conceptual -ms.date: 08/14/2024 +ms.date: 08/28/2024 ms.custom: references_regions, build-2023, build-2023-dataai, refefences_regions manager: nitinme author: mrbullwinkle #ChrisHMSFT @@ -26,24 +26,6 @@ Azure OpenAI Service is powered by a diverse set of models with different capabi | [Whisper](#whisper-models) | A series of models in preview that can transcribe and translate speech to text. | | [Text to speech](#text-to-speech-models-preview) (Preview) | A series of models in preview that can synthesize text to speech. | -## Early access playground (preview) - -On August 6, 2024, OpenAI [announced](https://openai.com/index/introducing-structured-outputs-in-the-api/) the latest version of their flagship GPT-4o model version `2024-08-06`. GPT-4o `2024-08-06` has all the capabilities of the previous version as well as: - -* An enhanced ability to support complex structured outputs. -* Max output tokens have been increased from 4,096 to 16,384. - -Azure customers can test out GPT-4o `2024-08-06` today in the new AI Studio early access playground (preview). - -Unlike the previous early access playground, the AI Studio early access playground (preview) does not require you to have a resource in a specific region. - -> [!NOTE] -> Prompts and completions made through the early access playground (preview) may be processed in any Azure OpenAI region, and are currently subject to a 10 request per minute per Azure subscription limit. This limit may change in the future. -> -> Azure OpenAI Service abuse monitoring is enabled for all early access playground users even if approved for modification; default content filters are enabled and cannot be modified. - -To test out GPT-4o `2024-08-06`, sign-in to the Azure AI early access playground (preview) using this [link](https://aka.ms/oai/docs/earlyaccessplayground). - ## GPT-4o and GPT-4 Turbo GPT-4o integrates text and images in a single model, enabling it to handle multiple data types simultaneously. This multimodal approach enhances accuracy and responsiveness in human-computer interactions. GPT-4o matches GPT-4 Turbo in English text and coding tasks while offering superior performance in non-English languages and vision tasks, setting new benchmarks for AI capabilities. @@ -56,6 +38,7 @@ You need to [create](../how-to/create-resource.md) or use an existing resource i When your resource is created, you can [deploy](../how-to/create-resource.md#deploy-a-model) the GPT-4o models. If you are performing a programmatic deployment, the **model** names are: +- `gpt-4o` **Version** `2024-08-06` - `gpt-4o`, **Version** `2024-05-13` - `gpt-4o-mini` **Version** `2024-07-18` @@ -83,8 +66,9 @@ See [model versions](../concepts/model-versions.md) to learn about how Azure Ope | Model ID | Description | Max Request (tokens) | Training Data (up to) | | --- | :--- |:--- |:---: | +|`gpt-4o` (2024-08-06)
**GPT-4o (Omni)** | **Latest large GA model**
- Structured outputs
- Text, image processing
- JSON Mode
- parallel function calling
- Enhanced accuracy and responsiveness
- Parity with English text and coding tasks compared to GPT-4 Turbo with Vision
- Superior performance in non-English languages and in vision tasks |Input: 128,000
Output: 4,096| Oct 2023 | |`gpt-4o-mini` (2024-07-18)
**GPT-4o mini** | **Latest small GA model**
- Fast, inexpensive, capable model ideal for replacing GPT-3.5 Turbo series models.
- Text, image processing
- JSON Mode
- parallel function calling | Input: 128,000
Output: 16,384 | Oct 2023 | -|`gpt-4o` (2024-05-13)
**GPT-4o (Omni)** | **Latest large GA model**
- Text, image processing
- JSON Mode
- parallel function calling
- Enhanced accuracy and responsiveness
- Parity with English text and coding tasks compared to GPT-4 Turbo with Vision
- Superior performance in non-English languages and in vision tasks |Input: 128,000
Output: 4,096| Oct 2023 | +|`gpt-4o` (2024-05-13)
**GPT-4o (Omni)** | Text, image processing
- JSON Mode
- parallel function calling
- Enhanced accuracy and responsiveness
- Parity with English text and coding tasks compared to GPT-4 Turbo with Vision
- Superior performance in non-English languages and in vision tasks |Input: 128,000
Output: 4,096| Oct 2023 | | `gpt-4` (turbo-2024-04-09)
**GPT-4 Turbo with Vision** | **New GA model**
- Replacement for all previous GPT-4 preview models (`vision-preview`, `1106-Preview`, `0125-Preview`).
- [**Feature availability**](#gpt-4o-and-gpt-4-turbo) is currently different depending on method of input, and deployment type. | Input: 128,000
Output: 4,096 | Dec 2023 | | `gpt-4` (0125-Preview)*
**GPT-4 Turbo Preview** | **Preview Model**
-Replaces 1106-Preview
- Better code generation performance
- Reduces cases where the model doesn't complete a task
- JSON Mode
- parallel function calling
- reproducible output (preview) | Input: 128,000
Output: 4,096 | Dec 2023 | | `gpt-4` (vision-preview)
**GPT-4 Turbo with Vision Preview** | **Preview model**
- Accepts text and image input.
- Supports enhancements
- JSON Mode
- parallel function calling
- reproducible output (preview) | Input: 128,000
Output: 4,096 | Apr 2023 | @@ -188,6 +172,17 @@ For more information on Provisioned deployments, see our [Provisioned guidance]( ### Global standard model availability +`gpt-4o` **Version:** `2024-08-06` + +**Supported regions:** +- eastus +- eastus2 +- northcentralus +- southcentralus +- swedencentral +- westus +- westus3 + `gpt-4o` **Version:** `2024-05-13` **Supported regions:** diff --git a/articles/ai-services/openai/how-to/json-mode.md b/articles/ai-services/openai/how-to/json-mode.md index eba513720d7..f609c4ee5c7 100644 --- a/articles/ai-services/openai/how-to/json-mode.md +++ b/articles/ai-services/openai/how-to/json-mode.md @@ -6,7 +6,7 @@ services: cognitive-services manager: nitinme ms.service: azure-ai-openai ms.topic: how-to -ms.date: 12/05/2023 +ms.date: 08/29/2024 author: mrbullwinkle ms.author: mbullwin recommendations: false @@ -17,6 +17,9 @@ recommendations: false JSON mode allows you to set the models response format to return a valid JSON object as part of a chat completion. While generating valid JSON was possible previously, there could be issues with response consistency that would lead to invalid JSON objects being generated. +> [!NOTE] +> While JSON mode is still supported, when possible we recommend using [structured outputs](./structured-outputs.md). Like JSON mode structured outputs generates valid JSON, but with the added benefit that you can constrain the model to use a specific JSON schema. + ## JSON mode support JSON mode is only currently supported with the following models: @@ -27,6 +30,8 @@ JSON mode is only currently supported with the following models: * `gpt-35-turbo` (0125) * `gpt-4` (1106-Preview) * `gpt-4` (0125-Preview) +* `gpt-4o` +* `gpt-4o-mini` ### API support @@ -47,7 +52,7 @@ client = AzureOpenAI( ) response = client.chat.completions.create( - model="gpt-4-0125-Preview", # Model = should match the deployment name you chose for your 0125-Preview model deployment + model="YOUR-MODEL_DEPLOYMENT_NAME", # Model = should match the deployment name you chose for your model deployment response_format={ "type": "json_object" }, messages=[ {"role": "system", "content": "You are a helpful assistant designed to output JSON."}, diff --git a/articles/ai-services/openai/how-to/structured-outputs.md b/articles/ai-services/openai/how-to/structured-outputs.md new file mode 100644 index 00000000000..51b2d95cfa4 --- /dev/null +++ b/articles/ai-services/openai/how-to/structured-outputs.md @@ -0,0 +1,743 @@ +--- +title: 'How to use structured outputs with Azure OpenAI Service' +titleSuffix: Azure OpenAI +description: Learn how to improve your model responses with structured outputs +services: cognitive-services +manager: nitinme +ms.service: azure-ai-openai +ms.topic: how-to +ms.date: 08/28/2024 +author: mrbullwinkle +ms.author: mbullwin +recommendations: false +--- + +# Structured outputs + +Structured outputs make a model follow a [JSON Schema](https://json-schema.org/overview/what-is-jsonschema) definition that you provide as part of your inference API call. This is in contrast to the older [JSON mode](./json-mode.md) feature, which guaranteed valid JSON would be generated, but was unable to ensure strict adherence to the supplied schema. Structured outputs is recommended for function calling, extracting structured data, and building complex multi-step workflows. + +## Supported models + +Currently only `gpt-4o` version: `2024-08-06` supports structured outputs. + +## API support + +Support for structured outputs was first added in API version [`2024-08-01-preview`](https://github.com/Azure/azure-rest-api-specs/blob/main/specification/cognitiveservices/data-plane/AzureOpenAI/inference/preview/2024-08-01-preview/inference.json). + +## Getting started + +# [Python (Microsoft Entra ID)](#tab/python-secure) + +You can use [`Pydantic`](https://docs.pydantic.dev/latest/) to define object schemas in Python. Depending on what version of the [OpenAI](https://pypi.org/project/openai/) and [`Pydantic` libraries](https://pypi.org/project/pydantic/) you're running you may need to upgrade to a newer version. These examples were tested against `openai 1.42.0` and `pydantic 2.8.2`. + +```cmd +pip install openai pydantic --upgrade +``` + +If you new to using Microsoft Entra ID for authentication see [How to configure Azure OpenAI Service with Microsoft Entra ID authentication](./managed-identity.md). + +```python +from pydantic import BaseModel +from openai import AzureOpenAI +from azure.identity import DefaultAzureCredential, get_bearer_token_provider + +token_provider = get_bearer_token_provider( + DefaultAzureCredential(), "https://cognitiveservices.azure.com/.default" +) + +client = AzureOpenAI( + azure_endpoint = os.getenv("AZURE_OPENAI_ENDPOINT"), + azure_ad_token_provider=token_provider, + api_version="2024-08-01-preview" +) + + +class CalendarEvent(BaseModel): + name: str + date: str + participants: list[str] + +completion = client.beta.chat.completions.parse( + model="MODEL_DEPLOYMENT_NAME", # replace with the model deployment name of your gpt-4o 2024-08-06 deployment + messages=[ + {"role": "system", "content": "Extract the event information."}, + {"role": "user", "content": "Alice and Bob are going to a science fair on Friday."}, + ], + response_format=CalendarEvent, +) + +event = completion.choices[0].message.parsed + +print(event) +print(completion.model_dump_json(indent=2)) +``` + +### Output + +```json +name='Science Fair' date='Friday' participants=['Alice', 'Bob'] +{ + "id": "chatcmpl-A1EUP2fAmL4SeB1lVMinwM7I2vcqG", + "choices": [ + { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"name\": \"Science Fair\",\n \"date\": \"Friday\",\n \"participants\": [\"Alice\", \"Bob\"]\n}", + "refusal": null, + "role": "assistant", + "function_call": null, + "tool_calls": [], + "parsed": { + "name": "Science Fair", + "date": "Friday", + "participants": [ + "Alice", + "Bob" + ] + } + } + } + ], + "created": 1724857389, + "model": "gpt-4o-2024-08-06", + "object": "chat.completion", + "service_tier": null, + "system_fingerprint": "fp_1c2eaec9fe", + "usage": { + "completion_tokens": 27, + "prompt_tokens": 32, + "total_tokens": 59 + } +} +``` + +# [Python (key-based auth)](#tab/python) + +You can use [`Pydantic`](https://docs.pydantic.dev/latest/) to define object schemas in Python. Depending on what version of the [OpenAI](https://pypi.org/project/openai/) and [`Pydantic` libraries](https://pypi.org/project/pydantic/) you're running you may need to upgrade to a newer version. These examples were tested against `openai 1.42.0` and `pydantic 2.8.2`. + +```cmd +pip install openai pydantic --upgrade +``` + +```python +from pydantic import BaseModel +from openai import AzureOpenAI + +client = AzureOpenAI( + azure_endpoint = os.getenv("AZURE_OPENAI_ENDPOINT"), + api_key=os.getenv("AZURE_OPENAI_API_KEY"), + api_version="2024-08-01-preview" +) + + +class CalendarEvent(BaseModel): + name: str + date: str + participants: list[str] + +completion = client.beta.chat.completions.parse( + model="MODEL_DEPLOYMENT_NAME", # replace with the model deployment name of your gpt-4o 2024-08-06 deployment + messages=[ + {"role": "system", "content": "Extract the event information."}, + {"role": "user", "content": "Alice and Bob are going to a science fair on Friday."}, + ], + response_format=CalendarEvent, +) + +event = completion.choices[0].message.parsed + +print(event) +print(completion.model_dump_json(indent=2)) +``` + +### Output + +```json +name='Science Fair' date='Friday' participants=['Alice', 'Bob'] +{ + "id": "chatcmpl-A1EUP2fAmL4SeB1lVMinwM7I2vcqG", + "choices": [ + { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"name\": \"Science Fair\",\n \"date\": \"Friday\",\n \"participants\": [\"Alice\", \"Bob\"]\n}", + "refusal": null, + "role": "assistant", + "function_call": null, + "tool_calls": [], + "parsed": { + "name": "Science Fair", + "date": "Friday", + "participants": [ + "Alice", + "Bob" + ] + } + } + } + ], + "created": 1724857389, + "model": "gpt-4o-2024-08-06", + "object": "chat.completion", + "service_tier": null, + "system_fingerprint": "fp_1c2eaec9fe", + "usage": { + "completion_tokens": 27, + "prompt_tokens": 32, + "total_tokens": 59 + } +} +``` + +# [REST](#tab/rest) + +`response_format` is set to `json_schema` with `strict: true` set. + +```bash +curl -X POST https://YOUR_RESOURCE_NAME.openai.azure.com/openai/deployments/YOUR_MODEL_DEPLOYMENT_NAME/chat/completions?api-version=2024-08-01-preview \ + -H "api-key: $AZURE_OPENAI_API_KEY" \ + -H "Content-Type: application/json" \ + -d '{ + "messages": [ + {"role": "system", "content": "Extract the event information."}, + {"role": "user", "content": "Alice and Bob are going to a science fair on Friday."} + ], + "response_format": { + "type": "json_schema", + "json_schema": { + "name": "CalendarEventResponse", + "strict": true, + "schema": { + "type": "object", + "properties": { + "name": { + "type": "string" + }, + "date": { + "type": "string" + }, + "participants": { + "type": "array", + "items": { + "type": "string" + } + } + }, + "required": [ + "name", + "date", + "participants" + ], + "additionalProperties": false + } + } + } + }' +``` + +Output: + +```json +{ + "id": "chatcmpl-A1HKsHAe2hH9MEooYslRn9UmEwsag", + "object": "chat.completion", + "created": 1724868330, + "model": "gpt-4o-2024-08-06", + "choices": [ + { + "index": 0, + "message": { + "role": "assistant", + "content": "{\n \"name\": \"Science Fair\",\n \"date\": \"Friday\",\n \"participants\": [\"Alice\", \"Bob\"]\n}" + }, + "logprobs": null, + "finish_reason": "stop" + } + ], + "usage": { + "prompt_tokens": 33, + "completion_tokens": 27, + "total_tokens": 60 + }, + "system_fingerprint": "fp_1c2eaec9fe" +} + +``` + +--- + +## Function calling with structured outputs + +Structured Outputs for function calling can be enabled with a single parameter, by supplying `strict: true`. + +> [!NOTE] +> Structured outputs is not supported with parallel function calls. When using structured outputs set `parallel_tool_calls` to `false`. + +# [Python (Microsoft Entra ID)](#tab/python-secure) + +```python +from enum import Enum +from typing import Union +from pydantic import BaseModel +import openai +from openai import AzureOpenAI + +client = AzureOpenAI( + azure_endpoint = os.getenv("AZURE_OPENAI_ENDPOINT"), + api_key=os.getenv("AZURE_OPENAI_API_KEY"), + api_version="2024-08-01-preview" +) + + +class GetDeliveryDate(BaseModel): + order_id: str + +tools = [openai.pydantic_function_tool(GetDeliveryDate)] + +messages = [] +messages.append({"role": "system", "content": "You are a helpful customer support assistant. Use the supplied tools to assist the user."}) +messages.append({"role": "user", "content": "Hi, can you tell me the delivery date for my order #12345?"}) + +response = client.chat.completions.create( + model="MODEL_DEPLOYMENT_NAME", # replace with the model deployment name of your gpt-4o 2024-08-06 deployment + messages=messages, + tools=tools +) + +print(response.choices[0].message.tool_calls[0].function) +print(response.model_dump_json(indent=2)) +``` + +# [Python (key-based auth)](#tab/python) + +```python +from enum import Enum +from typing import Union +from pydantic import BaseModel +import openai +from openai import AzureOpenAI + +client = AzureOpenAI( + azure_endpoint = os.getenv("AZURE_OPENAI_ENDPOINT"), + api_key=os.getenv("AZURE_OPENAI_API_KEY"), + api_version="2024-08-01-preview" +) + +class GetDeliveryDate(BaseModel): + order_id: str + +tools = [openai.pydantic_function_tool(GetDeliveryDate)] + +messages = [] +messages.append({"role": "system", "content": "You are a helpful customer support assistant. Use the supplied tools to assist the user."}) +messages.append({"role": "user", "content": "Hi, can you tell me the delivery date for my order #12345?"}) + +response = client.chat.completions.create( + model="MODEL_DEPLOYMENT_NAME", # replace with the model deployment name of your gpt-4o 2024-08-06 deployment + messages=messages, + tools=tools +) + +print(response.choices[0].message.tool_calls[0].function) +print(response.model_dump_json(indent=2)) +``` + +# [REST](#tab/rest) + +```bash +curl -X POST https://YOUR_RESOURCE_NAME.openai.azure.com/openai/deployments/YOUR_MODEL_DEPLOYMENT_NAME/chat/completions?api-version=2024-08-01-preview \ + -H "api-key: $AZURE_OPENAI_API_KEY" \ + -H "Content-Type: application/json" \ + -d '{ + "messages": [ + { + "role": "system", + "content": "You are a helpful assistant. The current date is August 6, 2024. You help users query for the data they are looking for by calling the query function." + }, + { + "role": "user", + "content": "look up all my orders in may of last year that were fulfilled but not delivered on time" + } + ], + "tools": [ + { + "type": "function", + "function": { + "name": "query", + "description": "Execute a query.", + "strict": true, + "parameters": { + "type": "object", + "properties": { + "table_name": { + "type": "string", + "enum": ["orders"] + }, + "columns": { + "type": "array", + "items": { + "type": "string", + "enum": [ + "id", + "status", + "expected_delivery_date", + "delivered_at", + "shipped_at", + "ordered_at", + "canceled_at" + ] + } + }, + "conditions": { + "type": "array", + "items": { + "type": "object", + "properties": { + "column": { + "type": "string" + }, + "operator": { + "type": "string", + "enum": ["=", ">", "<", ">=", "<=", "!="] + }, + "value": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "number" + }, + { + "type": "object", + "properties": { + "column_name": { + "type": "string" + } + }, + "required": ["column_name"], + "additionalProperties": false + } + ] + } + }, + "required": ["column", "operator", "value"], + "additionalProperties": false + } + }, + "order_by": { + "type": "string", + "enum": ["asc", "desc"] + } + }, + "required": ["table_name", "columns", "conditions", "order_by"], + "additionalProperties": false + } + } + } + ] +}' +``` + +--- + +## Supported schemas and limitations + +Azure OpenAI structured outputs support the same subset of the [JSON Schema](https://json-schema.org/docs) as OpenAI. + +### Supported types + +- String +- Number +- Boolean +- Integer +- Object +- Array +- Enum +- anyOf + +> [!NOTE] +> Root objects cannot be the `anyOf` type. + +### All fields must be required + +All fields or function parameters must be included as required. In the example below `location`, and `unit` are both specified under `"required": ["location", "unit"]`. + +```json +{ + "name": "get_weather", + "description": "Fetches the weather in the given location", + "strict": true, + "parameters": { + "type": "object", + "properties": { + "location": { + "type": "string", + "description": "The location to get the weather for" + }, + "unit": { + "type": "string", + "description": "The unit to return the temperature in", + "enum": ["F", "C"] + } + }, + "additionalProperties": false, + "required": ["location", "unit"] + } +``` + +If needed, it's possible to emulate an optional parameter by using a union type with `null`. In this example, this is achieved with the line `"type": ["string", "null"],`. + +```json +{ + "name": "get_weather", + "description": "Fetches the weather in the given location", + "strict": true, + "parameters": { + "type": "object", + "properties": { + "location": { + "type": "string", + "description": "The location to get the weather for" + }, + "unit": { + "type": ["string", "null"], + "description": "The unit to return the temperature in", + "enum": ["F", "C"] + } + }, + "additionalProperties": false, + "required": [ + "location", "unit" + ] + } +} +``` + +### Nesting depth + +A schema may have up to 100 object properties total, with up to five levels of nesting + +### additionalProperties: false must always be set in objects + +This property controls if an object can have additional key value pairs that weren't defined in the JSON Schema. In order to use structured outputs, you must set this value to false. + +### Key ordering + +Structured outputs are ordered the same as the provided schema. To change the output order, modify the order of the schema that you send as part of your inference request. + +### Unsupported type-specific keywords + +| Type | Unsupported Keyword +|---|----| +| String | minlength
maxLength
pattern
format | +| Number | minimum
maximum
multipleOf | +| Objects | patternProperties
unevaluatedProperties
propertyNames
minProperties
maxProperties | +| Arrays | unevaluatedItems
contains
minContains
maxContains
minItems
maxItems
uniqueItems | + +### Nested schemas using anyOf must adhere to the overall JSON Schema subset + +Example supported `anyOf` schema: + +```json +{ + "type": "object", + "properties": { + "item": { + "anyOf": [ + { + "type": "object", + "description": "The user object to insert into the database", + "properties": { + "name": { + "type": "string", + "description": "The name of the user" + }, + "age": { + "type": "number", + "description": "The age of the user" + } + }, + "additionalProperties": false, + "required": [ + "name", + "age" + ] + }, + { + "type": "object", + "description": "The address object to insert into the database", + "properties": { + "number": { + "type": "string", + "description": "The number of the address. Eg. for 123 main st, this would be 123" + }, + "street": { + "type": "string", + "description": "The street name. Eg. for 123 main st, this would be main st" + }, + "city": { + "type": "string", + "description": "The city of the address" + } + }, + "additionalProperties": false, + "required": [ + "number", + "street", + "city" + ] + } + ] + } + }, + "additionalProperties": false, + "required": [ + "item" + ] +} +``` + +### Definitions are supported + +Supported example: + +```json +{ + "type": "object", + "properties": { + "steps": { + "type": "array", + "items": { + "$ref": "#/$defs/step" + } + }, + "final_answer": { + "type": "string" + } + }, + "$defs": { + "step": { + "type": "object", + "properties": { + "explanation": { + "type": "string" + }, + "output": { + "type": "string" + } + }, + "required": [ + "explanation", + "output" + ], + "additionalProperties": false + } + }, + "required": [ + "steps", + "final_answer" + ], + "additionalProperties": false +} +``` + +### Recursive schemas are supported + +Example using # for root recursion: + +```json +{ + "name": "ui", + "description": "Dynamically generated UI", + "strict": true, + "schema": { + "type": "object", + "properties": { + "type": { + "type": "string", + "description": "The type of the UI component", + "enum": ["div", "button", "header", "section", "field", "form"] + }, + "label": { + "type": "string", + "description": "The label of the UI component, used for buttons or form fields" + }, + "children": { + "type": "array", + "description": "Nested UI components", + "items": { + "$ref": "#" + } + }, + "attributes": { + "type": "array", + "description": "Arbitrary attributes for the UI component, suitable for any element", + "items": { + "type": "object", + "properties": { + "name": { + "type": "string", + "description": "The name of the attribute, for example onClick or className" + }, + "value": { + "type": "string", + "description": "The value of the attribute" + } + }, + "additionalProperties": false, + "required": ["name", "value"] + } + } + }, + "required": ["type", "label", "children", "attributes"], + "additionalProperties": false + } + } +``` + +Example of explicit recursion: + +```json +{ + "type": "object", + "properties": { + "linked_list": { + "$ref": "#/$defs/linked_list_node" + } + }, + "$defs": { + "linked_list_node": { + "type": "object", + "properties": { + "value": { + "type": "number" + }, + "next": { + "anyOf": [ + { + "$ref": "#/$defs/linked_list_node" + }, + { + "type": "null" + } + ] + } + }, + "additionalProperties": false, + "required": [ + "next", + "value" + ] + } + }, + "additionalProperties": false, + "required": [ + "linked_list" + ] +} +``` \ No newline at end of file diff --git a/articles/ai-services/openai/includes/ai-search-ingestion.md b/articles/ai-services/openai/includes/ai-search-ingestion.md index 6e42cd6aac4..a8c0132f884 100644 --- a/articles/ai-services/openai/includes/ai-search-ingestion.md +++ b/articles/ai-services/openai/includes/ai-search-ingestion.md @@ -12,7 +12,7 @@ author: aahill Data is ingested into Azure AI search using the following process: -1. Ingestion assets are created in Azure AI Search resource and Azure storage account. Currently these assets are: indexers, indexes, data sources, a [custom skill](/azure/search/cognitive-search-custom-skill-interface) in the search resource, and a container (later called the chunks container) in the Azure storage account. You can specify the input Azure storage container using the [Azure OpenAI studio](https://oai.azure.com/), or the [ingestion API (preview)](/rest/api/azureopenai/ingestion-jobs). +1. Ingestion assets are created in Azure AI Search resource and Azure storage account. Currently these assets are: indexers, indexes, data sources, a [custom skill](/azure/search/cognitive-search-custom-skill-interface) in the search resource, and a container (later called the chunks container) in the Azure storage account. You can specify the input Azure storage container using the [Azure OpenAI studio](https://oai.azure.com/), or the [ingestion API (preview)](/rest/api/azureopenai/ingestion-jobs). By default, text is assumed to use the UTF-8 encoding. To specify a different encoding, use the encoding configuration property. See the [.NET documentation](/dotnet/fundamentals/runtime-libraries/system-text-encoding#list-of-encodings) for a list of supported encodings. 2. Data is read from the input container, contents are opened and chunked into small chunks with a maximum of 1,024 tokens each. If vector search is enabled, the service calculates the vector representing the embeddings on each chunk. The output of this step (called the "preprocessed" or "chunked" data) is stored in the chunks container created in the previous step. diff --git a/articles/ai-services/openai/toc.yml b/articles/ai-services/openai/toc.yml index 674a657c67c..ea43cfb3053 100644 --- a/articles/ai-services/openai/toc.yml +++ b/articles/ai-services/openai/toc.yml @@ -124,6 +124,8 @@ items: href: ./how-to/json-mode.md - name: Reproducible output href: ./how-to/reproducible-output.md + - name: Structured outputs + href: ./how-to/structured-outputs.md - name: Work with code href: ./how-to/work-with-code.md - name: Use with large datasets diff --git a/articles/ai-services/openai/whats-new.md b/articles/ai-services/openai/whats-new.md index 7ebc84a31d1..6dbf98821fc 100644 --- a/articles/ai-services/openai/whats-new.md +++ b/articles/ai-services/openai/whats-new.md @@ -10,7 +10,7 @@ ms.custom: - ignite-2023 - references_regions ms.topic: whats-new -ms.date: 08/30/2024 +ms.date: 9/03/2024 recommendations: false --- @@ -20,6 +20,11 @@ This article provides a summary of the latest releases and major documentation u ## August 2024 +### GPT-4o 2024-08-06 structured outputs + +- Available for standard and global deployments in [all US regions and Sweden Central](./concepts/models.md#global-standard-model-availability). +- This model adds support for [structured outputs](https://aka.ms/oai/docs/structured-outputs) when used in conjunction with the new `2024-08-01-preview` inference API. + ### GPT-4o mini provisioned deployments GPT-4o mini is now available for provisioned deployments in Canada East, East US, East US2, North Central US, and Sweden Central. diff --git a/articles/machine-learning/how-to-manage-files.md b/articles/machine-learning/how-to-manage-files.md index 1de3b1d7467..bfc1579ae9e 100644 --- a/articles/machine-learning/how-to-manage-files.md +++ b/articles/machine-learning/how-to-manage-files.md @@ -1,7 +1,7 @@ --- title: Create and manage files in your workspace titleSuffix: Azure Machine Learning -description: Learn how create and manage files in your workspace in Azure Machine Learning studio. +description: In this article, you learn how create and manage files in your workspace in Azure Machine Learning studio. services: machine-learning author: sdgilley ms.author: sgilley @@ -9,20 +9,23 @@ ms.reviewer: kritifaujdar ms.service: azure-machine-learning ms.subservice: core ms.topic: how-to -ms.date: 03/25/2024 +ms.date: 09/03/2024 #Customer intent: As a data scientist, I want to create and manage the files in my workspace in Azure Machine Learning studio. --- # How to create and manage files in your workspace -Learn how to create and manage the files in your Azure Machine Learning workspace. These files are stored in the default workspace storage. Files and folders can be shared with anyone else with read access to the workspace, and can be used from any compute instances in the workspace. +Learn how to create and manage the files in your Azure Machine Learning workspace. These files are stored in the default workspace storage. Files and folders can be shared with anyone else with read access to the workspace, and can be used from any compute instances in the workspace. ## Prerequisites * An Azure subscription. If you don't have an Azure subscription, create a [free account](https://azure.microsoft.com/free/) before you begin. * A Machine Learning workspace. [Create workspace resources](quickstart-create-resources.md). -## Create files +> [!TIP] +> This article shows how to manage files inside Azure Machine Learning studio. You can also connect to your workspace files from Visual Studio Code. To connect to your workspace files from VS Code, see [Work in VS Code remotely connected to a compute instance](how-to-work-in-vs-code-remote.md). + +## Create files in studio To create a new file in your default folder (`Users > yourname`): @@ -37,10 +40,10 @@ To create a new file in your default folder (`Users > yourname`): 1. Select a file type. 1. Select **Create**. -Notebooks and most text file types display in the preview section. Most other file types don't have a preview. +Notebooks and most text file types display in the preview section. Most other file types don't have a preview. > [!TIP] -> If you don't see the correct preview for a notebook, make sure it has `.ipynb` as its extension. Hover over the filename in the list to select **...** if you need to rename the file. +> If you don't see the correct preview for a notebook, make sure it has `.ipynb` as its extension. Hover over the filename in the list to select **...** if you need to rename the file. To create a new file in a different folder: @@ -48,17 +51,17 @@ To create a new file in a different folder: 1. Select **Create new file**. > [!IMPORTANT] -> Content in notebooks and scripts can potentially read data from your sessions and access data without your organization in Azure. Only load files from trusted sources. For more information, see [Secure code best practices](concept-secure-code-best-practice.md#azure-machine-learning-studio-notebooks). +> Content in notebooks and scripts can potentially read data from your sessions and access data without your organization in Azure. Only load files from trusted sources. For more information, see [Secure code best practices](concept-secure-code-best-practice.md#azure-machine-learning-studio-notebooks). ## Customize your file editing experience -In the Azure Machine Learning studio file editor, you can customize your editing experience with Command Palette and relevant keyboard shortcuts. When you invoke the Command Palette, you will see a selectable list of many options to customize your editing experience. +In the Azure Machine Learning studio file editor, you can customize your editing experience with Command Palette and relevant keyboard shortcuts. When you invoke the Command Palette, you see a selectable list of many options to customize your editing experience. :::image type="content" source="media/how-to-manage-files/command-palette.png" alt-text="Screenshot shows command palette in file editor."::: To invoke the Command Palette on a file, either use **F1** or right-select anywhere in the editing space and select **Command Palette** from the menu. -For example, choose "Indent using spaces" if you want your editor to auto-indent with spaces instead of tabs. Take a few moments to explore the different options you have in the Command Palette. +For example, choose "Indent using spaces" if you want your editor to autoindent with spaces instead of tabs. Take a few moments to explore the different options you have in the Command Palette. ## Manage files with Git @@ -67,23 +70,23 @@ For example, choose "Indent using spaces" if you want your editor to auto-indent ## Clone samples -Your workspace contains a **Samples** folder with notebooks designed to help you explore the SDK and serve as examples for your own machine learning projects. Clone these notebooks into your own folder to run and edit them. +Your workspace contains a **Samples** folder with notebooks designed to help you explore the SDK and serve as examples for your own machine learning projects. Clone these notebooks into your own folder to run and edit them. ## Share files -Copy and paste the URL to share a file. Only other users of the workspace can access this URL. Learn more about [granting access to your workspace](how-to-assign-roles.md). +Copy and paste the URL to share a file. Only other users of the workspace can access this URL. Learn more about [granting access to your workspace](how-to-assign-roles.md). ## Delete a file -You *can't* delete the **Samples** files. These files are part of the studio and are updated each time a new SDK is published. +You *can't* delete the **Samples** files. These files are part of the studio and are updated each time a new SDK is published. You *can* delete files found in your **Files** section in any of these ways: -* In the studio, select the **...** at the end of a folder or file. Make sure to use a supported browser (Microsoft Edge, Chrome, or Firefox). +* In the studio, select the **...** at the end of a folder or file. Make sure to use a supported browser (Microsoft Edge, Chrome, or Firefox). * [Use a terminal](how-to-access-terminal.md) from any compute instance in your workspace. The folder **~/cloudfiles** is mapped to storage on your workspace storage account. * In either Jupyter or JupyterLab with their tools. -## Next steps +## Related content * [Run Jupyter notebooks in your workspace](how-to-run-jupyter-notebooks.md) * [Access a compute instance terminal in your workspace](how-to-access-terminal.md) \ No newline at end of file diff --git a/articles/machine-learning/media/how-to-manage-files/command-palette.png b/articles/machine-learning/media/how-to-manage-files/command-palette.png index 4eea1c07dbf..24dcf244c05 100644 Binary files a/articles/machine-learning/media/how-to-manage-files/command-palette.png and b/articles/machine-learning/media/how-to-manage-files/command-palette.png differ diff --git a/articles/machine-learning/media/how-to-run-jupyter-notebooks/create-new-file.png b/articles/machine-learning/media/how-to-run-jupyter-notebooks/create-new-file.png index 3a68a9d666d..c5571200ba6 100644 Binary files a/articles/machine-learning/media/how-to-run-jupyter-notebooks/create-new-file.png and b/articles/machine-learning/media/how-to-run-jupyter-notebooks/create-new-file.png differ