Use model router for Microsoft Foundry

Model router is a trained language model that selects the best large language model (LLM) to respond to a prompt in real time. It uses different preexisting models to deliver high performance and save on compute costs, all in one model deployment. To learn more about how model router works, its advantages, and limitations, see the Model router concepts guide. To understand the architecture and routing logic, see How model router works.

Supported models

• ¹Requires registration.
• ²Model router support is in preview.
• ³Model router support is in preview. Requires deployment of model for use with Model router.

Deploy a model router model

Model router is packaged as a single Foundry model that you deploy. Start by following the steps in the resource deployment guide.

To deploy programmatically without the portal, use the REST API examples in the deployment sections that follow.

By default, model router deploys with the Balanced routing mode and routes across the full supported model set. You only need to change the routing mode or select a model subset when you want custom routing behavior.

Default deployment

Go to the Microsoft Foundry portal and navigate to the model catalog. Find model-router in the Models list and select it. Choose Default settings for the Balanced routing mode and route between all supported models.

Before you run the REST examples, sign in with Azure CLI and save a management-plane bearer token as AZURE_AI_AUTH_TOKEN.

export AZURE_AI_AUTH_TOKEN=$(az account get-access-token --resource https://management.azure.com --query accessToken -o tsv)

Deploy model router programmatically with the Azure Management REST API. The following example creates a default deployment and relies on the built-in Balanced routing mode and full supported model set.

curl -X PUT "https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/my-resource-group/providers/Microsoft.CognitiveServices/accounts/my-foundry-account/deployments/model-router-deployment?api-version=2025-10-01-preview" \
    -H "Content-Type: application/json" \
    -H "Authorization: Bearer $AZURE_AI_AUTH_TOKEN" \
    -d '{
        "sku": {"name": "GlobalStandard", "capacity": 10},
        "properties": {
                "model": {"format": "OpenAI", "name": "model-router", "version": "2025-11-18"}
        }
}'

Optional: customize deployment settings

To enable more configuration options, choose Custom settings.

Optional: change the routing mode

Sign in to Microsoft Foundry. Make sure the New Foundry toggle is on. These steps refer to Foundry (new).

Use the Routing mode dropdown to select a routing profile. This sets the routing logic for your deployment.

When to use each mode:

• Balanced (default): Most workloads. Optimizes cost while maintaining quality.
• Quality: Critical tasks like legal review, medical summaries, or complex reasoning.
• Cost: High-volume, budget-sensitive workloads like content classification or simple Q&A.

Optional: route to a model subset

Sign in to Microsoft Foundry. Make sure the New Foundry toggle is on. These steps refer to Foundry (new).

The latest version of model router supports custom subsets: you can specify which underlying models to include in routing decisions. This gives you more control over cost, compliance, and performance characteristics.

In the model router deployment pane, select Route to a subset of models. Then select the underlying models you want to enable. You must select at least one model for routing. If no models are selected, the deployment uses the default model set for your routing mode.

New models introduced later are excluded by default until explicitly added.

Configure custom settings with the REST API

Use the following example when you want to set both the routing mode and a model subset in the same deployment request.

Add a routing block only when you want to override the default Balanced mode or restrict the routed model set. The following example keeps the combined custom request with both a routing mode and a model subset.

curl -X PUT "https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/my-resource-group/providers/Microsoft.CognitiveServices/accounts/my-foundry-account/deployments/model-router-deployment?api-version=2025-10-01-preview" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $AZURE_AI_AUTH_TOKEN" \
  -d '{
    "sku": {"name": "GlobalStandard", "capacity": 10},
    "properties": {
        "model": {"format": "OpenAI", "name": "model-router", "version": "2025-11-18"},
        "routing": {
            "mode": "balanced",
            "models": [
                {"format": "OpenAI", "name": "gpt-4.1", "version": "2025-04-14"},
                {"format": "OpenAI", "name": "gpt-5.2-chat", "version": "2025-12-11"},
                {"format": "Meta", "name": "Llama-4-Maverick-17B-128E-Instruct-FP8", "version": "1"}
            ]
        }
    }
}'

Test model router with Foundry Responses and Chat Completions

Call model router the same way you call any OpenAI chat model. Set the model parameter to the name of your model router deployment. You can use either the Microsoft Foundry SDK with the Responses API or the OpenAI Python SDK with the Chat Completions API.

with (
    DefaultAzureCredential() as credential,
    AIProjectClient(endpoint=project_endpoint, credential=credential) as project_client,
    project_client.get_openai_client() as openai_client,
):
    response = openai_client.responses.create(
        model=deployment,
        input="In one sentence, name the most popular tourist destination in Seattle.",
    )

client = AzureOpenAI(
    azure_endpoint=endpoint,
    api_key=api_key,
    api_version="2024-10-21",
)

response = client.chat.completions.create(
    model=deployment,
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {
            "role": "user",
            "content": "In one sentence, name the most popular tourist destination in Seattle.",
        },
    ],
)

• Reference: AzureOpenAI (OpenAI Python SDK)
• Reference: AIProjectClient

Test model router in the playground

In the Foundry portal, go to your model router deployment on the Models + endpoints page and select it to open the model playground. In the playground, enter messages and see the model's responses. Each response shows which underlying model the router selected.

Connect model router to a Foundry agent

Sign in to Microsoft Foundry. Make sure the New Foundry toggle is on. These steps refer to Foundry (new).

If you've created an AI agent in Foundry, you can connect your model router deployment to be used as the agent's base model. Select it from the model dropdown menu in the agent playground. Your agent will have all the tools and instructions you've configured for it, but the underlying model that processes its responses will be selected by model router.

Output format

The JSON response you receive from a model router model is identical to the standard chat completions API response. Note that the "model" field reveals which underlying model was selected to respond to the prompt.

The following example response was generated using API version 2025-11-18:

{
    "success": true,
    "data": {
        "choices": [
            {
                "content_filter_results": {
                    "hate": {
                        "filtered": false,
                        "severity": "safe"
                    },
                    "protected_material_code": {
                        "filtered": false,
                        "detected": false
                    },
                    "protected_material_text": {
                        "filtered": false,
                        "detected": false
                    },
                    "self_harm": {
                        "filtered": false,
                        "severity": "safe"
                    },
                    "sexual": {
                        "filtered": false,
                        "severity": "safe"
                    },
                    "violence": {
                        "filtered": false,
                        "severity": "safe"
                    }
                },
                "finish_reason": "stop",
                "index": 0,
                "logprobs": null,
                "message": {
                    "annotations": [],
                    "content": "Charismatic and bold—combining brash showmanship and poetic wit with fierce competitiveness, moral conviction, and unwavering activism.",
                    "refusal": null,
                    "role": "assistant"
                }
            }
        ],
        "created": 1774543376,
        "id": "xxxx-yyyy-zzzz",
        "model": "gpt-5-mini-2025-08-07",
        "object": "chat.completion",
        "prompt_filter_results": [
            {
                "prompt_index": 0,
                "content_filter_results": {
                    "hate": {
                        "filtered": false,
                        "severity": "safe"
                    },
                    "jailbreak": {
                        "filtered": false,
                        "detected": false
                    },
                    "self_harm": {
                        "filtered": false,
                        "severity": "safe"
                    },
                    "sexual": {
                        "filtered": false,
                        "severity": "safe"
                    },
                    "violence": {
                        "filtered": false,
                        "severity": "safe"
                    }
                }
            }
        ],
        "system_fingerprint": null,
        "usage": {
            "completion_tokens": 163,
            "completion_tokens_details": {
                "accepted_prediction_tokens": 0,
                "audio_tokens": 0,
                "reasoning_tokens": 128,
                "rejected_prediction_tokens": 0
            },
            "prompt_tokens": 3254,
            "prompt_tokens_details": {
                "audio_tokens": 0,
                "cached_tokens": 3200
            },
            "total_tokens": 3417
        }
    }
}

Monitor model router metrics

Monitor performance

Monitor the performance of your model router deployment in Azure Monitor (AzMon) in the Azure portal.

1. Go to the Monitoring > Metrics page for your Azure OpenAI resource in the Azure portal.
2. Filter by the deployment name of your model router model.
3. Split the metrics by underlying models if needed.

Monitor costs

You can monitor the costs of model router, which is the sum of the costs incurred by the underlying models.

1. Visit the Resource Management -> Cost analysis page in the Azure portal.
2. If needed, filter by Azure resource.
3. Then, filter by deployment name: Filter by "Tag", select Deployment as the type of the tag, and then select your model router deployment name as the value.

Troubleshoot model router

Common issues

Error codes

For API error codes and troubleshooting, see the Azure OpenAI REST API reference.

Resources

The following open-source repositories demonstrate model router in different scenarios. Each repo is on GitHub — learn, fork, and extend to accelerate your learning. Most samples require an existing model router deployment; see Deploy a model router model to get started.

Next steps

• Model router concepts - Learn how routing modes work
• Quotas and limits - Rate limits for model router
• Create an agent - Use model router with Foundry agents