Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
A dedicated cluster in Azure Monitor provides advanced security and control capabilities, and cost optimization. You can link new or existing workspaces to a dedicated cluster without interrupting ingestion and query operations.
Advanced capabilities
Azure Monitor Logs is a fully managed, cloud‑scale service designed to automatically handle ingestion, indexing, and querying across large and fluctuating workloads. Its underlying engine employs built‑in mechanisms that optimize query execution, distribute processing, and automatically scale resources seamlessly without user intervention. This high performing service is the framework that default Log Analytics workspaces, or shared clusters, are built on. The following additional capabilities are unlocked when you create a dedicated cluster:
- Customer-managed keys - Encrypt data by using a key that you provide and control.
- Lockbox - Control Microsoft support engineer access to your data.
- Double encryption - Extra layer of encryption for your data.
- Cross-workspace optimization - Cross-workspace queries run faster when on the same cluster.
- Cost optimization - Link workspaces in the same region to the cluster, and enjoy a commitment tier discount for data ingested from all linked workspaces.
- Availability zones - Protect your data with datacenters in different physical locations, equipped with independent power, cooling, and networking. Azure Monitor availability zones extends your Azure Monitor resilience automatically. Azure Monitor enables dedicated clusters for availability zones (
isAvailabilityZonesEnabled: 'true') by default in all regions that support availability zones. Availability zone supported regions include support for dedicated clusters and shared clusters. - Ingest from Azure Event Hubs - Lets you ingest data directly from Event Hubs into a Log Analytics workspace.
Note
Dedicated clusters aren't a general way to make all queries faster. As with any large analytical system, running queries across very large datasets requires extra compute resources and might impact query performance. For better query performance beyond the cross-workspace optimization of dedicated clusters, optimize your queries. This strategy is especially effective with large datasets and when querying over long time ranges.
Cluster pricing model
Log Analytics dedicated clusters use a commitment tier pricing model starting at 100 GB per day. Ingestion that exceeds the commitment tier level is charged based on the per-GB rate. You can increase a commitment tier at any time, but it has a 31-day commitment period before it can be reduced. See Azure Monitor Logs pricing details for details on commitment tiers.
The cluster billing type has two possible values:
- Cluster (default) - The costs for your cluster are attributed to the cluster resource.
- Workspaces - The costs for your cluster are attributed proportionately to the workspaces in the Cluster, with the cluster resource being billed some of the usage if the total ingested data for the day is under the commitment tier. See Log Analytics Dedicated Clusters to learn more about the cluster pricing model.
Required permissions
To perform cluster-related actions, you need these permissions:
| Action | Permissions or role needed |
|---|---|
| Create a dedicated cluster | Microsoft.Resources/deployments/* and Microsoft.OperationalInsights/clusters/write permissions, as provided by the Log Analytics Contributor built-in role, for example |
| Change cluster properties | Microsoft.OperationalInsights/clusters/write permissions, as provided by the Log Analytics Contributor built-in role, for example |
| Link workspaces to a cluster | Microsoft.OperationalInsights/clusters/write, Microsoft.OperationalInsights/workspaces/write, and Microsoft.OperationalInsights/workspaces/linkedservices/write permissions, as provided by the Log Analytics Contributor built-in role, for example |
| Check workspace link status | Microsoft.OperationalInsights/workspaces/read permissions to the workspace, as provided by the Log Analytics Reader built-in role, for example |
| Get clusters or check a cluster's provisioning status | Microsoft.OperationalInsights/clusters/read permissions, as provided by the Log Analytics Reader built-in role, for example |
| Update commitment tier or billingType in a cluster | Microsoft.OperationalInsights/clusters/write permissions, as provided by the Log Analytics Contributor built-in role, for example |
| Grant the required permissions | Owner or Contributor role that has */write permissions, or the Log Analytics Contributor built-in role, which has Microsoft.OperationalInsights/* permissions |
| Unlink a workspace from cluster | Microsoft.OperationalInsights/workspaces/linkedServices/delete permissions, as provided by the Log Analytics Contributor built-in role, for example |
| Delete a dedicated cluster | Microsoft.OperationalInsights/clusters/delete permissions, as provided by the Log Analytics Contributor built-in role, for example |
For more information on Log Analytics permissions, see Manage access to log data and workspaces in Azure Monitor.
Resource Manager template samples
This article includes sample Azure Resource Manager (ARM) templates to create and configure Log Analytics clusters in Azure Monitor. Each sample includes a template file and a parameters file with sample values to provide to the template.
Note
See Azure Resource Manager samples for Azure Monitor for a list of samples that are available and guidance on deploying them in your Azure subscription.
Template references
Preparation
Cluster commitment tier billing starts as soon as you create the cluster, regardless of data ingestion. Have the following items ready before you start:
- The subscription for creating the cluster.
- A list of workspaces that you want to link to the cluster. These workspaces must be in the same region as the cluster.
- A decision on the billing type and attribution, whether to set to the cluster (default) or to the linked workspaces proportionally.
- Verification of your permissions to create a cluster and link workspaces.
Note
- Cluster creation and linking workspaces are asynchronous operations that can take a few hours to complete.
- Linking or unlinking workspaces from a cluster has no effect on ingestion or queries during the operations.
Create a dedicated cluster
Provide the following properties when creating a new dedicated cluster:
ClusterName: Must be unique for the resource group.
ResourceGroupName: Use a central IT resource group because many teams in the organization usually share clusters. For more design considerations, review Design a Log Analytics workspace configuration.
Location
SkuCapacity: Valid commitment tiers are 100, 200, 300, 400, 500, 1000, 2000, 5000, 10000, 25000, or 50000 GB per day. The minimum commitment tier supported in the CLI and deployment templates is 500 GB. Use the REST API to configure commitment tiers lower than 500 GB. For more information on cluster costs, see Dedicated clusters.
Managed identity: Clusters support two managed identity types:
System-assigned managed identity - Generated automatically with the cluster creation when identity
typeis set to "SystemAssigned". Use this identity to grant storage access to your Key Vault for wrap and unwrap operations.Identity in Cluster's REST Call
{ "identity": { "type": "SystemAssigned" } }User-assigned managed identity - By using this identity, you can configure a customer-managed key at cluster creation, when granting it permissions in your Key Vault before cluster creation.
Identity in Cluster's REST Call
{ "identity": { "type": "UserAssigned", "userAssignedIdentities": { "subscriptions/<subscription-id>/resourcegroups/<resource-group-name>/providers/Microsoft.ManagedIdentity/UserAssignedIdentities/<cluster-assigned-managed-identity>" } } }
After you create your cluster resource, you can edit properties such as sku, keyVaultProperties, or billingType. See more details below.
Deleted clusters take two weeks to be completely removed. You can have up to seven clusters per subscription and region - five active, and two deleted in the past two weeks.
Note
Creating a cluster involves multiple resources and the operation typically completes in two hours. A dedicated cluster is billed once provisioned regardless of data ingestion. Prepare the deployment to expedite the provisioning and workspaces link to the cluster. Verify the following:
- A list of initial workspaces to be linked to the cluster is identified
- You have permissions to the subscription intended for the cluster and any workspace to be linked
Select Create in the Log Analytics dedicated clusters menu in the Azure portal. You're prompted for details such as the name of the cluster and the commitment tier.
Check cluster provisioning status
Provisioning the Log Analytics cluster takes a while to complete. Use one of the following methods to check the ProvisioningState property. The value is Creating while provisioning and Succeeded when completed.
The portal provides a status as the cluster is being provisioned.
Link a workspace to a cluster
Note
- Only link a workspace after the portal finishes provisioning the Log Analytics cluster.
- Linking a workspace to a cluster syncs multiple backend components and cache hydration, which typically completes in two hours.
- When you link a Log Analytics workspace, the workspace billing plan changes to LACluster. Remove the SKU in the workspace template to prevent a conflict during workspace deployment.
- Other than the billing aspects that the cluster plan governs, all workspace configurations and query aspects remain unchanged during and after the link.
You need 'write' permissions to both the workspace and the cluster resource for the workspace link operation:
- In the workspace: Microsoft.OperationalInsights/workspaces/write
- In the cluster resource: Microsoft.OperationalInsights/clusters/write
After you link a Log Analytics workspace to a dedicated cluster, new data you send to the workspace goes to your dedicated cluster, while previously ingested data stays in the Log Analytics cluster. Linking a workspace doesn't affect workspace operation, including ingestion and query experiences. The Log Analytics query engine automatically stitches data from old and new clusters, so the results of queries are complete.
Clusters are regional and can link to up to 1,000 workspaces located in the same region as the cluster. To prevent data fragmentation, you can't link a workspace to a cluster more than twice a month.
Linked workspaces can be in different subscriptions from the subscription the cluster is in. If you use Azure Lighthouse to map both of them to a single tenant, the workspace and cluster can be in different tenants.
When you configure a dedicated cluster with a customer-managed key (CMK), the newly ingested data is encrypted with your key, while older data remains encrypted with a Microsoft-managed key (MMK). Log Analytics abstracts the key configuration, and queries across old and new data encryptions are performed seamlessly.
Use the following steps to link a workspace to a cluster. You can use automation for linking multiple workspaces:
Select your cluster from the Log Analytics dedicated clusters menu in the Azure portal. Select Linked workspaces to view all workspaces currently linked to the dedicated cluster. Select Link workspaces to link additional workspaces.
Check workspace link status
The workspace link operation can take up to 90 minutes to complete. You can check the status on both the linked workspaces and the cluster. When completed, the workspace resources include the clusterResourceId property under features, and the cluster includes linked workspaces under the associatedWorkspaces section.
When you configure a cluster with a customer-managed key, the data ingested is encrypted with your key to the workspaces after the link operation completes.
On the Overview page for your dedicated cluster, select JSON View. The associatedWorkspaces section lists the workspaces linked to the cluster.
Change cluster properties
After you create your cluster resource and it's fully provisioned, you can edit cluster properties by using CLI, PowerShell, or REST API. You can set the following properties after the cluster is provisioned:
- keyVaultProperties - Contains the key in Azure Key Vault with the following parameters: KeyVaultUri, KeyName, KeyVersion. See Update dedicated cluster with Key identifier details.
- Identity - The identity used to authenticate to your Key Vault. This identity can be system-assigned or user-assigned.
- billingType - Billing attribution for the cluster resource and its data. Includes the following values:
- Cluster (default) - The costs for your cluster are attributed to the cluster resource.
- Workspaces - The costs for your cluster are attributed proportionately to the workspaces in the Cluster, with the cluster resource being billed some of the usage if the total ingested data for the day is under the commitment tier. See Log Analytics Dedicated Clusters to learn more about the cluster pricing model.
Important
A single cluster update shouldn't include both identity and key identifier details in the same operation. If you need to update both, make the update in two consecutive operations.
Get all clusters in resource group
From the Log Analytics dedicated clusters menu in the Azure portal, select the Resource group filter.
Get all clusters in subscription
From the Log Analytics dedicated clusters menu in the Azure portal, select the Subscription filter.
Update commitment tier in cluster
When the data volume to linked workspaces changes over time, update the Commitment Tier level to optimize cost. Specify the tier in units of gigabytes (GB). The tier can have values of 100, 200, 300, 400, 500, 1,000, 2,000, 5,000, 10,000, 25,000, or 50,000 GB per day. You don't need to provide the full REST request body, but you must include the SKU.
During the commitment period, you can change to a higher commitment tier, which restarts the 31-day commitment period. You can't move back to pay-as-you-go or to a lower commitment tier until after you finish the commitment period.
Select your cluster from the Log Analytics dedicated clusters menu in the Azure portal. Select Change next to Commitment tier.
Unlink a workspace from cluster
Warning
Unlinking a workspace doesn't move workspace data out of the cluster. Any data collected for a workspace while linked to a dedicated cluster, remains in the cluster for the retention period defined by the workspace, and accessible as long as the cluster isn't deleted.
You can unlink a workspace from a cluster at any time. Here's what happens when a workspace is unlinked
- The workspace pricing tier is changed to per-GB
- Data ingested to the cluster before the unlink operation remains in the cluster
- New data sent to the workspace gets ingested to the workspace, not the dedicated cluster
- Queries aren't affected when a workspace is unlinked - the Log Analytics service performs cross-cluster queries seamlessly
- If the dedicated cluster was configured with a customer-managed key (CMK), data ingested to the workspace while it was linked remains encrypted with your key in the dedicated cluster and accessible as long as your key and permissions to Key Vault remain
Note
- To prevent data distribution across clusters, you can perform only two link operations for a specific workspace within a month. Contact support if you reach the limit.
- Unlinked workspaces move to a pay-as-you-go pricing tier.
Use the following commands to unlink a workspace from cluster:
Select your cluster from Log Analytics dedicated clusters menu in the Azure portal. Select Linked workspaces to view all workspaces currently linked to the dedicated cluster. Select any workspaces you want to unlink and select Unlink.
Delete cluster
You need write permissions on the cluster resource.
Delete a cluster with caution. This operation can't be undone. All ingested data to the cluster from linked workspaces is permanently deleted.
The cluster's billing stops when you delete the cluster, regardless of the 31-day commitment period defined in cluster.
If you delete a cluster that has linked workspaces, the workspaces automatically unlink from the cluster. They move to a pay-as-you-go pricing tier, and new data sent to the workspaces is ingested to Log Analytics clusters instead. You can query a workspace across the time range before it was linked to the cluster, and after the unlink, the service performs cross-cluster queries seamlessly.
Note
- There's a limit of seven clusters per subscription and region: five active clusters, plus two that were deleted in the past two weeks.
- A cluster's name remains reserved two weeks after deletion during which you can't use it for creating a new cluster.
Use the following commands to delete a cluster:
Select your cluster from the Log Analytics dedicated clusters menu in the Azure portal. Then select Delete.
Change managed identity type
You can change the identity type after creating the cluster without interrupting ingestion or queries. Consider the following:
- Updating
SystemAssignedtoUserAssigned- Grant theUserAssignidentity in Key Vault, and then update the identity type in the cluster. - Updating
UserAssignedtoSystemAssigned- Since the system-assigned managed identity is created after updating the cluster identity type withSystemAssigned, follow these steps:- Update the cluster to remove the key - set
keyVaultUri,keyName, andkeyVersionto value"". - Update the cluster identity type to
SystemAssigned. - Update Key Vault and grant permissions to the identity.
- Update key in dedicated cluster.
- Update the cluster to remove the key - set
Limits and constraints
You can create up to five active clusters in each region and subscription.
You can have up to seven clusters per subscription and region: five active clusters plus two clusters that you deleted in the past two weeks.
You can link up to 1,000 Log Analytics workspaces to a cluster.
You can perform up to two workspace link operations on a particular workspace in a 30-day period.
You can't move a cluster to another resource group or subscription.
You can't move a cluster to another region.
You shouldn't include both identity and key identifier details in the same operation when updating a cluster. If you need to update both, make two consecutive update operations.
Lockbox isn't currently available in China.
Lockbox can't currently be applied to tables with the Auxiliary plan.
Double encryption is configured automatically for clusters created from October 2020 in supported regions. You can verify if your cluster is configured for double encryption by sending a GET request on the cluster and observing that the
isDoubleEncryptionEnabledvalue istruefor clusters with Double encryption enabled.- If you create a cluster and get an error "region-name doesn't support Double Encryption for clusters.", you can still create the cluster without Double encryption by adding
"properties": {"isDoubleEncryptionEnabled": false}in the REST request body. - You can't change the double encryption setting after creating the cluster.
- If you create a cluster and get an error "region-name doesn't support Double Encryption for clusters.", you can still create the cluster without Double encryption by adding
You can delete a workspace while it's linked to a cluster. If you recover the workspace during the soft-delete period, the workspace returns to its previous state and remains linked to cluster.
During the commitment period, you can change to a higher commitment tier, which restarts the 31-day commitment period. You can't move back to pay-as-you-go or to a lower commitment tier until after you finish the commitment period.
Troubleshooting
If you get a conflict error when creating a cluster, the cluster might be deleted and still in the deletion process. The cluster name remains reserved during the two-week deletion period and you can't create a new cluster with that name.
If you update your cluster while the cluster is in the provisioning or updating state, the update fails.
Some operations are long and can take a while to complete. These operations are cluster create, cluster key update, and cluster delete. You can check the operation status by sending a GET request to the cluster or workspace and observe the response. For example, an unlinked workspace doesn't have the clusterResourceId under features.
If you attempt to link a Log Analytics workspace that's already linked to another cluster, the operation fails.
Error messages
Cluster Create
- 400--Cluster name isn't valid. Cluster name can contain characters a-z, A-Z, 0-9 and a length of 3-63.
- 400--The body of the request is null or in bad format.
- 400--SKU name is invalid. Set SKU name to capacityReservation.
- 400--Capacity was provided but SKU isn't capacityReservation. Set SKU name to capacityReservation.
- 400--Missing Capacity in SKU. Set Capacity value to 100, 200, 300, 400, 500, 1,000, 2,000, 5,000, 10,000, 25,000, 50,000 GB per day.
- 400--Capacity is locked for 30 days. Decreasing capacity is permitted 30 days after update.
- 400--No SKU was set. Set the SKU name to capacityReservation and Capacity value to 100, 200, 300, 400, 500, 1,000, 2,000, 5,000, 10,000, 25,000, 50,000 GB per day.
- 400--Operation can't be executed now. Async operation is in a state other than succeeded. Cluster must complete its operation before any update operation is performed.
Cluster update
- 400--Cluster is in deleting state. Async operation is in progress. Cluster must complete its operation before any update operation is performed.
- 400--KeyVaultProperties isn't empty but has a bad format. See key identifier update.
- 400--Failed to validate key in Key Vault. Could be due to lack of permissions or when key doesn't exist. Verify that you set key and access policy in Key Vault.
- 400--Key isn't recoverable. Key Vault must be set to Soft-delete and Purge-protection. See Key Vault documentation
- 400--Operation can't be executed now. Wait for the Async operation to complete and try again.
- 400--Cluster is in deleting state. Wait for the Async operation to complete and try again.
Cluster Get
- 404--Cluster not found, the cluster might have been deleted. If you try to create a cluster with that name and get a conflict, the cluster is in deletion process.
Cluster Delete
- 409--Can't delete a cluster while in provisioning state. Wait for the Async operation to complete and try again.
Workspace link
- 404--Workspace not found. The workspace you specified doesn't exist or was deleted.
- 409--Workspace link or unlink operation in process.
- 400--Cluster not found, the cluster you specified doesn't exist or was deleted.
Workspace unlink
- 404--Workspace not found. The workspace you specified doesn't exist or was deleted.
- 409--Workspace link or unlink operation in process.
Next steps
- Learn about Log Analytics dedicated cluster billing.
- Learn about proper design of Log Analytics workspaces.
- Get other sample templates for Azure Monitor.