Uredi

Durable Task Scheduler

The Durable Task Scheduler provides durable execution in Azure. Durable execution is a fault-tolerant approach to running code that handles failures and interruptions through automatic retries and state persistence. Durable execution helps with scenarios such as:

  • Distributed transactions
  • Multi-agent orchestration
  • Data processing
  • Infrastructure management

Durable Task Scheduler is the recommended storage provider for Durable Functions and the Durable Task SDKs.

Supported SKUs

For Durable Functions, you can use the Durable Task Scheduler with any of the Functions SKUs.

For the Durable Task SDKs, you can use Durable Task Scheduler with any compute.

The scheduler itself offers two billing SKUs:

Supported regions

Durable Task Scheduler is available in most Azure regions. Run the following command to get the current list of available regions:

az provider show --namespace Microsoft.DurableTask --query "resourceTypes[?resourceType=='schedulers'].locations | [0]" --out table

Consider using the same region for your Durable Functions app and the Durable Task Scheduler resources to optimize performance and certain network-related functionality.

Orchestration frameworks

Durable Task Scheduler works with both Durable Functions and the Durable Task SDKs. Choose which framework works best for your project.

Architecture

For all Durable Task Scheduler orchestration frameworks, you can create scheduler instances of type Microsoft.DurableTask/scheduler using Azure Resource Manager. Each scheduler resource internally has its own dedicated compute and memory resources optimized for:

  • Dispatching orchestrator, activity, and entity work items
  • Storing and querying history at scale with minimal latency
  • Providing a rich monitoring experience through the Durable Task Scheduler dashboard

Unlike the BYO storage providers, the Durable Task Scheduler provider is a purpose-built backend-as-a-service optimized for the specific needs of the Durable Task Framework.

The following diagram shows the architecture of the Durable Task Scheduler backend and its interaction with connected apps.

Screenshot of the Durable Task Scheduler architecture showing the backend service connecting to apps via gRPC.

Operational separation

The Durable Task Scheduler runs in Azure as a separate resource from your app. This isolation is important for several reasons:

  • Reduced resource consumption
    Offloading state management to a managed scheduler reduces CPU and memory consumption in your app compared to using a BYO storage provider.

  • Fault isolation
    Separating the scheduler from the app reduces the risk of cascading failures and improves overall reliability in your connected apps.

  • Independent scaling
    The scheduler resource can be scaled independently of the app for better infrastructure resource management and cost optimization. For example, multiple apps can share the same scheduler resource, which is helpful for organizations with multiple teams or projects.

  • Improved support experience
    The Durable Task Scheduler is a managed service, providing streamlined support and diagnostics for issues regarding the underlying infrastructure.

App connectivity

Your apps connect to the scheduler resource via a gRPC connection, secured using TLS and authenticated by the app's identity. The endpoint address is in a format similar to {scheduler-name}.{region}.durabletask.io. For example, myscheduler-123.westus2.durabletask.io.

For scenarios that require private connectivity, you can use private endpoints to route traffic to the scheduler over a private link within your virtual network instead of the public internet.

Work items are streamed from the scheduler to the app using a push model, improving end-to-end latency and removing the need for polling. Your apps can process multiple work items in parallel and send responses back to the scheduler when the corresponding orchestration, activity, or entity task is complete.

State management

The Durable Task Scheduler manages the state of orchestrations and entities internally, without a separate storage account for state management. The internal state store is highly optimized for use with Durable Functions and the Durable Task SDKs, resulting in better durability and reliability and reduced latency.

The scheduler uses a combination of in-memory and persistent internal storage to manage state.

  • The in-memory store is used for short-lived state.
  • The persistent store is used for recovery and for multi-instance query operations.

Feature highlights

When you implement one of the Durable Task Scheduler orchestration frameworks, you benefit from several key highlights.

Durable Task Scheduler dashboard

When a scheduler resource is created, a corresponding dashboard is provided out-of-the-box. The dashboard provides an overview of all orchestrations and entity instances and allows you to:

  • Quickly filter by different criteria.
  • Gather data about an orchestration instance, such as status, duration, input/output, etc.
  • Drill into an instance to get data about sub-orchestrations and activities.
  • Perform management operations, such as pausing, terminating, or restarting an orchestration instance.

Access to the dashboard is secured by identity and role-based access controls.

For more information, see Debug and manage orchestrations using the Durable Task Scheduler dashboard.

Multiple task hubs

State is durably persisted in a task hub. A task hub:

  • Is a logical container for orchestration and entity instances.
  • Provides a way to partition the state store.

With one scheduler instance, you can create multiple task hubs that can be used by different apps. Each task hub gets its own monitoring dashboard. To access a task hub, the caller's identity must have the required role-based access control (RBAC) permissions.

Creating multiple task hubs isolates different workloads that can be managed independently. For example, you can:

  • Create a task hub for each environment (dev, test, prod).
  • Create task hubs for different teams within your organization.
  • Share the same scheduler instance across multiple apps.

Scheduler sharing is a great way to optimize cost when multiple teams have scenarios requiring orchestrations.

Note

Although you can create multiple task hubs in one scheduler instance, they share the same resources. If one task hub is heavily loaded, it can affect the performance of other task hubs on the same scheduler.

Emulator for local development

The Durable Task Scheduler emulator is a lightweight version of the scheduler backend that runs locally in a Docker container. With it, you can:

  • Develop and test your app without deploying to Azure.
  • Monitor and manage your orchestrations and entities just like you would in Azure.

For setup instructions, see Run the Durable Task Scheduler emulator.

Note

The emulator internally stores orchestration and entity state in local memory, so it isn't suitable for production use.

Autopurge retention policies

Stale orchestration data should be purged periodically to ensure efficient storage usage. The autopurge feature for Durable Task Scheduler provides a streamlined, configurable solution to manage orchestration instance clean-up automatically. Learn more about setting autopurge retention policies for Durable Task Scheduler.

Limitations and considerations

  • Scheduler quota:

    You're limited in how many schedulers you can create depending on your billing SKU.

  • Task hub quota:

    You're limited in how many task hubs you can use depending on your billing SKU.

    For more quota, contact support.

  • Max payload size:

    The Durable Task Scheduler has a maximum payload size restriction for the following JSON-serialized data types:

    Data type Max size
    Orchestrator inputs and outputs 1 MB
    Activity inputs and outputs 1 MB
    External event data 1 MB
    Orchestration custom status 1 MB
    Entity state 1 MB

    If your data exceeds these limits, see Large payload support for available workarounds.

  • Orchestration instance ID length:

    Orchestration instance IDs are limited to a maximum length of 100 characters.

    • Allowed characters: Printable ASCII only (letters, numbers, symbols like -, _, ., etc. Characters 0x20 through 0x7E)
    • Minimum length: 1 character (cannot be empty)
    • Instance IDs starting with @ are reserved for entities

  • Feature parity:

    Extended sessions aren't supported by Durable Task Scheduler.

Related content

  • Last updated on