Test Plans FAQs

Azure Test Plans uses three access levels:

• Stakeholder: Can provide feedback through the Test & Feedback extension but can't access the Test Plans portal.
• Basic: Can execute test cases, mark test outcomes, and view charts and reports.
• Basic + Test Plans: Full capabilities, including creating and managing test plans, test suites, test cases, configurations, and parameters. Visual Studio Enterprise, Visual Studio Test Professional, and MSDN Platforms subscriptions include equivalent access.

For the complete permissions matrix, see Manual test access and permissions.

The Define tab is only available to users with Basic + Test Plans access or equivalent. Users with Basic access can use the Execute and Chart tabs but can't author or manage test cases through the Define tab. To get access, ask your admin to assign you the Basic + Test Plans access level.

Azure Test Plans supports three types of test suites:

• Static test suites: Manually organize test cases into groups. Use static suites when you want to hand-pick which test cases belong together.
• Requirement-based test suites: Automatically link test cases to backlog items (user stories, product backlog items). Use requirement-based suites to track test coverage against requirements — this suite type is the only way to support end-to-end requirement traceability.
• Query-based test suites: Automatically populate test cases based on a work item query (for example, all test cases with Priority=1). The suite updates whenever the query results change.

For more information, see Test objects and terms.

Yes. Depending on your desired action, you can copy or clone test plans and import or clone test suites. To learn how, see Copy or clone test plans, test suites, and test cases.

Yes. Select Print in the Export dialog box, then choose Cancel in the Print dialog box to display the data in the report. Select all the text and copy it into a Word document. The report formatting is retained.

The test case still exists in your project but is removed from the test suite and no longer linked to the backlog item for that suite.

This issue can occur if you're prompted to enter credentials when you select the link. Without signing out of Azure DevOps, select View Tests again to see the correct test suite and tests.

In Test Plans, use the directory to find your test plans:

• Mine: Shows test plans for teams you belong to, plus your favorites. Plans are grouped by team.
• All: Shows all test plans in the project. You can add plans to favorites from this view.

Use the filter controls to search by name, team, state, or iteration. For more information, see Navigate Test Plans.

Yes. See Copy test cases.

Yes. Press Shift+Enter in the action or expected results field to add an extra line.

Select a test step. Press Alt+P to insert a new test step above the selected step.

See Delete test artifacts.

You can import and export test cases in bulk using CSV or XLSX files. Import lets you create new test cases or update existing ones (by including test case IDs). Export lets you download test case details, including custom columns.

For step-by-step instructions, see Bulk import and export test cases.

Yes. In the same CSV or XLSX file, leave the ID field empty for new test cases and include the existing ID for updates.

The import wizard validates your file at each stage — file upload, field mapping, and before final import. It displays errors inline and you must resolve them before the import can proceed.

Common errors and solutions:

To fix errors, correct your CSV or XLSX file, reupload it, and complete the import.

The import process supports only Test Case work items. To reference existing shared steps, include their ID in your file. The import can't create new shared steps — create them in the web interface first, then reference their ID.

For other work item types (User Stories, Tasks, Bugs), see Bulk import or update (CSV).

Include the following nine headers with exact spelling:

There's no single-action undo. However, each import creates a revision for every affected test case. View the History tab on individual test cases to see what changed and manually revert fields. For large-scale rollbacks, reimport the original exported file.

The following limitations apply:

• Test cases must be in Design state during import.
• Test case titles can't exceed 128 characters.
• Import and export files have a 20-MB size limit.
• You must have permissions for the area and iteration paths of the target test plan and test suite.
• Operations fail if a test case has more than 1,000 related links.

Shared steps let you define a reusable sequence of test steps (such as a common sign-in flow) that can be referenced by multiple test cases. When you update shared steps, the changes automatically apply to all test cases that use them.

To create shared steps, select one or more steps in a test case, then choose the Create shared steps icon. For more information, see Share steps between test cases.

You execute test points, not test cases directly. A test point is a unique combination of a test case, test suite, configuration, and tester. For example, if a test case is assigned two browser configurations (Chrome and Edge), that creates two test points — one for each configuration. The Execute tab shows the latest execution result for each test point.

When you run tests from the Execute tab, you can choose from the following runners:

• Web browser-based runner: Runs manual tests in the browser. You can optionally select a specific build to associate results with.
• Test Runner client (desktop): A desktop application for testing desktop applications.
• Automated tests using a release stage: Triggers automated test execution from a build and release pipeline.

For more information, see Run manual tests.

Yes. The Test Runner Client for Windows is scheduled for retirement. After the retirement date, it will no longer be available or supported. Transition to the web-based test runner, which provides the same functionality with improved performance and ongoing development.

For more information, see Run manual tests.

During a manual test run, you can collect the following diagnostic data:

• Screen captures: Take annotated screenshots during test execution.
• Image action log: Automatically captures your interactions with the application as a step-by-step visual log.
• Screen recordings: Record your screen during testing. Recordings auto-stop after 10 minutes.

For more information, see Collect diagnostic data while testing.

For test case charts, if a test case gets added to multiple test suites in a plan, then the test only gets counted once. For test result charts, each instance of a test that is run is counted for each of the test suites separately.

To create charts, you need to be assigned at least Basic access.

Choose Configure and the option you want from the chart's context menu.

The Progress Report has the following limitations:

• Shows data for one or more test plans in a single project only. For cross-project reporting, use OData APIs.
• Data updates approximately every 15 minutes and isn't real-time.
• Percentage values don't display decimal places.
• Outcomes like Blocked and Not Applicable aren't reflected in Passed% or Failed%, which can show a gap between Run% and the sum of Passed% and Failed%.
• Data from test plans migrated from on-premises Azure DevOps Server doesn't appear.

For more information, see Progress Report.

It's better to use test configurations. With test case parameters, you run the different parameter values one after another, which makes it difficult to switch from one platform to another. For more information, see Test different configurations.

You need the project-level Manage test configurations permission set to Allow. By default, this permission is granted to members of the Contributors and Project Administrator groups.

You can associate automated test methods with test case work items so that you can run them from Test Plans. In Visual Studio, open Test Explorer, select a test method, and choose Associate to Test Case. You can also associate tests through a build pipeline in Azure DevOps.

For more information, see Associate automated tests with test cases.

The following test frameworks are supported:

• Visual Studio association: MSTest v1/v2, NUnit, xUnit, Selenium, Coded UI
• Azure DevOps association: Java (Maven/Gradle with JUnit), JavaScript (Jest), Python (PyTest)
• .NET Core: Supported via Visual Studio 15.9 or later with a .runsettings file

Tests from GitHub repositories are also supported when run through Azure Pipelines with the VSTest or PublishTestResults tasks.

Yes. You can use both YAML and Classic pipelines to run automated tests from Test Plans. Configure the build pipeline in the test plan settings, and set up a release pipeline (Classic or YAML) for on-demand automated test execution.

For setup instructions, see Run automated tests from test plans.

Yes. Use the Run with options command. Open the shortcut menu for the test suite and select Run with options, then specify:

• Test type and runner: Select Automated tests using Release Stage.
• Build: Select the build that has the test binaries. Test results are associated with this build.
• Release Pipeline: Select a pipeline that can consume the selected build artifact.
• Release Stage: Select the stage configured in your release pipeline.

Azure Pipelines provides an orchestration workflow to obtain test binaries as artifacts and run tests. This workflow uses the same concepts as scheduled testing, so you can clone an existing scheduled testing release pipeline to get started quickly.

Release stages also give you access to the full task catalog for activities before and after test execution, such as preparing test data or managing configuration files.

We recommend a separate release pipeline and stage for on-demand automated testing because:

• Scheduled stages typically deploy the app first — you might not want a full deployment every time you run a few tests.
• Every on-demand run triggers a new release. High volumes of on-demand releases can make it hard to find your scheduled testing and production releases.
• You might want to configure the Visual Studio Test task with a Test run identifier to trace what triggered each release.

If you run UI tests (coded UI or Selenium), the agent must run in interactive mode with autologon enabled so it can launch a web browser. If you use a headless browser, the agent can run as a service or in interactive mode.

For more information, see Build and release agents, Deploy an agent on Windows, and Agent pools.

When Select tests using is set to Test run, the test management subsystem passes the list of selected tests through a test run object. The Visual Studio Test task looks up the test run identifier, extracts test execution information (container and test method names), runs the tests, updates the results, and sets the associated test points.

This also provides an audit trail linking historical releases and test run identifiers to the tests submitted for on-demand execution.

Use a runsettings file to pass values as parameters. For example, in a release with several stages, you can pass the appropriate app URL to each stage's test task. Specify the runsettings file and override parameters in the Visual Studio Test task.

Yes, if the following conditions are met:

• The agent pool has enough agents to handle parallel requests. If agents aren't available, releases queue until agents free up.
• You have enough parallel jobs configured.
• Testers don't run the same tests in parallel, as results might be overwritten depending on execution order.

Set the stage trigger option for behavior when multiple releases are waiting to be deployed to Allow multiple releases to be deployed at the same time (if your app supports parallel testing) or Allow only one active deployment at a time.

The on-demand automated testing workflow isn't currently configuration-aware. Selecting multiple configurations for the same test doesn't create separate test runs per configuration.

The on-demand workflow is optimized for a single Azure Pipelines build. Support for multi-artifact releases and non-Azure Pipelines artifact sources (such as Jenkins) is evaluated based on user feedback.

See Get started with Selenium testing.

By default, Azure DevOps deletes all test results after one year (365 days), unless you indefinitely retain a build associated with those results. Older projects may have no automatic deletion configured.

For more information, see Set test retention policies.

See Set test retention policies.

See Set retention policies for builds, releases, and tests.

The Test Run Hub provides an enhanced interface for managing test execution in Azure Test Plans. You can view both manual and automated test runs, filter by timeline and run type, search by test run ID, customize columns, and drill into run details including pass rates, attachments, and analytics breakdowns by outcome, priority, configuration, and failure type.

Access the Test Run Hub from Test Plans > Runs. For more information, see Test runs.

You can view the video recordings created by the Test & Feedback extension in Google Chrome browser and in the VLC Video Player.

The Test & Feedback extension supports Azure DevOps Server (formerly Team Foundation Server) 2015 and later versions. All users, including users granted Stakeholder access, can use the extension in Connected mode. Functionality associated with session insights and the request and provide feedback flow require Azure DevOps Server 2017 or later versions.

Yes, the extension automatically shows bugs that might be related to the one you're creating and allows you to add your screenshots, notes, and videos to this existing bug. For more information, see Add findings to existing bugs with exploratory testing.

The Test & Feedback extension is available for Google Chrome and Microsoft Edge. Feature availability varies by browser — for the full compatibility matrix, see Install the Test & Feedback extension.

• Connected mode: The extension connects to Azure DevOps or Azure DevOps Server. You can create bugs and tasks that are automatically linked to your exploratory testing session, view session insights, and use the request/provide feedback flow.
• Standalone mode: Use the extension without connecting to Azure DevOps. You can capture screenshots, notes, and screen recordings, then export them as an HTML report. Standalone mode is useful for ad-hoc testing.

For more information, see Exploratory testing with the Test & Feedback extension in Connected mode and Standalone mode.

Related content

• Create a test plan
• Create test cases
• Run manual tests
• Run automated tests from test plans
• Track test status
• Manage test access and permissions