Visual Studio 11 Beta Testing Documentation
The Visual Studio 11 Beta is released and along with it comes some updated testing documentation. The updated content can be found in the MSDN library within the Visual Studio 11 Beta documentation. The updated functionality and content specific to Visual Stuio ALM testing includes the following:
Test project compatibility and upgradability with Visual Studio 2010
Test projects that you created in Visual Studio 2010 SP1 that contain unit tests, coded UI tests, Web performance or load tests can be opened in Visual Studio 11 Beta. Therefore, your team can continue to use Visual Studio 2010 SP1 or Visual Studio 11 Beta to work with these test projects. In some cases, like when projects contain coded UI tests, the projects might be repaired when they are opened in Visual Studio 11 Beta. The repair process enables the test project to work correctly with both Visual Studio 2010 SP1 and Visual Studio 11 Beta. For more information, see:
Compatibility of Microsoft Test Manager with Visual Studio 2010
Compatibility of Coded UI test projects with Visual Studio 2010
Compatibility of Web performance and load tests with Visual Studio 2010
New test project templates
Visual Studio 11 Beta introduces new test project templates. In Visual Studio 2010, there was a single test project template. Visual Studio 11 Beta has these test project templates:
Unit Test Project – Can include unit tests, third-party unit tests, ordered tests, and generic tests.
Coded UI Test Project – Can include the test types listed for Unit Test Project above, and coded UI tests.
Web Performance and Load Test Project – Can include Web performance tests and load tests.
For more information, see Upgrading Tests from Earlier Versions of Visual Studio.
Upgrading Lab Management
If you are using lab environments, the version of your test controller, test agents, Team Foundation Server, and workflow definitions must be the same. Lab environments that you created with Microsoft Test Manager 2010 on Team Foundation Server 2010 can be opened with Microsoft Test Manager on Visual Studio 11 Team Foundation Server Beta after you upgrade your Lab Management components. When you upgrade your Lab Management components, you must perform specific steps during the upgrade process or your lab environments won’t work.
If you upgrade to Visual Studio 11 Team Foundation Server Beta and Visual Studio 11 Beta, you do not have to upgrade SCVMM 2008 R2 to SCVMM 2012 for you lab environments to work. Both versions of SCVMM are compatible with Lab Management on Visual Studio 11 Beta. See Upgrading Lab Management Components from Visual Studio 2010.
Microsoft Test Manager
Compatibility of Microsoft Test Manager with Visual Studio 2010
Microsoft Test Manager 2010 with Visual Studio 11 Team Foundation Server Beta:
You can connect to Visual Studio 11 Team Foundation Server Beta from Microsoft Test Manager 2010 and use most of the features. However, to connect to this version of Team Foundation Server, you must first install Visual Studio 2010 Service Pack 1 and all the latest software updates on the machine running Microsoft Test Manager 2010.
You can’t create or edit lab environments. You can, however, view the ones that have already been created, and use them for running manual or automated tests, and for remote data collection.
You can’t use any of the newer features of Visual Studio 11 Team Foundation Server Beta such as Exploratory Testing. To use these features, you must also upgrade to Microsoft Test Manager.
Microsoft Test Manager with Team Foundation Server 2010:
To use Microsoft Test Manager with Team Foundation Server 2010, you must first ensure that you are using Team Foundation Server 2010 SP1, and then you must install the latest software updates for the Team Foundation Server. Similarly, all the test controllers and test agents installed in lab environments must be running version 2010 SP1 components, and you must install the latest software updates on all of them.
The Environments and Library tabs in Microsoft Test Manager are disabled when you connect to Team Foundation Server 2010. For this functionality, you must install Microsoft Test Manager 2010 SP1 side-by-side with Microsoft Test Manager.
You can’t use any of the newer features of Microsoft Test Manager such as Exploratory Testing. To use these features, you must also upgrade the server to Visual Studio 11 Team Foundation Server Beta.
See Upgrading Microsoft Test Manager from Visual Studio 2010 and Upgrading Tests from Earlier Versions of Visual Studio.
Manual testing of Windows Metro style apps
You can use Microsoft Test Manager to run manual tests to help you identify problems in your Metro style apps that are running on a remote device, such as a tablet.
See Testing Metro style apps Running On a Device with Microsoft Test Manager.
Enhanced action logs for Windows Metro style apps
The action log file associated with testing Metro style apps or Internet Explorer 10 sessions on Windows 8 Consumer Preview are saved as .html files containing screenshots for each action step conducted during the test.
This .html file can be viewed in your Web browser. The enhanced action log makes reproducing bugs easier since you can see the exact steps taken by the tester.
See Testing Windows Metro style apps Running on a Device with Microsoft Test Manager and How to: View Manual Test Case Results Using Microsoft Test Manager.
Exploratory testing window
You can conduct exploratory testing from Microsoft Test Manager. Exploratory testing is the testing of an application without a set of tests defined in advance. It can also be described as simultaneous test planning and test execution. During an exploratory test session, you are not restricted to a script or a set of predetermined steps like you are with a manual test.
Exploratory test sessions can be associated with requirements work item types, such as user stories. You can start an exploratory test session from Microsoft Test Manager and conduct appropriate testing on the application using the Exploratory Testing window to verify that the product backlog item behaves correctly. This testing is unscripted and doesn’t have predefined test cases. This can be useful if you are using a design and test approach while developing your application, because the Exploratory Testing window provides an easy way to create manual test cases based on the action steps during your session. If you create a manual test based on the action steps that are recorded during your exploratory session, the manual test case is automatically associated with your product backlog item and placed in the appropriate test suite. All bugs that you file during this session will also be automatically linked with the product backlog item. For more information about creating user stories in Microsoft Test Manager, see How to: Add Product Backlog Items, User Story, or Requirements Work Items to Your Test Plan and How to: Create a Work Item using Microsoft Test Manager.
From Microsoft Test Manager, you can also conduct general exploratory testing that is not associated with a product backlog item.
While you explore your application using the Exploratory Testing window, you can create bugs that include the action steps performed and any comments, screenshots, and file attachments you have added.
See Performing Exploratory Testing Using Microsoft Test Manager.
Manual test steps can include multiple lines
Test steps can include multiple lines to consolidate related actions within a single test step in your test cases.
see Add action and validation test steps to your manual test case.
Manual tests includes rich text
Microsoft Test Manager includes a toolbar that you can use to format the text of your test steps. You can use various formatting options, for example bold, underline, or color highlighting to emphasis key points.
See Add action and validation test steps to your manual test case.
Microsoft Test Manager test plan results
The Plan tab in the center group menu bar of Microsoft Test Manager includes Results which provides a report on the status of your test plan.
See How to: View Test Plan Results in Microsoft Test Manager.
Cloning test plans for new iterations
You can clone a test plan. Cloning lets you work on different releases in parallel. For example, if you already have a test plan called “Contoso V1 – Milestone 1” and your team decides to make version V2, you can create a clone of the test plan to be used for the V2 source code branch. When you clone a test plan, the new destination test plan will have the same area path as the original source test plan, but will have a different iteration path. After cloning the test plans, you and your team can work on both versions of the applications simultaneously.
See Copying and Cloning Test Suites into Other Plans.
Coded UI tests
Coded UI Test Editor
The Coded UI Test Editor is included in Visual Studio 11 Beta. The Coded UI Test Editor was available to MSDN subscribers as a feature pack in Visual Studio 2010. See Microsoft Visual Studio 2010 Feature Pack 2.
The Coded UI Test Editor lets you easily modify your coded UI tests. Using the Coded UI Test Editor, you can locate, view, and edit your test methods, UI actions and their associated controls in the UI control map. See Editing Coded UI Tests Using the Coded UI Test Editor.
Compatibility of coded UI test projects with Visual Studio 2010
Test projects that were created in Visual Studio 2010 are silently repaired when they are opened in Visual Studio 11 Beta. If the test projects are checked into source control, the project files are checked out for this repair. Once repaired, these test projects containing coded UI tests will work in both Visual Studio 2010 SP1 and Visual Studio 11 Beta.
For more information, see Upgrading Coded UI Tests from Visual Studio 2010 and Upgrading Tests from Earlier Versions of Visual Studio.
Support for HTML5 in Windows Internet Explorer 9 and Internet Explorer 10
Coded UI tests support record, playback, and validation of HTML5 controls including Audio, Video, ProgressBar and Slider.
See Using HTML5 Controls in Coded UI Tests and Supported Configurations and Platforms for Coded UI Tests and Action Recordings.
Support for Windows Internet Explorer 10 spell checking
Internet Explorer 10 includes spell checking capabilities for all text boxes allowing you to choose from a list of suggested corrections. Coded UI Test will ignore user actions such as selecting an alternative spelling suggestion. Only the final text typed into the text box will be recorded.
The following actions are recorded for coded UI tests that use the spell checking control: Add to Dictionary, Copy, Select All, Add To Dictionary, and Ignore.
Support for 64-bit Windows Internet Explorer with Windows 8 Consumer Preview
Previously 64-bit versions of Internet Explorer were not supported for recording and playback. With Visual Studio 11 Beta, coded UI tests have been enabled for 64-bit versions of Internet Explorer running under Windows 8 Consumer Preview.
Support for pinned sites in Windows Internet Explorer 9
In Internet Explorer 9, pinned sites were introduced. With pinned sites, you can get to your favorite sites directly from the Windows taskbar—without having to open Internet Explorer first. Coded UI tests can now generate intent aware actions on pinned sites. See Pinned Sites.
Support for Windows Internet Explorer 9 semantic tags
Internet Explorer 9 introduced the following semantic tags: section, nav, article, aside, hgroup, header, footer, figure, figcaption and mark. Coded UI tests ignore all of these semantic tags while recording. You can add assertions on these tags using the Coded UI Test Builder. You can use the navigation dial in the Coded UI Test Builder to navigate to any of these elements and view their properties.
Seamless handling of white space characters between versions of Windows Internet Explorer
There are differences in between the way Internet Explorer 8, Internet Explorer 9, and Internet Explorer 10 handle white space characters. Coded UI Test handles these differences seamlessly. Therefore, a coded UI test created in Internet Explorer 8 for example, will playback successfully in Internet Explorer 9, and Internet Explorer 10.
The notification area of Windows Internet Explorer is Now Recorded With the “Continue on Error” Attribute Set
All actions on the notification area of Internet Explorer are now recorded with the “Continue on Error” attribute set. If the notification bar doesn’t appear during playback, the actions on it will be ignored and coded UI test will continue with the next action.
New coded UI test APIs
The coded UI test APIs include new functionality. Some of this new functionality includes:
Find column names and column values for List View controls.
Get content of controls like Data Grid, or List and validate it.
Get tool tip text of control.
Find broken links on the page.
For more information, see API Reference for Testing Tools for Visual Studio ALM.
Web Performance and Load Tests
Compatibility of Web performance and load tests with Visual Studio 2010
Your existing Web performance and load tests that were created in Visual Studio 2010 will continue to run in Visual Studio 11 Beta. However, Visual Studio 11 Beta has deprecated the Test View window that you might have used previously to run your Web performance and load tests. To run your Web performance and load tests, you must now do so from the Web Performance Test Editor or the Load Test Editor window. See Running Load and Web Performance Tests.
If you are using test controllers from Visual Studio for Web performance or load testing, then the version of test controller must match the version of Visual Studio. For more information about upgrading test controllers, see Upgrading Test Controllers from Visual Studio 2010 and Installing and Configuring Test Agents and Test Controllers.
For more information about upgrading Web performance and load tests, see Upgrading Web Performance and Load Tests from Visual Studio 2010 and Upgrading Tests from Earlier Versions of Visual Studio.
Simplified agent configuration
Lab agents, build agents, and test agents from previous versions of Lab Management have been combined in to a single agent—a test agent. When you use Microsoft Test Manager to run tests, you no longer have to install test agents on the computers that you compose into a lab environment. Lab Management does this for you when you create or repair your lab environments.
For more information, see Creating an SCVMM Environment and Creating a Standard Environment.
SCVMM 2012 and Clustered Host Support
Lab Management for Visual Studio 11 Beta supports SCVMM 2012 and clustered Hyper-V hosts. For more information, see Creating an SCVMM Environment. For information about upgrading SCVMM 2008 R2 to SCVMM 2012, see Upgrading SCVMM 2008 R2 to SCVMM 2012.
Compatibility with SCVMM 2008 RC2
You do not have to upgrade SCVMM 2008 R2 to SCVMM 2012. Lab Management on Visual Studio 11 Beta supports both versions of SCVMM.