Microsoft Test Manager Tips and Tricks
| This article is in a draft stage.|
Microsoft Test Manager 2010 (MTM) is a testing utility for managing and running manual and automated test cases for a TFS Team Project. It integrates really well with Visual Studio 2010 and the Visual Studio 2010 web client (http://icstfsweb). However, there are several helpful tips and tricks that can make using MTM a better overall experience. This article contains some of those helpful hints and will continue to grow as new things are discovered.
Installing SP1 and SP2
The first thing that must be done to improve the MTM experience is to install Visual Studio 2010 Service Pack 1. There were some significant bug fixes and improvements to MTM that were released in SP1 due to a lot of customer complaints when it was initially released in April 2010. The Microsoft site has the full list of SP1 improvements to MTM. To install SP1, go to: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=75568aa6-8107-475d-948a-ef22627e57a5&displaylang=en.
The Visual Studio 2010 Feature Pack 2 has a number of testing features that expand MTM including capture and playback for Silverlight 4 web applications, using action recording to fast forward through manual tests, and support for coded UI tests using Firefox. It is cumulative so FP2 includes everything in FP1. For more information about Visual Studio 2010 FP2, go to: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=75568aa6-8107-475d-948a-ef22627e57a5&displaylang=en.
Pausing and Resuming a Manual Test
There is a handy little trick that Microsoft did a poor job of making visible, so you may not be aware of it. When running a manual test in Test Runner, there may be a need for return to MTM while running a test (that is, to look at the Test Case details). There is a Pause button, but that will not return to MTM to view details about the Test Plan or Test Cases.
At the top of the Test Runner GUI, there is an icon that will allow you to pause the current run (even if you are creating an Action Recording). If you hover over the icon, the tooltip says, "Return to the Testing Center" (see Figure 1).
|"Return to the Testing Center" icon (Figure 1).|
When inside the Testing Center, you can view Test Cases, change Test Plan properties, or whatever you wish. When you want to resume the running of the manual test, click on the Return to Test Runner icon in the upper right-hand area of the Testing Center (see Figure 2). When hovering over the icon, the tooltip displays "Return to Test Runner." This icon is only visible when you have paused a manual test run.
|"Return to Test Runner" icon (Figure 2).|
Speeding Things Up in Test Runner
By default, a local set of Test Settings is created so you can immediately start running manual tests in Test Runner. However, the default settings may not be the best for the testing you are doing. You can go into the Lab Center and create a new set of Test Settings under the Settings tab, by clicking on the New button (see Figure 3).
|Creating New Test Settings (Figure 3).|
This will open up a new form where you can specify your customized Test Settings. On the first page of the New Test Settings Manager, type in a name, description (optional), and specify if these will be used for Manual Tests or Automated Tests (see Figure 4). Then click on the Next button.
|General Test Settings (Figure 4).|
On the next page you can select the Roles that your test machine will use. Typically this would be either Desktop Client or Web Client depending the type of application you are testing. The simplest Role would be Local if you are just running manual tests on your laptop. After selecting one (or more) of the Roles, click the Next button.
The next page is where you can select and configure the Data and Diagnostics that you want to have saved off for each running Test Case. These settings can affect your performance (see Figure 5).
|Data and Diagnostics Test Settings (Figure 5).|
The biggest slow down can come if you have IntelliTrace turned on. Typically you would want this turned off until you have a failing test that you want to debug. In that scenario you would turn it on so you get more information to determine what the problem is.
The Video Recorder setting is a great option to turn on when you have automated tests running overnight. You can actually see what happened while your test was running (you will need Microsoft Expression Encoder installed). There is a setting in the configuration of the Video Recorder options to save videos of passing tests. It will save a lot of disk space if you deselect that option – videos of failing tests will still be saved (see Figure 6).
|Configure Video Recorder (Figure 6).|
Click the Next button to see the Summary page, and then click Finish. You can set these Test Settings to be used by default on all manual tests by going to the Test Plan Properties page.
Test Controller on a lab Virtual Machine
There is a trick to getting a Test Controller to work properly on a Virtual Machine (VM) in the lab. The problem is that if you are at your desk on your machine, you will not be able to send tests that you want to run to that Test Controller. It has a problem resolving the name of the machine. Even though the machine is registered and added to the TFS Collection, your machine will not be able to resolve the name. It has to do with DNS. You can do either of two things to overcome the problem (the second option is recommended).
- You can add the IP address of the VM to your local hosts file. To do this you would find out the IP address of the VM, then go to (usually)
C:\Windows\System32\drivers\etc, and then add the IP address of the VM to the hosts file.
- The other option is to contact the lab team (Alan Keeler, Rodney Harrison, among others) and ask them to add the VM bane to the DNS on ldschurc.org. This will allow any user that has rights to see the Test Controller to send tests to it.
One last thing to remember is that you must have Test Agents registered with the Test Controller to actually run the tests. The Test Controller only does the coordination of sending out the tests to the Agents and routing the results back to the TFS database.
You can turn on Video Recording which will take a video of Tests that are running or even ad hoc manual testing. To make this option available, it will be required to install Microsoft Expression Encoder 4.0. This is a free download from http://www.microsoft.com/download/en/details.aspx?id=24601. The files are stored in a "Expression Encoder Screen Capture" format and saved it with a ".XESC" extension. To turn this feature on, see Figure 5 above in the Speeding Things Up in Test Runner section.
There may be some people that will want to replay the recording but don't want to install Expression Encoder. There is a way to get the minimal pieces to see the encoding by installing the codec at http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=10732 and then installing the video data adapter.
There is one other option which is that you could open the video recording .XESC file in Microsoft Expression Encoder and save it in WMV or MP4 format. Test Runner will save the video recording file automatically to a bug when created from a failed Test Case in the default XESC format. It is possible to open the file, convert it, attach the WMV or MP4 file to the bug, and then delete the XESC file. Then anyone can replay the videos without installing anything.
Adding Special Suites
By default, the only Work Item Type (WIT) that can be added as a suite (also known as a requirement) to a Test Plan in MTM 2010 is a "User Story." If you try to add any other type of item to the Test Plan it will fail and possibly crash. The way to add more WITs as acceptable suites or requirements is to use the witadmin utility which is located in
Program Files\Microsoft Visual Studio 10.0\Common7\IDE.
To see the options available to pass in on the command-line to witadmin.exe, type:
First, export the existing Categories settings document which specifies the Work Item Types that can be used throughout TFS, Visual Studio, and MTM. To export the XML file, type:
Witadmin exportcategories /collection:http://icstfs.ldschurch.org/tfs/UpgradeFromICSTFS2008 /p:<Your Project Name> /f:”C:\Users\<Your User Name>\Documents\TFScategories.xml”
Once that is downloaded to your machine, you can modify it and then import it back to the TFS Server. Open the TFSCategories.xml file and find the category to be changed. In the case of adding more acceptable WITs to become suites in MTM, look for the following section:
<CATEGORY refname="Microsoft.RequirementCategory" name="Requirement Category"> <DEFAULTWORKITEMTYPE name="User Story" /> </CATEGORY>
Make the desired modifications by adding lines that specify the other WITs to be added. The changes should look something like the following:
<CATEGORY refname="Microsoft.RequirementCategory" name="Requirement Category"> <DEFAULTWORKITEMTYPE name="User Story" /> <WORKITEMTYPE name="Requirement" /> <WORKITEMTYPE name="Task" /> </CATEGORY>
Once the XML file contains the desired changes, it needs to be saved and imported to TFS. Type the following to import it:
Witadmin importcategories /collection:http://icstfs.ldschurch.org/tfs/UpgradeFromICSTFS2008 /p:<Your Project Name> /f:”C:\Users\<Your User Name>\Documents\TFScategories.xml”
It will be necessary to close and restart Microsoft Test Manager if it was open while making the changes. After restarting MTM, it should be possible to add Work Items of the newly added types to the Test Plan as suites (or requirements).
Further information about Microsoft Test Manager: