Tag Archives: coded ui testing

How To : Understanding and Use the Search logic for Silverlight controls in Coded UI Test

Understanding the Search logic for Silverlight controls in Coded UI Test

 

One of the primary objectives during recording in Coded UI Test is to generate a robust search condition for a UI control to be uniquely identifiable during playback. In this post I’ll mention some of the search logic specific to the Silverlight UI Automation support within Coded UI Test introduced in the VS 2010 Feature Pack 2.

Search condition generation during Recording

 

For Silverlight control, Coded UI Test relies primarily on the Automation properties of the control. The sequence of looking for a search property in order descending of priority is

AutomationId,

Name,

LabeledBy,

HelpText,

AccessKey,

AcceleratorKey

Specific controls support additional searchable properties. For instance, Button supports  “DisplayText”, Image supports “Source”, DataGrid Cell supports “ColumnIndex” searchable property and likewise. The various search configurations mentioned here are applicable to Silverlight control search too (except for the SearchConfiguration.VisibleOnly configuration).

microsoft-silverlight[1]

For a Silverlight object hosted in IE, the search hierarchy will consist of an IE search part and Silverlight search part –

 

Top Level Window à {IE Search Hierarchy} à Silverlight Root Visual Element à Parent of Target Element à Target Element.

 

Additional hierarchy can be generated in between the Parent and Silverlight Root Visual element based on the specific control requirement. For example, certain controls such as Datagrid, Tree, TreeItem, Tab, List Item are, at almost all times, included in the search hierarchy if they are found in the ancestor hierarchy of the target element. As an example, the extended search hierarchy of a DataGrid Cell will show up as something like

 

TopLevelWindow à {IE Search Hierarchy} à Root Visual Element (Silverlight) à DataGridTable (Silverlight) à DataGridRow(Silverlight) à DataGridCell (Silverlight)

 

 

Search path during Playback

 

The overall search logic remains identical to what is followed in other UI technologies. It is a breadth first search wherein the top level window is first searched and used as a container for searching the next control in the search condition hierarchy. This is done recursively until the leaf control in the search hierarchy is found.

 

As an example, for a simple button inside a Silverlight page, the search hierarchy will be typically of the format –

 

TopLevelWindow à Document (IE BODY Tag) à Pane (IE DIV Tag) à Custom (IE OBJECT Tag) à Root Visual Element (Silverlight) à Button (Silverlight).

 

For the top-down search till the IE Object Tag, the existing search features and settings in Coded UI Test are applicable. Once the search switches to the Silverlight technology (i.e. Root Visual element of the Silverlight page), there are few limitations to the search.

microsoft-silverlight2-developer-reference[1]

 

What is missing currently in Silverlight control search?

 

  1. PlaybackSettings.ShouldSearchFailFast
  • This setting is not honored currently. However, there is some level of customization that can be done using the PlaybackSettings.SearchTimeout and the Playback.PlaybackSettings.WaitForReadyTimeout settings to tweak the timeout at which the search should abort. The later may not seem obvious, and is hence explained in more detail in a section below.

 

  1. PlaybackSettings.MatchExactHierarchy

Silverlight control search does not currently honor  MatchExactHierarchy = false. So, the search condition specified for the entire Silverlight hierarchy needs to be accurate for the search to succeed. In the above example, it is the Root Visual Element and the Button control.

 

  1. Playback.PlaybackSettings.SmartMatchOptions
  • Control level smart match is not currently supported i.e. Control

Note that Regex match is not yet supported in Coded UI Test. So the only option available is to specify the PropertyExpressionOperator.Contains condition operator in the search properties.

For example, if the button’s name is of format “Submit<SomeDynamicId>”, the search property can be defined as –

uISubmitButton.SearchProperties.Add(“Name”, “Submit”, PropertyExpressionOperator.Contains);

 

  1. There is no concept of FilterProperties as supported in Web Technology in Coded UI Test.

 

 

How to handle search failures because of slow page loading?

 

If the XAP download takes a huge amount of time to load, the search during playback would fail since the Silverlight controls will not have been rendered in the visual tree. The internal search algorithm uses a wait and retry logic to search for a control while checking the visual tree rendering status at each wait interval (this time interval is upped exponentially on each iteration).

I will not be explaining the details here, but the important thing to note is that if there is no rendering happening within a polling interval, the search will return with failure status.

This polling interval is currently set to half of the Playback.PlaybackSettings.WaitForReadyTimeout which has a default value of 60 seconds (i.e. the default polling interval is 30 seconds). So to tackle slow page loading time, you can configure this Playback.PlaybackSettings.WaitForReadyTimeout to a desired value.

Note: Playback.PlaybackSettings.WaitForReadyTimeout does affect the normal search failure time in scenarios where there is some visual rendering happening in the Silverlight page. So you would need to strike an appropriate balance based on the type of application you are testing.

How To : Automate a Test Case in Microsoft Test Manager & Setup Automated Build-Deploy-Test Workflows

To automate a test case, link it to a coded test method. You can link any unit test, coded UI test, or generic test to a test case. You’ll want to link a test method that performs the test described by the test case. Typically these are integration tests.

The results of automated and manual tests appear together. If the test cases are linked to backlog items, stories, or other requirements, you can review the test results by requirement.

You can make links one at a time, or you can generate test cases from an assembly of test classes.

  1. Using Visual Studio, create or choose a test method. It can be an ordinary test method, a coded UI test, ordered test, or a generic test method.

    Check the method into Team Foundation Server.

    Keep the solution open in Visual Studio.

  2. Open the test case in Visual Studio.
    Open Test Case Using Microsoft Visual Studio
  3. Associate the test method with your test case.
    Associate Automation With Test Case

    If you want to change or delete the association later, choose Remove Association.

We don’t recommend linking load tests or web tests to test cases.

  1. Open a Developer Command Prompt, and change directory to the output director of your Visual Studio solution.

    cd MySolution\MyProject\bin\Debug

  2. To import all the test methods from the solution:

    tcm testcase /collection: CollectionUrl /teamproject:MyProject /import /storage:MyAssembly.dll /category:”MyIntegrationTestCategory

    The category parameter is optional but recommended. You only want to create test cases from integration or system tests, which you can mark by using the [TestCategory (“category”)] attribute.

  3. In the Test hub in Team Web Access or in Microsoft Test Manager, use Add Existing to add the test cases to a test suite.
Ads by CeheuapMMeAd Options

Provide the build location so that the test method can be found.

  1. In Microsoft Test Manager, choose Testing Center, , Properties.
  2. Under Builds, set Filter for builds. You can set the build definition and quality attribute of the builds you want to choose from.
  3. Choose Modify to assign a build to the test plan. You can compare your current build with a build you plan to take. The associated items list shows the changes to work items between the builds. You can then assign the latest build to take and use for testing with this plan. For more information, see What development has been done since a previous build?.
I’m not using Team Foundation Build to build my application and tests. How can I run automated lab tests?
Create a build definition that contains just the location where your assemblies are shared. Then create a fake instance of this build from the developer command prompt:

TfsCreateBuild.exe /collection:http://tfsservername:8080/tfs/collectionname /project: projectname /builddefinition:”MyBuildDefinition” /buildnumber:”FakeBuild_1.0″

Specify the build definition in your test plan.

To run your automated tests tests using Microsoft Test Manager, you must use a lab environment. It must have roles for each of the client and server machines used in your tests. (If you’ve used lab environments for manual tests, notice that automated tests must have a machine for the client role.)

  1. Create or choose either a standard lab environment or an SCVMM lab environment.

    If you create a new environment, choose a machine for each role.

    The machines tab in the new environment wizard.

    If you’re planning to run coded UI tests, configure it on the Advanced page of the wizard. This sets the test agent to run as a user. You have to supply a user name under which the agent will run.

    We recommend that you use a different user account than the lab service account used by the test controller.

    The advanced tab in the new environment wizard.
  2. Set the test plan to use your environment for automated tests.
    Automation on test plan properties
  3. If you want to collect more than the basic diagnostic data from the test machines, create a test settings file.
    New test settings

    In the test settings wizard, choose the data you want to collect for each machine.

    Select diagnostics for each machine role

Start automated tests the same way you do manual tests.

In Microsoft Test Manager, choose Testing Center, . Then select a test suite or an individual test and choose .

If you want to run a test in a different environment or with different test settings, choose Run with Options.

If you want to run an automated test manually, choose Run with Options.

If you have multiple build configurations, the test assemblies to run the automated tests are searched for recursively from the root directory of the build drop folder. If it is important which assemblies are selected when you run your automated tests, you should use Run with options to specify the build configuration.

Ads by CeheuapMMeAd Options
  1. In Microsoft Test Manager, choose Testing Center, , Analyze Test Runs.
  2. Double-click a test run to open it and view the details. You can:
    • Update the title of the test run to reflect the outcome.
    • Choose Resolution to indicate a reason, if the test failed.
    • Add comments.
    • View the details of an individual test.
    • Create a bug.
Q: Can I generate the test method from a manual run of the test case?
Yes. Verifying Code by Using UI Automation (Various Blog Post can be found on my Blog about Coded UI Test Automation)

Q: I want my automated test to repeat with different data. Do I use the same test parameters that the manual version of the test case uses?
To make the automated test iterate over different data, write that into the code of the test method.

Test parameters are only used with the manual version of the test. They aren’t visible to the automated test code.

Automated build-deploy-test workflows

You can use a build-deploy-test on Team Foundation Server to deploy and test your application when you run a build. This lets you schedule and run the build, deployment, and testing of your application with one build process. Build-deploy-test work with Lab Management to deploy your applications to a lab environment and run tests on them as part of the build process.

If your lab environment is an SCVMM environment, you can also use workflows to create and restore snapshots that automatically create a clean environment before you run tests, and to the state of your environment when a test fails. This ensures that each test isn’t influenced by changes to the lab environment from previous test runs. In addition, it ensures that testers can accurately reproduce that state of a lab environment when they reproduce bugs.

Requirements

  • Visual Studio Ultimate, Visual Studio Premium, Visual Studio Test Professional

You can use a build-deploy-test in the following scenarios:

Tip Tip
Build, or Build and Test: If you are building your application in a drop folder without deploying it to a lab environment, then you can use the default build process template. For more information, see Use the Default Template for your build process. If you also want to test your application without deploying it, see Run tests in your build process
  • Build, Deploy, and Test − Build your application, then deploy it and run tests on it in a lab environment. This workflow enables you to run a series of tests from a test plan, on a deployed application, as part of your build process. This scenario is common when running build verification tests.
  • Deploy and Test − This scenario is similar to the “build, deploy, and test” scenario, except a new build isn’t created during the workflow. Instead, the workflow uses an existing build from a drop folder.
  • Deploy Only – Deploy an existing build from a drop folder to a lab environment without running automated tests during the workflow. Once a build has passed your build verification tests, and is ready to be sent to a test team, you might want to send that build to the test team so they can run additional tests that aren’t part of your workflow. This scenario is common when running manual tests.
  • Build and Deploy – This scenario is similar to the “deploy only” scenario, except a new build is created during the workflow.

A build-deploy-test workflow is a Windows Workflow file that defines how a build definition will run a build, deploy an application, and run tests. A build-deploy-test workflow is created in a build definition by choosing a build process template called the lab default template (LabDefaultTemplate.11.xaml), and configuring the settings.

You can also create a customized build process template for your workflow depending on your requirements. You configure your build definition after you set up your build machine, test machines, and lab environments.

The deployment settings in a -deploy-test workflow define how an application is deployed by specifying the deployment scripts to run on in your lab environment. You can specify a lab management role to run each deployment script on, or you can specify a machine in your lab environment.

Creating deployment scripts is a major part of setting up -deploy-test workflows. Deployment scripts copy files from your build to your lab environment, and then run your installation packages.

The following diagram describes how a build is deployed by a build-deploy-test workflow:

Dataflow for deployment scripts.

The following steps are displayed in the diagram above.

  1. The build-deploy-test workflow starts a build, and then gets the deployment scripts.
  2. The build definition copies the build files to the drop location.
  3. The workflow runs each deployment script in the working directory of the specific machine or machine role that the script is assigned to.
  4. Each deployment script retrieves build files from the drop location.
  5. Each deployment script copies or installs the specified build files onto machines in the lab environment.