Test Automation with Free Tools
Posted: Tue Sep 22, 2009 1:17 pm
Test Automation with Free Tools was originally posted on the main page of LDSTech. It was written by Ronald Jenkins.
-------------------------
Since the mid ‘90s, test automation has grown from a handful of crude macro-recording tools and custom-built one-off applications to a suite of high-priced, high-powered frameworks. While the frameworks tend to perform as advertised, the pricing typically leaves small software shops out in the cold and mid-size test teams struggling to justify the budget. The framework itself can also have some limitations imposed by the limited flexibility of the scripting language behind it.
In the last few years, the open source movement has produced a series of tools that the enterprising tester can combine into a free framework with all the power and flexibility of full-fledged programming languages. One combination that I’ve used to test various Web-based applications consists of nUnit and WatiN.
nUnit is based on the well-known jUnit unit testing framework. It’s geared towards applications that use the Microsoft .Net Framework. Its large following has produced a tool that’s free and performs most of the grunt work that an automated test framework needs: test execution and results reporting. There’s a community of developers contributing code, bug-fixes, and plug-ins for the framework. It’s easy to find people who know how to use it.
WatiN (watin.sourceforge.net) is an open source library written in C#. It’s useful for driving browsers for Web application testing. Originally designed to be IE-specific, it has grown in the last couple of years to include support for Firefox. Because it interfaces with IE through the Direct Object Model (DOM), anything that the browser sees in the DOM can be manipulated in code through a robust set of functions in this library.
Over the course of building various automation frameworks, I keep coming back to a design that seems to offer the sweet-spot of flexibility combined with light-weight coding requirements for test automation needs. It consists of a three-part system: page descriptors, page fixtures, and test scripts.
Page Descriptors
A page descriptor is a class that consists of a set of attributes describing each element on a single page. This allows you to standardize how the code refers to elements and gives you a single place to update the descriptors (name, id, index number, whatever) when they invariably change in the app. Breaking this information out into its own layer helps ensure that the rest of the framework remains agnostic to what the page actually uses, thus insulating the test scripts from the whims of the developers and/or development frameworks (that frequently and automatically specify and change element descriptors).
The query text field of the Google home page would have a descriptor that looks something like this:
<code>AttributeConstraint _searchField = Find.ByName("q");</code>
Page Fixtures
This is the class that encapsulates the functionality of the web pages into discrete methods. A method might be as simple as a single line of code for clicking a button or returning a label value from the page. It might be complex enough to cover multiple functions across multiple pages. How to design the fixture is up to you, but be aware that since you’re writing code that needs to be maintained, you must be careful not to introduce needless bugs into the process. Good design and programming habits are imperative.
The query text field of the Google home page would have a fixture that looks something like this:
<code>
public string SearchText
{
get { return _ie.TextField(_homeDescriptor._searchField).Value; }
set { _ie.TextField(_homeDescriptor._searchField).Value = value; }
}
</code>
Test Scripts
Test scripts are where the meat of the testing occurs. Functions from the page fixtures are used to drive the application and verify behaviors. How these are designed depends heavily on what tool you’re using to run your tests. In the case of nUnit, the test scripts are compiled into a DLL with attributes on the classes and methods.
Assigning a value to the text field in a test script then looks like this:
<code>_homeFixture.SearchText = "site:tech.lds.org Test automation";</code>
Performing an assert on the same field using nUnit would resemble something like this:
<code>Assert.AreEqual(_homeFixture.SearchText, "site:tech.lds.org Test automation");</code>
In general, your page descriptors and fixtures should remain agnostic to what you’re using to drive test scripts. The test scripts should likewise limit themselves as much as possible to the fixtures and remain agnostic to what goes on in the page descriptors.
Summary
While there are plenty of high-priced automation packages out there, an enterprising tester can easily create a framework from open source components, getting testing power and flexibility for much less cost.
Ronald Jenkins is a software engineer for the Church.
-------------------------
Since the mid ‘90s, test automation has grown from a handful of crude macro-recording tools and custom-built one-off applications to a suite of high-priced, high-powered frameworks. While the frameworks tend to perform as advertised, the pricing typically leaves small software shops out in the cold and mid-size test teams struggling to justify the budget. The framework itself can also have some limitations imposed by the limited flexibility of the scripting language behind it.
In the last few years, the open source movement has produced a series of tools that the enterprising tester can combine into a free framework with all the power and flexibility of full-fledged programming languages. One combination that I’ve used to test various Web-based applications consists of nUnit and WatiN.
nUnit is based on the well-known jUnit unit testing framework. It’s geared towards applications that use the Microsoft .Net Framework. Its large following has produced a tool that’s free and performs most of the grunt work that an automated test framework needs: test execution and results reporting. There’s a community of developers contributing code, bug-fixes, and plug-ins for the framework. It’s easy to find people who know how to use it.
WatiN (watin.sourceforge.net) is an open source library written in C#. It’s useful for driving browsers for Web application testing. Originally designed to be IE-specific, it has grown in the last couple of years to include support for Firefox. Because it interfaces with IE through the Direct Object Model (DOM), anything that the browser sees in the DOM can be manipulated in code through a robust set of functions in this library.
Over the course of building various automation frameworks, I keep coming back to a design that seems to offer the sweet-spot of flexibility combined with light-weight coding requirements for test automation needs. It consists of a three-part system: page descriptors, page fixtures, and test scripts.
Page Descriptors
A page descriptor is a class that consists of a set of attributes describing each element on a single page. This allows you to standardize how the code refers to elements and gives you a single place to update the descriptors (name, id, index number, whatever) when they invariably change in the app. Breaking this information out into its own layer helps ensure that the rest of the framework remains agnostic to what the page actually uses, thus insulating the test scripts from the whims of the developers and/or development frameworks (that frequently and automatically specify and change element descriptors).
The query text field of the Google home page would have a descriptor that looks something like this:
<code>AttributeConstraint _searchField = Find.ByName("q");</code>
Page Fixtures
This is the class that encapsulates the functionality of the web pages into discrete methods. A method might be as simple as a single line of code for clicking a button or returning a label value from the page. It might be complex enough to cover multiple functions across multiple pages. How to design the fixture is up to you, but be aware that since you’re writing code that needs to be maintained, you must be careful not to introduce needless bugs into the process. Good design and programming habits are imperative.
The query text field of the Google home page would have a fixture that looks something like this:
<code>
public string SearchText
{
get { return _ie.TextField(_homeDescriptor._searchField).Value; }
set { _ie.TextField(_homeDescriptor._searchField).Value = value; }
}
</code>
Test Scripts
Test scripts are where the meat of the testing occurs. Functions from the page fixtures are used to drive the application and verify behaviors. How these are designed depends heavily on what tool you’re using to run your tests. In the case of nUnit, the test scripts are compiled into a DLL with attributes on the classes and methods.
Assigning a value to the text field in a test script then looks like this:
<code>_homeFixture.SearchText = "site:tech.lds.org Test automation";</code>
Performing an assert on the same field using nUnit would resemble something like this:
<code>Assert.AreEqual(_homeFixture.SearchText, "site:tech.lds.org Test automation");</code>
In general, your page descriptors and fixtures should remain agnostic to what you’re using to drive test scripts. The test scripts should likewise limit themselves as much as possible to the fixtures and remain agnostic to what goes on in the page descriptors.
Summary
While there are plenty of high-priced automation packages out there, an enterprising tester can easily create a framework from open source components, getting testing power and flexibility for much less cost.
Ronald Jenkins is a software engineer for the Church.