Top Five Things to Consider in a Test Automation Framework was originally posted on the main page of LDSTech. It was written by Bryan Robinson.
-----------------------------------------------------
Software quality assurance has become more and more prevalent over the last several years. The demand to produce high-quality, reliable software is ever-increasing, and developers and quality assurance engineers strive to fulfill this demand. In doing so, one cannot overlook the need for a well-organized test automation framework.
A test automation framework has various benefits for the project. It can allow the testers to spend their time testing new development, rather than regressing the system to find where new code has introduced bugs in existing code. Manually testing the entire application would simply take too long and is prone to errors.
In my opinion, several points can be considered in a test framework. Here are the top five on my list and how our team uses each item.
1. Test Script modularity: This concept encourages testers to write code in an object-oriented manner. This allows reusable code to be incorporated throughout any given test. It can also assist in set-up tasks, such as preparing an environment or creating or refreshing database tables.
We use WatiN, NUnit and CruiseControl.NET. We have created a useful wrapper for WatiN (which we call WatiNQA). This wrapper gives us added XPath support that is not native to WatiN. C# gives us our test modularity. The WatiNQA wrapper is separated from the actual test framework to maintain the modularity of the frameworks in use.
2. Data-driven or keyword-driven testing: You may have specific scenarios that require an object in a specific state. As a good measure of verification, you can use Microsoft Excel spreadsheets or database queries to gather information to be tested. You can also leverage this to include database verification after a task has been completed.
We query the database to find specific data scenarios. This removes the process of setting up each data scenario through the user interface. After the test is complete, we verify that it is being saved correctly in the database.
3. Environment to support the test framework: This would include a continuous build system to launch a test suite at will, on schedule, or even after another event has occurred. With this environment, you can also run different suites of tests, such as smoke tests, functional tests, or browser-based tests. The ability to review test suite results and to view log files are necessary to provide feedback on the success or failure of the test suite. This helps ensure the tests are contributing to the testing efforts. It is also convenient to have a method of linking automated tests with their corresponding manual test cases and any bugs reported on each feature.
Using CruiseControl.NET with NUnit allows us to maintain the test framework with ease. We have it configured to launch several different configurations of test suites, and viewing the results is easy. We’ve also added the convenience of posting the automated test results directly into the test case management tool by creating a NUnit add-in. We use additional test decorators: <code>[Test, TestCaseKey(“”), BugKey(“”)]</code>.
4. Maintenance friendly: The framework offers an appropriate solution for the project at hand. It must be easy to use and maintain. By keeping clean code in an organized fashion, it becomes more of a tool than a hindrance. Documentation is always helpful when using the framework.
We include comments throughout the tests and supporting class files. The objects are configured to leverage reusable code to make writing a test a simple process. We also implement the Façade pattern for our helper methods. This helps us to simplify the tests but still have access to lower-level functions as needed.
5. The right tools: Even though many tools exist for automation use, not every tool is adequate for each project. A stable tool that has longevity can provide a standard of excellence on the project. When this happens, the framework is used more and used correctly. Also, the test results have a higher level of trust. This will help avoid automation frameworks to be discarded when changing personnel on the team.
We use C# and WatiN for several reasons. C# is a stable tool that has longevity and great documentation. WatiN is open source and has an active community working to improve it. Though it was originally created for Internet Explorer, a version is available which supports FireFox testing. WatiN deals directly with the Document Object Model (DOM) and can truly represent how the user would truly interact with the Web site.
Test automation is a full-time job. Something can always be automated to contribute to the project’s progress. It will take time to set up, but once it is established, it can add great value to the project.
Top Five Things to Consider in a Test Automation Framework
Discuss the feature articles on the Tech Home Page.
- McDanielCA
- Member
- Posts: 486
- Joined: Wed Jul 18, 2007 4:38 pm
- Location: Salt Lake City, Utah
Return to “Featured Article Discussions”
Jump to
- What's New
- ↳ Announcements & Policies
- ↳ Virtual Meetings
- ↳ In The News
- ↳ Links & Resources
- Church Technology for Members
- ↳ Main Church Website
- ↳ Calendar
- ↳ Ward Directory and Map
- ↳ Leader and Clerk Resources
- ↳ Notes and Journal, and Online Scriptures
- ↳ Online Donations
- ↳ Beta Testing
- ↳ Beta Calendar
- ↳ Beta Directory
- ↳ Beta Lesson Schedules
- ↳ Beta Maps
- ↳ Beta Newsletter
- ↳ Beta Clerk Resources
- ↳ Other Websites
- ↳ Mobile Applications
- ↳ Gospel Library
- ↳ Android Gospel Library
- ↳ iPhone/iPad Gospel Library
- ↳ Gospel Library for Windows
- ↳ Gospel Living App
- ↳ Gospel Media
- ↳ Member Tools
- ↳ Android Member Tools App
- ↳ iPhone/iPad Member Tools App
- ↳ Mormon Radio Channel
- ↳ Church Account
- ↳ Family History
- ↳ FamilySearch Products
- ↳ FamilySearch Family Tree Application
- ↳ FamilySearch Indexing
- ↳ Family History Centers
- ↳ Genealogy Industry Issues
- ↳ Software Development
- ↳ Other Member Technologies
- ↳ Church Historical Timeline
- ↳ Emerging Technologies
- ↳ Accessibility to Church Technology Products
- ↳ ePublishing
- ↳ Audiobooks
- ↳ E-Books
- ↳ Interactive Books & Magazines
- Community Discussions
- ↳ General Discussions
- ↳ Family Safety with Technology
- Clerk Support
- ↳ Getting Started for New Clerks
- ↳ Reports and Forms
- ↳ Local Unit Finance
- ↳ Membership Help
- ↳ General Clerk Discussions
- Meetinghouse Technologies
- ↳ Meetinghouse Internet
- ↳ Clerk Computers
- ↳ Printers, Copiers, and Multi-Function Devices
- ↳ FAQs for Local Unit Copier Program
- ↳ Broadcasts and Conferences
- ↳ Interactive Virtual Meetings
- ↳ Non-Interactive Webcasting
- ↳ Stake Conference with Remote Participants
- ↳ Satellite
- ↳ Receiving Internet Broadcasts
- ↳ Rich Media in Lessons and Training
- ↳ Meetinghouse Audio
- ↳ Phones
- ↳ Other Meetinghouse Technologies
- ↳ Stake Technology Specialist Training
- Audiovisual Media and Emerging Technologies
- ↳ Getting Started
- ↳ Digital Media
- ↳ General Media Discussions
- Help & Feedback
- ↳ Help with this forum
- ↳ Archive
- ↳ Classic Ward & Stake Sites (LUWS)
- ↳ Ideas & Suggestions
- ↳ Adobe Flex
- ↳ Enterprise Change Management
- ↳ Personal Ancestral File (PAF)
- ↳ Temple-Ready
- ↳ FamilySearch Wiki
- ↳ Australia Area Technical Support
- ↳ Lesson Schedules
- ↳ Newsletter
- ↳ BlackBerry Gospel Library
- ↳ WebOS Gospel Library
- ↳ Windows Phone Gospel Library
- ↳ BlackBerry LDS Tools
- ↳ WebOS LDS Tools
- ↳ Windows 8 LDS Tools
- ↳ Windows Phone LDS Tools
- ↳ Feedback on Tech
- ↳ Gospel App & Game Contest
- ↳ Help Center Website (Formerly RKATS)
- ↳ Community Development Projects
- ↳ Emma
- ↳ Mobile Development
- ↳ Interactive Music Project
- ↳ Java Web Project Support (Stack)
- ↳ Featured Article Discussions
- ↳ Technical Recruiting
- ↳ Development Help Wanted
- ↳ Personal Video Conferencing
- ↳ MLS Support, Help, and Feedback