Ten Concepts for Successful Automation Scripts

Ten Concepts for Successful Automation Scripts By Edward J. Correia

September 9, 2008 — There isn't much use in automating a testing process if the automation scripts can't be used again and again. The more such scripts have to be tweaked or modified to be applied to the latest revision, the more likely they are to end up in the trash. It pays to develop scripts that are easily maintained so that any necessary changes are minimal and benefit to the project is maximized.

"Most [people] think of automated testing as a way to record then play back various functions from an application, but this is just a small part of this type of testing," says Matthew Hoffman, manager of systems verification and validation at Concurrent Technologies, a software development services consultancy. He offers 10 things an automated tester can do to keep tests useful and free of major maintenance requirements.

He says it's helpful for the automation team to have some software development experience. A basic understanding of programming concepts is helpful for tasks such as setting variables, using regular expressions, creating loops and extracting header data.

It's also a good practice to plan out your test scenarios ahead of any automation activities. "It is crucial to design the tests before recording the scripts. Many testers will [also] examine the functional specifications and design tests to cover the requirements," he says. Useful questions an automated tester should ask when designing the scripts include:

≫ What areas of the application will be reused on each/several of the scripts (i.e., login and logout)? ≫ Are there any variables that will be reused on each/several of the scripts (i.e., user name, password, hostname and port number)?

Organization of the tests is a key ingredient of successful test automation, Hoffman says, adding that test scripts should be organized with modularization in mind. "The scripts should be structured to parallel the flow of the functional areas of an application." For example, if your script is testing an invoicing system, the script might be organized with the following sections:

≫ Login ≫ Create Invoice ≫ Edit Invoice ≫ Delete Invoice ≫ Logout

Scripts should clean up the data created. "It is important to clear any data that was created by the automated tests," he says. Remnants of test data can wreak havoc for the development team and the functional testers, particularly when they're verifying system data. For example, if your test script creates an invoice, there should be a corresponding script that deletes the invoice when tests are done. "It is also useful to enable logging within the automated test tool to verify that information was created [and] deleted."

Be certain of your assertions. Assertions determine whether a test passes or fails, and they are critical components of any successful test script. They should be kept simple and used to validate individual program components, such as:

≫ The page name displayed for each request ≫ The correct fields being displayed ≫ A message displayed for incorrect data ≫ Correct text color (i.e., a required field might need red text) ≫ The proper functions/buttons being enabled/disabled ≫ Correct confirmation messages

Scripts should test for negative scenarios. "The purpose of any type of testing is to break the system before the customer breaks it." Hoffman suggests using automated scripts to attempt scenarios that can break the system, such as entering bad data into forms, entering incorrect login information, submitting data and trying to navigate back to a previous page to re-enter duplicate data, and entering incorrect credit card information.

Tests should execute on various environments. Applications often are meant to run in alpha, beta and production environments, so each environment will need to be tested. Scripts must be flexible enough to run in each of the various environments. Hoffman offers these suggestions for creating scripts that will execute on these environments:

≫ Use regular expressions when asserting specific data ≫ Differentiate the environments by passing a variable with the alpha, beta or production URL at runtime

Use configuration management. The automated scripts are essentially development files, so basic configuration management practices should be applied to the process of creating and managing them. "The scripts should be in source control software at all times. The test team should develop a document that describes the deployment of the scripts within a development and production environment. The document should also consist of script development standards so each person on the team creates the scripts in the same manner."

Schedule regular test runs. To ensure that a system is functioning correctly at all times, it can be beneficial to schedule automated tests on a regular basis. The scheduler can be set up to run when all users have logged off the system or in the early mornings before anyone has started their work.

Report and distribute results. Results should be reported after the scheduled tests execute, and then sent to the development team for analysis. Not all script failures are due to faulty code. "Many times the scripts fail because of a network outage, change to the system or a script error. The automated test developer must determine what kind of error[s] are in the reports and communicate it with the team."

Automated testing is more than a matter of record and playback. To give your automation project the greatest chance for success, it's important to apply consistent concepts—to organize, execute and report—to minimize script maintenance and maximize benefit.

In October's issue of Software Test & Performance magazine, automation expert Elfriede Dustin describes some of the major problems that lead to automation project failure and offers sage advice on how to avoid them. For now, don't miss the September issue of Software Test & Performance magazine, available now for free download. This month's issue packs a .NET punch!

Share this link: http://www.sdtimes.com/link/32824