Executable Test Specifications

The most advanced automation software such as Selenium, is aimed directly at the development community. This severely restricts its broader usability.

For any automation solution to be fully effective, the direct and continuous participation of business domain experts is a must. This is a great challenge.

One of the biggest pitfalls of test automation is one generally known as the pesticide principle. Test specifications may start off relevant but quickly degrade when an application, risk or business priority changes. If they stay in place as originally defined they will quickly lose their value. Specifications have to stay relevant and must be maintainable by those who need to change them.

The W3QA Framework is enabling to non technical testers and Business Analysts. All tests are specified using a non programming narrative, through the Human Readable - Machine Executable Specification (HRMES - pronounced hermes) format.

HRMES has a very clear and intuitive structure derived from FIT and Seleniums Fitnesse formats.

Business domain experts can develop and maintain the definition of test scenarios through Executable Specifications.

Designing usable automation frameworks usually means compromising on power and extendibility. Much like implementations of Selenium Core, which is highly usable but hard to build in advanced features. Selenium users at some point make a decision to migrate from Selenium Core to Selenium RC (SRC). If they want their test framework to easily control the flow of a test case or interact with databases and filesystems they have little option. Selenium allows for the easy migration from their html based IDE into any mainstream programming language. So migration is straightforward.

However, SRC is code driven and migrating comes with a cost of usability and will severely limit the continuing involvement of non-engineers. Which is some cases means the majority of domain experts and business stake holders.

HRMES works very much in the way Selenium IDE/Core works, but HRMES is based on SRC. SRC is very powerful but means rendering all your test specifications into code. HRMES preserves the usability of Selenium IDE/Core without the constraints on it's extensibility. You really do have power and usability with HRMES.

HRMES has the code abstraction and simple narrative of SIDE but without its functional limitations. This abstraction and independence from the automation code deliveries a very high level of test automation maturity. Archiving this criteria is widely recognised, by independent test specialists as one of the highest level of design criterion.

This separation of implementation and specification is pretty much unachievable with propriety automation software. Only an open source framework could deliver this flexibility in its design. However, OSS test software like Selenium do not come with frameworks. HRMES is a execution ready framework.

Low maintenance profile

The HRMES Framework breaks the link between writing more tests and having to write more code. This offers a highly maintainable solution.

Maintenance changes to HRMES are:

• Available instantly to system test and have no code integration and release cycles
• Implemented when the change is known rather than when the code is delivered
• Without involving very costly development resource
  

Business users can change the specification within HRMES without it requiring any changes to the automation framework.

Test specifications will harness the power of Selenium, without having to be written in code. This is as close to a code free paradigm as is possible.

In terms of classification, the W3QA Framework is a Action Based Test (ABT) framework. The W3QA Framework works through implementation mappings to the Selenium API.

ABT frameworks offer a high degree of usability and specification abstraction. They usually come with the cost of custom building the entire mapping between the specification layer and application layer.

However, as the W3QA Framework uses Selenium, all action mappings are open sourced and have been fully mapped to the W3QA Framework using the Seleniums API. This solution is available out of the box and under a free license.

The HRMES format is a simple, flexible and accessible XML based datasource that can be loaded via commonly used spreadsheet packages such as MS Office or OpenOffice.

Making use of OpenQA open source XUL based UI, this format can be fully generated via the Selenium IDE.

The W3QA Framework, including the HRMES format is open source and a free license is available upon request.