A Framework For All Levels Of Testing?

Thinking purely in terms of Selenium RC for a moment, I find it hard to find anyone who does not believe that they have a test automation framework. One minute someone has a Selenium IDE test suite and then suddenly they have a Selenium RC framework.

Often all they really have is a test suite usually written in Java for JUnit. So is this really a test framework? I would say; not without qualification. Unit test frameworks are not easily extended to cover tests at all test levels, so remain primarily and solely "unit" test frameworks.

Unless you think automating unit tests is sufficient, you will need a framework that is good for all test levels or different ones focused for particular test levels.

Almost all Selenium RC implementations are fairly technical and have been conceived within the development domain. Most have been written in Java and most of them written for JUnit. The things I would highlight at this stage, is the words "unit" and "development".

Unit test frameworks are there to support testing at the unit test level. Automation conceived exclusively with a view to support development objectives are unsurprisingly focused on doing so.

However, the biggest things determining the success of test automation projects comes down to what it produces in test assets and how in engages test stakeholders.

Frameworks are enabling and there are a whole range of capabilities a framework can enable. Which makes it easy to call anything a framework - even those that do very little.

Test automation frameworks have to fully enable the usability of test assets and the continuous involvement of it's non-developer stakeholders. Without this a framework can enable reporting, debugging, test data abstraction but still remain deficient.

Unit test frameworks, even good ones like TestNG, are enabling to developers and development methodologies. Development methodologies are neutral to automation and test automation methodologies and practices.

So can test automation conceived, built and operated in the development domain, like 95% plus of Selenium RC implementations really achieve all the goals of a test automation framework? You know my answer, but what do you think?