In the classic movie, Monty Python and the Holy Grail, King Arthur has an encounter with “Dennis” regarding their form of government. At one point, Dennis exclaims “Ah, now we see the violence inherent in the system!” While there will be no violence, I hope to contrast the benefits of Tosca (elaborated in a separate post) with the inherent systemic limitations of a solution like Selenium.
Selenium was initially released in 2004 but became much more popular in 2011 with the release of Selenium 2.0 and the incorporation of WebDriver. As open source tools became more accepted in the enterprise, organizations began to look to Selenium as an alternative. (It could be argued that Mercury Interactive’s tools WinRunner and QuickTest Pro were the dominant tools at the time. In the real world, selenium sulfide is a treatment for mercury poisoning.)
While tools like WinRunner, QTP, Tosca and other have built in development environments, Selenium is purely a set of libraries. Some see this as a benefit as users can leverage experience in Ruby, Python, C#, etc. But that said, you need considerable programming experience to leverage the libraries.
As discussed in my prior post, a key benefit of Tosca is its inherent “separation of concerns” (data and objects). While Selenium facilitates separation of concerns through the Page Object Model, it still requires design diligence to avoid test maintenance problems:
- Selenium alone doesn’t enforce separation of concerns. Without proper design, disorganized code can still occur.
- Page objects can become bloated and contain too much logic rather than just encapsulating page interactions.
- Lack of abstraction can lead to exposing underlying Selenium code across tests.
- No formal organization of reusable actions leads to code duplication.
- Hardcoded test data and environment details embedded in tests reduces maintainability.
- Mixing test logic across page objects, utility methods and test cases.
- Not isolating setup/teardown makes tests interdependent and fragile.
Selenium by itself does not guarantee well-structured code. Teams need engineering rigor to:
- Refactor bloated page objects into smaller, focused ones.
- Build a library of custom actions for common logic.
- Externalize test data from code into files/databases.
- Create setup/teardown modules for test environment.
- Follow coding standards and reviews.
- Use frameworks like TestNG to add organization.
- Minimize duplication through reusable utility classes.
- Modularize test code for readability and maintainability.
So while Selenium facilitates separation of concerns in principle, teams need coding discipline to fully realize maintainable test automation. Unfortunately, with many competing priorities, this discipline is often understandably neglected.
Yes, Selenium is free. But it takes teams of experienced user to effectively use the technology. These users themselves are expensive…and often better suited to tasks like application development instead of test creation and maintenance.
Customers frequently switch from Selenium to Tosca because they can not achieve their desired levels of test automation. Customers often report struggling to get to 15% test automation with Selenium while achieving 85-95% coverage with Tosca. And those who do accomplish high levels of test automation with Selenium acknowledge that the associated costs are exceedingly high.