SleepReplacer: A Novel Tool-based Approach for Replacing Thread Sleeps in Selenium WebDriver Test Code
Assuring quality of web applications is fundamental, given their relevance in the today's world. A possible way to reach this goal is through end-to-end (E2E) testing, an approach in which a web application is automatically tested by performing the actions that a user would do.
With modern web applications (for example, single-page applications) it is of great importance to properly handle asynchronous calls in the test suite. In E2E Selenium WebDriver test suites, asynchronous calls are usually managed in two ways: thread sleeps and explicit waits. The first is easier to use, but is inefficient and can lead to instability (also called flakiness, a problem often present in test suites that makes lose confidence in the testing phase), while the second is usually more effective but harder to use because, if the correct kind of wait is not carefully selected, it can introduce flakiness too.
To help developers, who often opt for the first strategy, we propose in this work a tool-based approach to automatically replace thread sleeps with explicit waits in an E2E test suite without introducing novel flakiness. We empirically validated our tool named SleepReplacer on four different Selenium WebDriver test suites, and found that it can correctly replace in automatic way from 81% to 100% of thread sleeps, leading to a reduction of the execution time of the test suite that goes from 13% to 71%.
All the material concerning SleepReplacer (both the tool implementation and the test suites used in the empirical study) can be found at https://github.com/OlianasD/SleepReplacer
