Automated on-page usability testing

Usability testing is a human-intensive process that must compete for time with other activities in the software development process, such as core function development and functional testing. In the case of web-based applications, the problem is compounded by frequent updates and shorter release cycles. One way to counteract this problem would be to automate aspects of the usability testing process so that developers could receive immediate feedback on their work. The two aspects covered in this project are: a) form-filling time estimates for different groups of people (using keystroke-level model), and b) the aesthetics of a web page (based on several published metrics).

Following a review of the relevant literature, a number of usability metrics were designed and implemented as part of a prototype in the form of a browser extension. Using this extension, the developer would be able to select which usability aspect to evaluate. Two aspects are supported: the first uses the keystroke-level model (KLM) metrics [1] to estimate the amount of time it would take different types of users to complete an on-page form. This factors in multiple influencers, such as typing time, hand movement, mouse cursor movement, clicking time, thinking time, and so on. The second aspect considers automated aesthetic measurements through a combination of 13 metrics presented in [2]. These take into account: balance, equilibrium, symmetry, sequence, cohesion, unity, proportion, simplicity, density, regularity, economy, homogeneity, and rhythm. The results are colour-coded so that the developers gauge how well the metrics were followed, at a glance.

The effectiveness of the extension was evaluated through quantitative and qualitative analyses, in the form of online studies with various participants. The quantitative analysis consisted in comparing the extension’s predicted results with those produced during study-specific tasks by participants. Expert feedback was also sought to generate further insights on the developed tool and associated techniques.

References/Bibliography

[1] S. K. Card, T. P. Moran and A. Newell, “The keystroke-level model for user performance time with interactive systems,” Communications of the ACM, vol. 23, no. 7, pp. 396-410, 1980.

[2] D. C. L. Ngo and J. G. Byrne, “Application of an aesthetic evaluation model to data entry screens,” Computers in Human Behaviour, vol. 17, no. 2, pp. 149-185, 2001.