A Tool for the Subjective Assessment of Speech System Interfaces (SASSI)


Speech input and output technology is expected to play a huge role in enabling future user interactions with computers. One of the most exciting opportunities is the potential for voice to offer access to e-content and services while people are on the move.   However, automatic speech recognition systems, even with highly limited vocabularies, cannot guarantee 100% performance.  Speech output can sound far from natural.  Creating usable systems within these constraints remains a challenge.

This project will make an important contribution to the aim of producing more usable speech systems by creating a tool for the Subjective Assessment of Speech System Interfaces (SASSI). Many researchers and developers agree that positive user attitudes towards speech systems are vital if those systems are to succeed.  Ultimately people will not use systems which they rate poorly.  However, the methods currently used in subjective evaluations of speech systems are unproven in terms of their reliability and validity. The development of a proven measurement tool will have a significant impact on speech system development.  It will provide a benchmark by which developers can compare their system to existing successful systems.  It will allow changes in the usability of a product to be tracked through its design cycle.  It will also contribute to longer term theoretical research aiming to determine which attributes of speech systems contribute to positive user attitudes. The findings of such research will allow developers to design better systems from the start.

Initial development work on SASSI has been conducted (Hone and Graham, in press, download paper in postscript format) while the authors were supported by previous grants from UK government research councils.  This research has produced a 34-item questionnaire measuring six factors which are hypothesised to contribute to user attitude towards speech recognition systems:  system response accuracy, likeability, cognitive demand, annoyance, habitability and speed.  Further work is now needed to turn this theoretical starting point into a real, validated, score-able tool.
 

The SASSI Research Proposal

Dr Kate Hone is preparing a research proposal to be submitted to the UK Engineering and Physical Sciences Research Council, seeking funding for a three year project to develop the SASSI tool.  Dr Hone is a lecturer in the Department of Information Systems and Computing at Brunel University and has a successful track record of research in this area (see Kate Hone's speech research page).  She is currently looking for industrial collaborators to support the SASSI project and thus ensure its relevance to the needs of industry.  There are two main modes of collaboration: In return the collaborating organisations would have early access to both the tool and research results.  Those companies unable to commit resources (either directly or indirectly) are encouraged to write a letter of support stating how the results of the research would be of value to them and to industry in general.  Should the research be funded there will be an opportunity for collaborating organisations to use early versions of the SASSI tool in return for anonymous use of their data.

Further details of how the EPSRC supports collaboration between industry and academia is available from their website at: http://www.epsrc.ac.uk/

For further information please contact:  Dr Kate Hone, Department of Information Systems and Computing, Brunel University, Uxbridge, Middlesex, UB8 3PH.
Email: kate.hone@brunel.ac.uk. Tel: +44 (0) 1895 203397.


[Kate Hone's Home Page]