The abstract for this paper is by necessity incomplete. The completed paper will evolve as a result of discussions and further consultations with attendees at the MeLLow workshop to be held at Brunel University in October 2006.
Background: The MeLLow project is a 3-year EPSRC funded research project. The main aim of the project was to develop a tool that would enable practitioners to select, a priori, the optimal prediction technique for their particular circumstances. In order to develop the tool we conducted a systematic review in which we identified empirical studies of software engineering prediction techniques and categorised salient features of the final project. In addition, so that we might carry out a meta-analysis that would further identify biases and effects within the studies; we recorded the results from the studies in our sample. The major obstacle we encountered was the range of research methodologies used by software engineering researchers. Because of this, we have been unable to effectively compare and combine the results of studies identified through our systematic review. Instead our efforts have turned to promoting a more rigorous, structured and uniform approach to research in software engineering prediction techniques.
Method: At a workshop organized by the MeLLow researchers, attendees will be asked to discuss their understanding of both the definitions and processes of systematic review and meta-analysis. Responses will be recorded and synthesized and brought to bear on the reverse panel discussion held at the end of the workshop. A paper will be drafted and circulated to attendees for clarification.
Results: A paper detailing a common understanding of the definitions and processes involved in systematic review and meta-analysis.
Conclusions: Given the range methodologies and practices in studies investigating software engineering prediction techniques and a lack of common understanding of what constitutes systematic review and meta-analysis, we intend to exploit the expert opinion available at the workshop to reach a consensus that can be applied in future work. The full paper will follow.