Work Package 5
UCLan occupied the position of the evaluator in this WP
This important Work Package concentrated on evaluation. It therefore included the identification of an evaluative framework to be used in relation to a) the teacher training and pilot testing and b) field testing. Appropriate criteria were identified and developed in communication with consortium members in a collaborative approach beginning during the initial stages of the project and continuing throughout the key phases. Important aspects of this related to developing criteria to evaluate the digital skillsets of learners, teacher-trainers and teachers using machinima and virtual environments to teach and learn.
A mixed methods approach guided and informed the process, utilising both quantitative and qualitative approaches to aid triangulation and the reliability of the data. Interviews, surveys and focus groups were used, as well as observation of teachers and learners, either directly and/or via screen capture software to examine their interactions during the creation and use of the machinima in the virtual environment. In this way data were collected from participants to construct a detailed insight into their interactions.
While a smaller sample of participants were engaged in semi-structured interviews during the pilot phase and field-testing, all participants completed detailed quantitative surveys. Qualitative data were analysed using a thematic analysis, while quantitative data were analysed with appropriate descriptive statistics. Data were mapped against each other to construct a thorough and sensitive understanding of synergies and challenges identified. Both interview data and survey data were digitally analysed with the use of computer applications (e.g,. Transana, SPSS, Google Forms). Findings were shared with the partners via a collaborative document (e.g., wiki) and impacted on subsequent stages of CAMELOT in a cyclical effect.