E1.3.1

=Institutional policies define requirements for student evaluations of the educational effectiveness of e-learning initiatives. =

Evidence
Attwell (2006) provides a comprehensive guide with examples of tools that can be used in the evaluation of e-learning programs. Attwell emphasizes the need for iteration between theory and practice. This means courses must be redesigned in an ongoing manner according to evaluation. Also, due to the initial costs of implementing e-learning programs it is important to conduct ongoing evaluation. Attwell further notes that a large portion of the evaluation literature on e-learning focuses on descriptive rather than analytic or predictive studies. There are surprisingly few robust comparison studies of e-learning compared with traditional learning. There are also surprisingly few return on investment studies. There is concern that e-learning is sometimes not succeeding in the way that had been expected, hence the need for evaluation and refinement. Attwell points out the ongoing need for technical developers and evaluators to engage in dialogue. Attwell identifies five areas of evaluation: individual learner variables, learning environment variables, contextual variables, technology variables, pedagogic variables. There is a need to undertake interpretative and analysis studies rather than merely descriptive ethnographic studies when evaluating e-learning.

Attwell describes the context that managers of e-learning are operating in: ‘Managers… are having to make decisions about the introduction and use of e-learning when e-learning itself is still in a stage of rapid evolution and instability. Major paradigm shifts are taking place in the pedagogical thinking underpinning e-learning, new ideas and policies are emerging on how e-learning should be developed and financed and there are continuing advances in information and communication technologies. It is in this context that managers are having to make decisions about investing in e-learning and one in which the consequences of making the wrong decisions are increasingly costly’ (2006, p. 40).

Resources
Evidence of capability in this practice is seen in the inclusion of a formal student evaluation plan in the design and development of projects and courses. This plan should include conducting multiple formal evaluations, both summative and formative, in a standard way that allows for comparison of results between projects and over time.

Given that there are four types of evaluation: context, input, process, product, Attwell (2006) describes a tool for the evaluation of e-learning based on these (derived from Stufflebeam’s CIPP model, http://www.wmich.edu/evalctr/checklists/cippchecklist.htm ).

RMIT University has a policy document setting out the way that student feedback ought to be consistently collected: http://www.rmit.edu.au/browse;ID=9pp3ic9obks7