E-Learning Maturity Model
Getting Started

The eMM is designed to support the assessment of institutional e-learning capability. Depending on your particular institution's characteristics, context and resources, the eMM can be applied in a variety of ways. This document will help you work through the decisions needed before starting an eMM assessment and guide you in establishing an assessment project. The actual details of assessing capability are described elsewhere.

Individual decision areas:

• Familiarising yourself with the eMM
• What type of outcome do you want from benchmarking
• Involvement of institutional leadership
• Collaborative context
• Review of the eMM processes and practices
• Identifying exemplar courses

Familiarising yourself with the eMM

If you have not already done so, it is strongly recommended that you read through the material provided in the eMM Key Concepts section. You may also find reading the 2006 New Zealand project report, particularly Part Five helpful.

What type of outcome do you want from benchmarking

Before commencing an assessment activity using any methodology it is useful to ask yourself what type of information you need and how you might use that information to improve your institutions e-learning activities. The eMM is not the only benchmarking methodology available to institutions.

Benchmarking means many things to different people and encompasses a wide range of possible activities:

• To audit the quality of an institution
• To demonstrate compliance with external standards
• To rank institutional performance across a sector
• To provide a description of process performance over time
• To determine what are the processes and practices that support effective e-learning
• To identify areas of institutional weakness and strength
• To assist strategic and operational planning and decision making
• To identify examples of good practice for analysis and emulation
• To provide an external point of reference regarding current processes and practices
• To determine the costs of current processes and practices
• To identify and forecast key trends
• To discover emerging technologies
• To search for best practice in multiple industries

The items in bold represent benchmarking objectives that the eMM is well suited to address. A key aspect of the eMM is that it does not rank institutions, but rather acknowledges the reality that all institutions will have aspects of strength and weakness that can be learnt from and improved. Any benchmarking approach that presumes particular e-learning technologies or pedagogies is unlikely to meaningfully assess a range of institutions within a single country, let alone allow for useful international collaboration and comparison, particularly over an extended period of time.

Before proceeding further you should be able to describe what questions about your organisation will be answered by the assessment process and subsequent analysis. Ideally these should be phrased in strategic terms and should not presume that your current knowledge of your organisations e-learning strengths and weaknesses is complete. A common, and valuable, outcome from applying benchmarking is the discovery of unanticipated information.

Involvement of institutional leadership

Having started defining the direction for your e-learning benchmarking activities it is vital that you involve an appropriately senior manager or managers. Early involvement of senior managers helps ensure that you will have support in obtaining the necessary resources to undertake an assessment and access to the information needed to inform the assessment itself. It also means that the process of motivating change in response to the outcomes of the assessment will be less difficult (although change is always difficult to motivate.

Identifying the appropriate level of management to engage with will depend very much on your own contacts, your organisational culture, and the level at which change can be realistically motivated. Ideally, you should involve managers at the level at which you are intending to make internal comparisons (see below) as this provides an immediate audience for the assessment information and a mechanism for acting on it.

Collaborative context

Management involvement is also essential if your assessment project is intended to involve collaboration with other institutions. Generally collaboration in benchmarking is found to be an important part of the process. By working with another institution engaging in a similar exercise you are able to discuss your assessments with an independent, but informed, observer. The different institutional perspectives and means of approaching the same practices and processes also helps identify options for improvement. On a personal level, the support and assistance obtained by working with colleagues on a similar task should also not be underestimated.

Many institutions already have collaborative arrangements with others, particularly in the area of e-learning. These can form a solid basis for a collaborative benchmarking project and the resulting assessments are potentially much more useful and pursuasive to institutional leadership as they allow comparisons with known and familiar partners.

Institutions interested applying the eMM are encouraged to contact Stephen Marshall as a number of collaborative projects are currently underway or under consideration and additional participants are often solicited.

Assessment by institution or within

A particularly important consideration, alluded to above, is whether to conduct a single eMM assessment for your entire organisation, or to conduct a series of semi-independent assessments within the organisation. Either approach is supported by the eMM.

Possible to entire organisation assessments alternatives include:

• Faculty, school or discipline level assessments
• Mode of delivery assessments (e.g. face-to-face vs distance)
• Geographically separated campus assessments

The exact division will depend on your circumstances and the questions you wish to address. Whatever division is used, it is important that it has meaning to your organisation's management and that you have access to the necessary information on each subdivision.

Where possible, your assessment project should include a representative from each subdivision able to assist in identifying the information sources used for the assessment and able to provide an informed context to any analysis.

Review of the eMM processes and practices

Prior to conducting the assessment you should review the assessment workbook and ensure that the processes and practices are meaningful in your context. The processes in the current version of the eMM are designed to be generally applicable but individual practices may not have any value in some contexts.

It is also possible that areas of particular concern in your context are not apparent in the eMM process set. One of the goals of the research informing the eMM identifying those factors that ensure success in e-learning and it is unlikely that these have all been identified in the research to date. However, the eMM is a complex model and it is possible to miss particular factors which are present.

However, if you have identified evidence for factors that are important for successful organisational e-learning implementations which are not present in the eMM, please contact Stephen Marshall with the details. The eMM is constantly evolving and contributions are welcome.

Identifying exemplar courses

The final step for planning the assessment is identifying the courses that provide the evidence of institutional capability. These courses should be selected as being representative of the particular institutional context being assessed, rather than being special or unusual examples. They should also be chosen on the basis of availability of the people involved in the development of the e-learning aspects of the course and the associated course and development documents. These courses, the people involved, and the documents form a core part of the evidence used to support assessments. This ensures that the assessment is being made on the basis of actual performance, not intended or idealised performance. Depending on the size and complexity of the courses you should need three to five course examples.