Evaluation

Evaluation is often at the end of systematic performance models.  The Instructional Systems Design (ISD) model, sometimes referred to the ADDIE model, includes the following stages: analysis, design, development, implementation, and evaluation.  Other disciplines practice models similar to the ADDIE model.  Six Sigma practices the DMAIC model: define, measure, analyze, improve, and control. In the DMAIC model measure would be comparable to evaluation.  Human Resource Development (HRD) practices analyze, propose, create, Implement, and assess.  In the HRD model, assess refers to assessment which is the same as evaluation in the ADDIE model. 

These systematic performance models are often viewed as linear, stet-by-step, models.  By viewing these models this way they become ineffective at improving performance for the long-term.  Each model is presented to be cyclical and interactive.  This means that each model is designed as a continuous improvement cycle with dynamic interactions between each stage.  In the case of evaluation, this stage affects each of the other four stages in the process.  Evaluation begins during the initial analysis phase and continues through each stage, then re-cycles again, as improvements to the new improved cycle are incorporated.  Wang and Wilcox (2006) support this view indicating: “the larger view of evaluation may not be treated as a separate phase during the process…. It is indeed an ongoing effort throughout all phases of the ADDIE process and culminating at the last phase” (p. 528).

Shrock and Geis identified evaluation as a “process of collecting information and feeding it back to those who need the information so that the system can succeed” (as cited in Stolovitch & Keeps, 1999, p. 185).  Evaluation should be designed to provide feedback during each stage in the process so that improvements can be made to the process.

Evaluation comes in two forms: formative evaluation and summative evaluation.

Scriven (1991) identified formative evaluation to be used “to provide information on improving program design and development” (as cited in Wang & Wilcox, 2009, p. 529).  Wang and Wilcox identified that the purpose of formative evaluation was “to identify weakness in instructional material, methods, or learning objectives” (p. 529). Formative evaluation can be used to evaluate the instructional methods during a training program. 

Following the training program summative evaluation will be used to determine the long-term effectiveness of the program and its instructional methods, including learning transfer.  Brown and Gerhardt (2002) described summative evaluation as those “efforts that assess the effectiveness of completed interventions in order to provide suggestions about their use” (p. 952).  A training program can be evaluated by its impact on the organization and its long-term effectiveness through summative evaluation.

A successful evaluation is one that utilizes both formative evaluation and summative evaluation.  Evaluation needs to be viewed as an iterative process that affects each of the stages in the training process that it is measuring.  Each systematic performance improvement endeavor needs to be addressed as a continuous improvement cycle with a strong emphasis on evaluation.  Evaluation is the key component that makes the systematic performance improvement process a continuous effort, allowing improvements to be made to the process during each stage.

REFERENCES

Brown, K. G. & Gerhardt, M. W. (2002).  Formative evaluation: An integrative practice model and case study.  Personnel Psychology, Vol. 55, pp. 951-983.

Stolovitch, H. D. & Keeps, E. J. (1999).  Handbook of human performance technology: Improving individual and organizational performance worldwide (2^nd ed.).  San Francisco, CA: Jossey-Bass Pfeiffer.

Wang, G. G. & Wilcox, D. (2006).  Training evaluation: Knowing more than is practiced.  Advances in Developing Human Resources, Vol. 8, No. 4, pp. 528-539.