Training Evaluation Misses the Critical Thinking Dimension

For typical training courses the following four or five evaluation steps are the norm:

Level 1: Reaction and Perceived Values

Measures reaction to, and satisfaction with, the medium, content, and value of the project or program.

Level 2: Learning and Confidence

Measures what participants understand or learned from the project or program (information, knowledge, skills, and contacts).

Level 3: Application and Implementation

Measures what participants understand or learned from the project or program (information, knowledge, skills, and contacts).

Level 4: Impact and Consequences

Measures progress after the program implemented (the use of information, knowledge, skills, and contacts).

Level 5: ROI

Monetary Benefits.

(Phillips, & Phillips, 2007)

Evaluation is further divided into formative and summative evaluation, with formative evaluation relating to evaluating the training program and summative evaluation relating to the long-term effects of the training program (see Evaluation). Level 2, Learning and Confidence, is more of a formative evaluation measure and identifies whether trainees learned the material presented in the training. Level 3, application and consequences, is categorized as summative evaluation in which it looks at whether the trainees on-the-job behavior represents their learning from the training program.  

For these two specific levels of evaluation Phillips and Phillips (2007) highlighted information, knowledge, skills, and contacts. Providing employees (trainees) with the information that they need to conduct their job functions is critical. Additionally, providing employees with the knowledge to utilize this information in a productive manner is key to success. Having the skills to perform one's job is self-explanatory, but, as experience has shown us, people often lack the proper skills required to perform their main job function. Having the right contacts as well as knowing who has the information when needed is equally important. 

Using the following variables to evaluate training programs (information, knowledge, skills, and contacts) have proven to be effective for years. However, expanding on these variables to improve the evaluation process follows the continuous improvement process. As an effort to expand on the accuracy of training I would pose adding critical thinking to the mix.

Training employees to think critically helps to eliminate issues such as functional fixedness and mental sets. Ollinger, Jones, and Knoblich (2008) termed mental set as: " the repeated application of a successful method makes blind any alternative approach, because of the mechanization of the particular solution method" (p. 270). Alternatively, Duncker (1945) identified functional fixedness as: "the tendency to fixate on the typical use of an object or one of its parts" (as cited in McCaffrey, 2012, p. 216).

Adding the dimension of critical thinking to training endeavors will help transform learners (trainees, employees) to effective learners. Brindley, Walti, and Blaschke (2009) identified effective learners as those who are capable of coping with "complexity, contradictions, and large quantities of information, who seek out various sources of knowledge" (p. 3). By seeking out new sources of knowledge employees will better avoid the aforementioned traps of mental sets and functional fixedness. 

Including critical thinking skills as part of the training program, as well as incorporating evaluation of employees critical thinking skills on-the-job, could prove to produce better training results and on-the-job performance results. Additionally, including critical thinking in both the instructional and evaluation phases of the training program could improve both the formative and summative evaluations of the overall program. 

References:

Brindley, Walti, & Blaschke (2009). Creating effective collaborative learning groups in an online environment. International Review of Research in Open and Distance Learning, 10(3), 1-18. 

McCaffrey, T. (2012). Innovation relies on the obscure: A key to overcoming the classic problem of functional fixedness. Psychological Science, 23(3), 215-218. dpi: 10.1177/0956797611429580

Ollinger, Jones, & Knoblich (2008). Investigating the effect of mental set in insight problem solving. Experimental Psychology, 55(4), 269-282. dpi: 10.1027/1618-3169.55.4.269

Phillips, & Phillips (2007). Show me the money: The use of ROI in performance improvement, part 1. Performance Improvement 46(9), 8-22. dpi: 10.1002/pfi.160