To What Extent Is Program Evaluation Conducted in Practice? - Training and Development

One stream of research in training evaluation has been to document the extent to which methods are used in practice. This research is important because it shows to what extent prescriptive models and methods are actually used in practice, which in turn should inform the development of new models and methods. Despite efforts to build new evaluation models, most surveys of evaluation practices use the Kirkpatrick framework because it is the most widely recognized. The data is too voluminous to include here (interested readers should see Swanson and Holton 2001; and Twitchell, Holton, and Trott 2000 for a review of surveys). However, the overall conclusions from these surveys are

  1. many organizations use Levels 1 and 2 evaluation for at least some programs;
  2. fewer than half the organizations even try Level 4; and
  3. only a small percentage of programs receive Level 3 and 4 evaluation. Overall, these findings present a very disappointing view of evaluation practices.

Furthermore, when following the literature back to the development of the four levels forty years ago, there seems to have been little change in the amount of evaluation conducted within business and industry. As early as 1953 (Wallace and Twitchell 1953), researchers were discussing the lack of training evaluation and the need for it. Today's literature contains parallel comments and the lack of training evaluation still exists. The trend they suggest is striking: Only modest gains have been made in the number of organizations using these evaluation practices. More specifically, it would appear that modest gains were made in Levels 1 and 2 from 1968 to 1989, but little gain in the last twelve years. Even more troubling is that very little gain has occurred at Levels 3 and 4, which are the most important levels for demonstrating training's effect on organizational performance.

The main reason training is not evaluated is usually the same for all levels— it is not required by the organization. The usual interpretation of this is that it continues to raise serious questions about whether training is valued by organizations as a core business process.

The second most important reason for not doing Levels 3 and 4 is usually lack of time, with lack of training close behind. Evaluation continues to be seen harder to accomplish than it should be.

If forty years of promoting evaluation has not changed the overall picture, something else must be needed. Clearly the Kirkpatrick model of training evaluation has not been effective in making evaluation an integral part of Human Resource Development (HRD) practice. It does not matter if you examine practitioners more highly trained in evaluation (instructional designers), those that have more well-defined outcomes (technical trainers, health care), those in more sophisticated organizations (American Society for Training and Development [ASTD] benchmarking group), or simply average training practitioners, the level of evaluation use is about the same and is not changing much.

All rights reserved © 2018 Wisdom IT Services India Pvt. Ltd Protection Status

Training and Development Topics