The Core Dimensions - Training and Development

Decision1: Which of the three result domains and six results options will be measured?

It is not always appropriate or necessary to choose all three domains or all six options. For example, when we were consulting with two separate divisions in a large corporation that were located in two states, it was just as if we were working with two different organizations. In one division, HRD had established itself as a major process and we could almost automatically think of all six results options (A-F) as being appropriate: system performance, financial performance,knowledge outcomes, expertise outcomes, participant perceptions, and stakeholder perceptions.

In contrast, the HRD department in the other division had never established itself as a major business process. For example, it never engaged in fundamental performance analysis (the discovery of performance requirements and solutions).Worse yet, the HRD department didn't even have a track record of assessing learning results from any of its programs. As a result, the idea of promising performance results was administratively outside the reach of the HRD workgroup. Their challenge was first to gain credibility through assessing and reporting the knowledge and expertise results (options C and D) from their programs and then grow into performance.

Decision2: When will results data be collected (before, during, and after the intervention)?

Data pertaining to the program, or intervention, will recollected a data collection timeline and recorded in column 2 of the worksheet.Each result option selected can potentially be measured before, during, and after the intervention. For example, it may be possible to assess system performance outcomes before,during, and after an intervention. In addition, it may be desirable to measure a result option several times before, during, and after the intervention to find an average score or to plot a trend.

As you can see by the Results Assessment Plan worksheet, this is represented by two intervals of time in the before stage (timeline boxes 1-2), two in the during stage (timeline boxes 3-4), and two in the after stage(timeline boxes 5-6). When the T&D officer or client/managers wish to prove absolute causality between the training and the improvement, they need to know that the differences between "pre" and "post" are statistically significant. For this level of analysis, a professional statistician should be involved. Only such technicians can answer the central question, "Are these changes well beyond coincidence?"

When profound numerical analysis is necessary, the statistical consultant should be involved in the process from the very beginning. It is dangerous, maybe fatal,to wait until the data have been gathered. By then the T&D director may already have "booted" it in one or more of several ways:

  • Necessary retraining data are missing.
  • The instruments asked the wrong questions.
  • There was contamination in the measurement process.
  • Key indicators are missing.

Decision3: Will the results be compared to another cycle, a standard, or a norm?

All results measures need to be compared to something to judge the quality of outcomes achieved. Comparison Options are shown in column three with boxes 7, 8, and 9.

  • Box 7 refers to another cycle of the same program (e.g., the May group in Executive Development compared to the August group in Executive Development).This could also be a control group that did not participate in the program.
  • Box 8 refers to a standard—an attainment level is set that has previously verified meaning. For example, having a 90 percent or higher on-time arrival of flights, a sales standard of 4 million dollars per month for the sales region, an average participant satisfaction rating that is positive. These standards should not be arbitrary. They should both be referenced to existing data and information.
  • Box 9 is a norm. A norm comes from a large database that provides averages used as a point of comparison. For example, industry averages or benchmarks might be used as a norm. At some point norms, if confirmed, can work their way into being standards.

One mechanism upon which statistical experts may insist is a "control group."Even if you don't have a statistician insisting upon such a control group, you have important reasons for establishing one. This simply means that a part of the "defective population" will not immediately receive training, or any other form of change program. They become the control group; those who are in the training are the "experimental group." The reasons for control groups should be obvious. If the change takes place only in the experimental group, there is some reason to credit the change program for the improvement.If, on the other hand, both the control group and the experimental group move in the same direction, evaluators must question the causal effect of the program. If they move in the same direction at the same speed, better drop the program. In situations like these, the professional statistician is a valuable resource for the evaluation process—valuable but not absolutely indispensable.

Decision4: How will the data be analyzed?

Just what data will be collected, compared,and presented as part of the final evaluation report for a specific intervention?The Data Analysis Plan allows planners to state what data will be compared for assessing the results for each of the six selected results options.

An example here is participant perceptions at the end of program (Participant 4)compared to a standard of 2.5 or higher on a 1-4 scale (Participant 8).
The data analysis plan is a comparison of the two: Participant 4Participant8. Here are two other examples:

  • average expertise before attending program compared to expertise after participating in the program (Expertise 2 Expertise 4); and
  • financial return-on-investment of staff-managed performance improvement efforts compared to external consultant managed efforts

(Financial6 Financial 7).

Decision5: What other information is needed to execute the assessment plan?

This fifth column records details about two critical areas of results assessment:

  1. measures to be used, and
  2. implementation details.

Implementation details might include any specifics about the actual measurement steps andtheir timing. For example, they might include how the measures will becollected, the instruments distributed and collected, and the timing of the datacollection. The general timing on the data collection timeline (Part 2) will needto made more specific, such as the day before the intervention, the last hour of the intervention, or sixty days following the intervention, and so forth.

All rights reserved © 2018 Wisdom IT Services India Pvt. Ltd Protection Status

Training and Development Topics