Making Results Assessment Practical and Credible - Training and Development

Does results assessment look hard to you? Look too difficult to complete? Swanson and Holton (1999) also offer some guidelines on how to make results assessments more practical and doable:

Deciding Which Interventions to Measure.It may surprise you to hear us say that it is not worth the effort to assess the results of all interventions. Assessing results takes resources—time, energy, and money. It makes sense to devote your resources only to assessing results that count. Assessment resources should be devoted to programs having a high potential of benefiting from assessment.

Consider these results quality criteria:
Is the intervention really being conducted for the purpose of improving performance?
If not, it probably means people don't care about results.
•Is the intervention forecasted to have significant and meaningful results?
If not, don't waste your measurement resources.
Are the costs of not getting results high enough to justify the effort?
If not, then sponsors may not care about your data.
•Are the people who care about the results important organizational partners?
If not, then what do you gain?
If the intervention has shown not to produce results, would it be eliminated or changed?
If not, then results must not really matter or there is as some other reason it has to be done.
Are the results really in question?
If you are sure it works, maybe you can postpone measuring results until you have looked at more questionable programs.
If you can't answer "yes" to all the above questions, then consider devoting your assessment resources to other interventions. Or ask yourself why you are doing the intervention in the first place.

More Than One Place to Start
The hard reality is that not every organization is ready to do everything at once.An organization that has never assessed any interventions may be lucky to implement knowledge assessment; another one that has conducted regular testing is ready to move ahead to performance assessment.

The most important thing is to start. It is okay to start where it is comfortable.
Butt's not okay to settle for just a low level of results assessment. If results are important, measurement systems should continue to improve until they address most of the measurement domains shown in this book and are providing the information for sound decisions. So start where comfortable, and then improve.

Use Existing Measurement Systems
It is always amazing to see the faces of HRD professionals when they realize how much results assessment information they need is already available in other organizational measurement systems. If the intervention is targeted at real performance- related needs that are important, the chances are very good that someone in the organization is already measuring it. Don't reinvent systems.

The closer assessment systems are to established organizational measurement systems,the more likely they are to last.
Don't Buy More Accuracy Than You Need
The purpose of results assessment is to enable the organization to make sound decisions about interventions. The level of accuracy needed for organizational decisions is considerably less than that needed for research. Your task is to find the"sweet spot" on the accuracy continuum so that you are buying enough accuracy to exceed management expectations slightly so that you earn maximum credibility.If you buy more accuracy than that, you are wasting resources.

For example, consider an organization development program that is likely to return the typical financial return of 800 percent in a year or less (Swanson 1998).Will it really matter to management if your measure is off by 50 percentage points—that is, you report 750 percent or 800 percent? Will you make any better decision about the program if you knew for sure the return was 800 percent? If the answer is yes, then pay for that degree of accuracy. If the answer is no,accept less. Often, giving up a modest amount of accuracy will save considerable resources.

Use Fast-Cycle Measurement Processes
Fast-cycle measurement is an iterative process that works in situations where interventions are repeated with some frequency. The idea is simple but powerful.
The first time you assess results, do so with a very wide confidence interval; that is, don't buy much accuracy. If the results are overwhelmingly good, then stop .Continuing the above example, if the initial measurement suggests a range of return from 500 percent to 1000 percent, that might be close enough. Many programs will never need measurement past the first step. Resources saved there can be devoted to more accurate measurement on others. This approach works particularly well in organizations that move too fast for complete assessment processes.

Practical Sampling
There is usually little reason to assess results every time an intervention is repeated. The basic principle of sampling is that unless conditions are substantially different, a measurement made on one group is a reasonable estimate of results for another group. So, for example, one class of manager sin a company is likely to be similar to another class if it is an open enrollment class. If so, assessing the results from one class may be enough.

There are many ways to sample, including:
periodic cycles (every other month offered)
subset of classes (one of every three classes offered this month)
subset of attendees (one of every four people involved)

It is up to the assessor to be sure that a sample is not biased. Be sure that the people assessed are approximately representative. Remember that when final statistics are reported, they will be used to make general statements, so make sure your sample is not likely to yield unusual or unrepresentative results.

Embed Results in Program Design
The more fully results assessment becomes integrated with the learning process, the more successful it will be. Unfortunately, the tendency is to treat it as a "research"project, often conducted by an outside expert, which takes place either parallel to or outside the learning process. It is not uncommon to see the data"disappear" with analysts, and the results shared with only a select group of people. When you integrate the results assessment, the process makes the training task easier and makes training more meaningful for the participants.

Create Stakeholder Ownership of Results
Assess mentis often something done "unto" participants by outside experts. The outcome is often increased resistance and participants who attack the data as invalid simply because they do not own it. Results assessments that work transfer much of the ownership of the results, and the results assessment, to the stakeholders—primarily the participants and their management. As long as results assessment "belongs" to the HRD department, stakeholders have little reason to take it seriously.

Here are some strategies to build stakeholder ownership of results assessment:

  1. Expect results to occur.
  2. Have stakeholders involved in determining the measures for performance, learning,and perceptions.
  3. Make participants accountable for successful performance and learning results.
  4. Make management responsible for collecting, monitoring, and reacting to performance results.
  5. Make sure all stakeholders receive reports of all results assessment data.
  6. Have stakeholders validate results data.
  7. Engage stakeholders is problem solving to correct out-of-conformity results.
  8. Have participants and managers conduct as much results assessment as is practical.
  9. Integrate results assessment into existing assessment systems, such as strategic planning or departmental performance reviews.

All rights reserved © 2018 Wisdom IT Services India Pvt. Ltd Protection Status

Training and Development Topics