Scales of Measurement - Training and Development

Skill in applying statistical methods is a great asset for the T&D manager who wishes to demonstrate beyond all doubt that T&D programs (not coincidence) have caused changes. Larger T&D departments employ such a statistical specialist. Smaller ones access similar experts employed in other parts of the organization. If the organization has nobody qualified to supply this statistical consulting service,outside help is usually available through local colleges and universities. However,a great deal of effective measurement can be achieved without professional help.Qite clearly, the first step is to determine what you're going to measure."Units of performance" is the inevitable answer in T&D measurements.

The Manager of Collections counted letters written and payments received. A shop operation might count and classify such things as defective units, accidents, and time lost due to accidents. Sales departments can count sales made, calls completed,volume of orders. In measuring management performance, there are such things as grievances, decisions, counseling sessions, turnover, budgets planned, budgets adhered to—and, above all, total production.

The next step is to select the scale to be used in the measurement. Nominal scales are often seen in T&D measurement. They offer a simple "classify-and-count"process. Nominal scales permit the measurer to classify "satisfactory"and "defective" units. Such a simple scale permits only a limited analysis—but those limits are broad enough to tell T&D specialists and client/managers what they want to know.

From nominal scales we legitimately arrive at totals, modes, and medians. Nobody needs a lesson in how to compute totals, but modes and medians are technical terms. Let's define them: The mode is, as the name implies, the most frequent occurrence; the median is the center number in a series. Thus if we measure the seniority of a group of trainees, our tally might look like this:

Example of a Simple Nominal Scale

The mode is quite clearly, "five years or more of service," because more people fall into that group than into any other group in the tally. The median seniority is "between three and four years." Why? Because that group is the midpoint;there are as many trainees with less seniority (ten) as there are with more seniority—also ten. That is, the same number of people is on each side of the"median" category. In nominal scales, when measuring the situation before and after a T&D effort,the mode and the median represent legitimate indices. An average, or mean, is not a valid index when using a nominal scale.

Ordinal scales are more useful in measuring such "invisible" elements as perceptions or values. Ordinal scales put items into a rank order. They tell who has more or less of something, or which values rank highest. For example, in determining training needs, T&D managers sometimes hand line managers a list of behavioral objectives with instructions to rank them from first to last in "importance for your subordinates."

Ordinal scales are extremely useful in post-training feedback. Graduates can indicate which of the acquired skills they value most, and so on down to where they designate "no value" in their on-the-job application. In such feedback, ordinal and nominal scales can be effectively combined. Graduates can also indicate how often they have used each of the new behaviors. This can be done in one of two ways: feedback forms that are tallied at short intervals, or perceptual to the best of your memory estimates.

In the feedback sheet, a typical form merely lists the objectives of the training. Graduates check off each time they use the behavior. Such forms are submitted to the immediate "boss" rather than to the T&D department. After the boss has analyzed the on-the-job application, two things happen. First comes positive reinforcement and counseling. Second, the sheets are forwarded to the T&Department, where organizational trends can be established and analyzed.

To review: The tallies represent nominal scales; the rank ordering is an ordinal scale activity. Ordinal scales also provide data, which may legitimately provide median sand modes but not means.

Interval scales are used on such measurement tools as yardsticks and thermometers. As the title implies, they measure distances between phenomena; they measure intervals. Accurate interval scales follow one important principle: The interval between checkpoints is always exactly equal to the interval between other checkpoints. If the distance between the 11 and the 12 on your ruler isn't precisely the same distance as the interval between the 10 and the 11, your ruler is faulty. One degree of temperature on the thermometer must be exactly the same as another degree. If it isn't, you may be sicker than you think!

Major problem with past T&D measurement has been the frequent use of interval scales on which the intervals were not defined—much less equal. The typical end-of-training form measures accurately when it asks for opinions; it is only pretending to measure legitimately when it asks for numbers on a scale posing as an interval scale—but it isn't. To compound the confusion, many T&D Managers take the data from non-equal interval scales and average it. Remember the hypothetical case we examined about the two sessions of the same course with the same average score!

When using legitimate interval scales, what legitimate mathematical functions may we properly perform? Modes, percentiles, and deviations are legitimate.

So are means, if the intervals are fixed and equal. Since that list includes two new terms, let's examine the concept of percentiles and deviations. Parents may have been told that their child is in the second quartile on a given achievement test. What does that mean? It means that of all the students taking the test, 50 percent were in lower quartiles—and 25 percent were in the higher or top quartile. Quartiles divide a population into four groups of equal size. Percentiles are more precise. If a child is in the99th percentile, 99 out of every 100 children earned a lower test score. Percentiles simply show a person's position in a population of 100.

More useful is the concept of deviation. It merely shows how far one unit deviates from the average unit. Deviations are not generally meaningful in themselves—only in comparison to other deviations. They cannot be calculated unless there is legitimate average, or mean, from which to operate.

Let' stake an example to illustrate their importance. Assume that you have just become the T&D manager and are looking over the ratings given two recent sessions of an introductory management program. The previous T&D manager used five-point scale to measure satisfaction with the course. You note that each session scored an average, or mean rating, of 3.15. This strikes you as unusual, so you compile a distribution tally of all the ratings for each session.Here is what your tally reveals:

Example of a Simple Nominal Scale

Why Frequencies Tell More Than Just Reporting the Mean.

One look at the raw data shows that the two sessions were remarkably different in perceived experiences—yet the averaging shows them to be identical.

In statistical terms, the standard deviation of session 1 is much larger than session 2, indicating wider disparity in opinion.

Finally,there are ratio scales. They can tell variability from an absolute condition.

This implies the necessity of an absolute zero—condition we don't see in developmental operations. Who can conceive of person with zero intelligence, or zero achievement? However, unless one disusing a legitimate ratio scale, with an absolute zero, it is really improper to make evaluations such as "Eve did twice as well on that as anyone else," or "Mary is picking this up beautifully. She did three times better today than she did yesterday!"


All rights reserved © 2018 Wisdom IT Services India Pvt. Ltd DMCA.com Protection Status

Training and Development Topics